All posts
Best Practices4 August 2026 10 min read

Running QA Agents On-Prem: Data Boundaries for Regulated Teams

Exactly what an on-prem test runner should send back, what must stay inside your network, and how to prove the boundary holds during an audit.

By The Assertive Team

Regulated teams rarely object to autonomous testing in principle. They object to a vendor's browser opening a session against production data and shipping the DOM somewhere they cannot see. The answer isn't a policy document — it's a boundary you can inspect.

Split generation from execution

Scenario generation and execution are separate concerns with different data appetites. Generation needs structure: routes, roles, accessible names. Execution needs the real environment: credentials, records, side effects. An on-prem Runner lets execution stay inside the perimeter while the governed artifact — the scenario set — remains reviewable in the platform.

What leaves the network

Status, timings, token counts, step outcomes, heal events, and sanitised error text. That is the payload the Runner posts back, and it is enough to render a full report. Page HTML, response bodies, cookies, vault plaintext, and screenshots you have not opted into never leave the host.

What never leaves

Persona passwords and TOTP seeds are decrypted only inside the run sandbox. Codes are derived at run time, masked in logs, and discarded with the sandbox. Internal grounding — the structural map of pages behind your auth wall — is stored against the project but sanitised of values before it is transmitted.

Verify the binary, not the promise

Releases are signed with Ed25519 and the daemon verifies the signature before applying an auto-update. Pin a version if your change-control process requires it. The outbound sanitiser is part of the open runner package, so your security team can read the exact filter rather than trusting a claim about it.

Prove it during an audit

Every check-in, scenario allocation and result delivery is recorded in a per-project runner activity log with the instance key and token that performed it. Pair that with sealed run metrics and you can reconstruct who ran what, from where, against which build.

Frequently asked questions

Does the on-prem Runner need inbound network access?
No. It polls outbound over HTTPS for work and posts sanitised results back. No inbound port needs to be opened, which is usually what makes it acceptable inside a restricted network.
Can screenshots or traces be disabled entirely?
Yes. Artifact capture is opt-in per project. With it off, the report shows status, timings, per-step telemetry and heal events without any visual capture leaving the host.
Where are credentials stored when running on-prem?
In the encrypted vault, referenced by token in scenario steps. The Runner resolves them inside the run sandbox; plaintext is never written to disk, logs, or the results payload.

Keep reading