Ephemeral environments and seed data
Give an agent a runnable test environment with isolated state and invented data. Learn what to preserve, what to clean up and which startup costs to measure.
An ephemeral test environment is created for a task and removed afterwards. It gives the agent the services and starting data needed to run meaningful checks, without interfering with another task or using real customer records as disposable test data.
To test an upload fix, the agent may need an application, a database and an object store. Start them with a known set of invented files and users. After the checks, save the changed code and results, then remove the temporary services and their data.
Cloning the repository may be enough to inspect code. It is not enough to run an integration test that needs a database. Decide what a task must check, then provide the smallest useful part of the application and its dependencies.
Provide the services the checks actually need#
An ephemeral environment exists for a limited period. For a coding task, it can start from a known toolchain image and code revision, create the required services and load a defined set of test data. It is then removed after the work is saved.
Choose the dependencies from the checks. A unit test may need only the language runtime. An upload integration test may need a database and object store. A message-processing test may also need a broker and a controlled substitute for an external service.
Testcontainers for Java is one example of tooling for temporary databases and other containerised dependencies. It helps start the services. The team still has to supply the schema, fixtures and assertions that make the test meaningful.
A running worker is not a ready environment
The Software Factory Design Patterns episode separates the machine that runs an agent from the environment needed to build and inspect its work. For a web application, that can include private service access and a reviewable preview, not just a language runtime. Treat this as a question about your checks, not a requirement to reproduce a large company’s entire development platform.
Give the upload task a short, repeatable setup check: identify the code revision and toolchain, start the required services, apply the expected schema, load the fixture version and confirm that the test identity can reach the permitted test endpoints. Save the result before running the application checks. Call this a preflight: it establishes specific prerequisites, not application correctness or complete security.
Try it: change what is ready for the upload test
- The worker process starts and the repository is present.
- The required test database cannot be reached.
- Record the setup failure and restore the permitted dependency before running the test.
Changing the upload comparison cannot repair missing connectivity. Do not remove the integration check to obtain a green result.
Illustrative upload-test states, not a real preflight or security assessment. No service is contacted. The initial blocked case remains readable without JavaScript.
When preflight fails, record which prerequisite is missing and who can restore it. Do not ask the coding agent to change upload behaviour, disable a test or loosen network access simply to continue. A repair to the environment may justify rerunning preflight; it is not a successful bug fix.
For browser checks, record which revision a preview serves, who can access it and when it expires. A reachable URL showing yesterday’s code cannot support today’s interface review. Keep previews private unless public access is deliberately approved. A preview lets a person inspect the result; it does not replace the recorded checks for that revision.
A stub can reproduce the responses your test defines. It does not establish that a real external provider behaves the same way. Record which dependencies were real, simulated or unavailable, and use appropriate contract or sandbox checks for the remaining questions.
Supply limited credentials at runtime, not inside the image or source tree. Restrict access to hosts and services that the task needs. A container is a packaging and isolation mechanism, not a complete security boundary by itself; host mounts and powerful runtime access can weaken it.
Keep the useful outputs outside disposable storage: the diff, relevant logs, check results, code revision and unresolved questions. Destroying the worker should not destroy the only copy of unfinished work.
Keep one run’s data changes away from another#
Imagine two agents testing an export against the same writable records. One deletes an order as part of its test while the other counts exported orders. The second result now depends on the first test’s timing, not just on the export implementation.
A separate database per run is a straightforward way to avoid that interference. It is not the only one. Separate schemas, transactions or namespaces may be sufficient when the application and tests respect them. They may be insufficient for migrations, global settings, database-level permissions or shared background workers.
Choose isolation based on what the test can change, and verify it. Apply the same reasoning to queues, object storage, filesystem paths and external test accounts. A private database does not prevent two runs from consuming the same queue messages.
The lifecycle shown is illustrative. Create the environment, wait for required services to become ready, load the starting data, run the task and save its outputs. Then remove services, volumes and other temporary resources that the run created. Check cleanup after cancellation and worker crashes as well as after success.
Measure startup, execution, waiting and cleanup alongside model costs. Include abandoned environments and failed attempts. Cost per merged change must not count only the successful agent call.
Choose the worker lifetime separately from task isolation
The episode contrasts long-lived, individually maintained workers with disposable, automatically provisioned ones, often called pets versus cattle. A prepared worker may be a practical starting point for a small queue or specialised toolchain. Disposable workers can simplify replacement and concurrent allocation, but only when provisioning, access and cleanup are reliable. Neither label guarantees isolation.
The host can remain alive while each task gets its own checkout and isolated writable test state. Conversely, a newly created worker can still interfere with other runs through a shared database or queue. Check the state boundary, not only whether the machine is new. Record how a worker is reset, how concurrent tasks are separated and what happens when reset fails.
Optimisations have conditions. A shallow clone omits history that an investigation might need. A schema snapshot speeds startup but does not test the migrations that produced it. A pool of prepared environments can save waiting while preserving isolation if each is clean and allocated safely. Reusing a dirty worker without a reliable reset is a different choice.
Use invented data that exercises real cases#
Seed data gives a test a known starting state. For routine agent runs, prefer deliberately invented records that meet the schema and exercise the required behaviour. Keep the generator or fixture files versioned with the code.
Do not assume that a production copy becomes anonymous after names are replaced. The GDPR distinguishes pseudonymisation from anonymisation, including the role of additional information that can identify a person. The exact assessment depends on the data and circumstances; a temporary environment does not provide an exemption.
The word “synthetic” is not a guarantee either. A generator based on private examples can reproduce details from them. Check what the fixtures contain and how they were produced. Even entirely invented data can reveal confidential business rules, so decide where it may be stored or sent.
For the upload example, include an empty file, files below and at the limit, a file above it and users with different permissions. Include an unusual filename where relevant. For an export, test missing optional fields and enough rows to exercise pagination. Use a separate volume fixture when checking memory use or timing.
Maintain the fixtures as behaviour changes. A new column populated by a database default may let old fixtures load without exercising the new rule. Review what the tests now cover, not only whether seeding succeeded.
When a defect escapes, recreate its relevant conditions with a small safe example and add a regression check. Avoid making a permanent production-data copy the shortcut for every later investigation. Private data inside an agent run also increases the consequences of an unwanted outbound action.
The result should be a task another worker can reproduce from its recorded starting point. Each disposable run also needs its own connection strings and credentials; the Twelve-Factor config rule keeps those deploy-specific values out of the image. The next guide asks whether managing these tasks needs a separate orchestrator or the pipeline already in use.
IF YOU REMEMBER ONE THING
Start from known code and test data, isolate the state a task can change, and save its work before cleanup. Choose the environment for the checks you need—not simply to provide a container.
Questions people also ask
5 QUESTIONSMust every task have a separate database process?
No. A documentation task may need no database, and some test suites can isolate state through schemas, transactions or namespaces. A separate database instance is useful when migrations, permissions or global state make weaker isolation unreliable. Test the property you need rather than choosing one design for every task.
Can a shared test environment work?
Yes, for some controlled checks. It needs a clear policy for concurrent writes and resetting state. When two tasks can alter each other's inputs, a passing or failing test becomes harder to attribute to the code. Isolate the relevant state or deliberately coordinate access rather than treating interference as random flakiness.
Is replacing names in production data enough?
No. Pseudonymisation is not automatically anonymisation; other fields or separately held information may still identify people. Prefer invented fixtures for routine agent tests. Any use of real data needs an appropriate privacy and security assessment, not just a disposable-container label.
How much seed data is enough?
Use small, understandable fixtures for behaviour and separate larger data sets for volume-sensitive checks. Include relevant missing fields, boundary values, permission differences and unusual characters. More rows do not help when all of them exercise the same easy path.
Does the environment cost more than the model?
It can, but measure it rather than assume. Include setup, execution, idle time, storage and cleanup, as well as failed runs. A cheaper model does not remove the cost of a slow test environment.