The coding agent and its repair loop
Follow a coding agent from a ready task to a tested change, including limited repair attempts, scope changes, blocked tests and a handover to a person.
A coding agent reads a prepared task, changes the code and uses test results to correct mistakes. Its workflow needs limits on what it may change, how long it may keep trying and when it must return unfinished work to a person.
Imagine an agent adding a missing CSV column. A test shows the column is in the wrong position, so it corrects the export and runs the test again. If the expected position was never decided, another coding attempt will not help; the agent needs to ask.
The builder is the agent that implements an agreed task. Writing code is only part of its run: it also needs the right starting version, a working test environment and a way to return results. When something goes wrong, the workflow must decide whether another edit is useful.
Follow one task from checkout to proposed change#
In this example workflow, the worker first claims a ready task. It prepares an isolated workspace at a known code revision, reads the requirement and relevant project instructions, and inspects the code around the suggested files. It then works on one testable part at a time, adding or updating appropriate tests and running checks as it goes.
The claim matters when tasks can start concurrently. Reading “unassigned” and then setting an assignee may let two workers both start. Use a conditional claim that only one can win, provided by a queue, workflow store or API with the necessary guarantee. A separate custom database is not mandatory.
A worker that can crash while owning a task may need an expiring claim, called a lease. Expiry permits another worker to take over, but it must also prevent the old worker from publishing after it loses ownership. Saved handovers and workflow state explains how the replacement finds the current task and results.
The figure shows a limited workflow with two useful outcomes: a proposed change for review, or unfinished work with an explanation for a person. Its retry count is an example setting. It is not a claim that all coding agents should use the same number.
A reviewable result includes the changed code, its revision, the test commands and their actual results. Add screenshots when they show relevant behaviour, and name remaining risks or manual steps. A self-review note can help direct the human review, but it does not replace the reviewer’s inspection.
The workflow creates a branch or pull request only when that publication is allowed. Some checks run only after a pull request exists; wait for them before calling the change ready. A proposed change is not a merged or deployed change.
Get feedback before the change grows#
Do not wait for a large generated implementation before finding out whether its first behaviour works. Matt Pocock’s test-driven development guidance recommends one failing behaviour test followed by the code needed to pass it, rather than batches of imagined tests followed by batches of implementation. Test-driven development is usually shortened to TDD.
For the illustrative CSV change, suppose the agreed header is name,status. Write a check against that exact header and run it before the edit. Confirm that it fails because the column is missing, not because the test environment cannot start. Add the column, rerun the check and then exercise a row’s status value. Each result guides the next small change.
Choose expected values independently of the implementation. If both the exporter and the test obtain the expected header by calling the same new column-order helper, the same ordering mistake can appear on both sides. Use the agreed format or a worked example instead, and inspect tests as code that can also be wrong.
The runnable AI-generated-tests exercise makes this inspection concrete: two deadline checks pass with incorrect code, while one independently chosen boundary expectation catches it. Inspect what an assertion distinguishes, not just whether the agent added tests.
For this workflow, keep design cleanup explicit: after a behaviour works, simplify the implementation where needed and rerun the checks. The familiar red–green–refactor cycle means failing test, working behaviour, then cleanup without changing that behaviour. A team may put cleanup in a separate review step; the point is not to let it disappear behind an always-growing pile of passing code.
Before cleanup of existing behaviour, capture selected observations in characterization tests. The booking example records a repeated cancellation before a refactor and detects an unrequested exception afterwards. Regenerating expected results from the rewritten code would hide the change the test was meant to expose.
Choose feedback that can detect the actual mistake. A type check cannot establish the agreed column order; a serializer test cannot establish which records the user may export. Run the relevant integration and end-to-end checks before handover too. For an unsettled interaction or a configuration-only change, a prototype or a targeted smoke check may answer more than a forced unit test. Planning a verifiable slice starts with that question.
Retry a known mistake, with a limit#
Suppose a test shows that the new CSV column appears in the wrong position. Return that assertion and the relevant output to the agent. It can inspect the export, make a focused correction and rerun the affected checks. This is the repair loop: use a result from the previous attempt to choose the next action.
Another attempt is not automatically better informed. Check that the failure reached the next model call and that the response addresses it. Repeating the same command without changing a relevant condition may only repeat the same failure.
Set an attempt limit before the run starts. Three total attempts is one possible policy, not a general recommendation. Define whether the initial implementation counts, and enforce time and spending limits across the whole task. Starting a new session must not silently reset those counters.
Stop earlier when another edit cannot solve the problem. An unavailable database means a required check is blocked. A missing product decision needs a question. A test that already failed on the unchanged starting version may be unrelated; verify that before modifying the code to satisfy it.
Anthropic’s long-running-agent report describes incremental work, saved progress and checks of inherited state. These mechanisms help organise attempts. They do not establish the right retry budget for your project.
Enforce important restrictions outside the model: permitted repositories and operations, protected branches and the ability to stop a run. Incoming issue text and code comments remain untrusted even when copied into a structured record. Formatting them as JSON does not remove prompt-injection risks.
Handle scope changes, weakened tests and interrupted work#
The change grows beyond its permission. A fix may reveal a dependency in another service. The agent should report why that extra change is needed and request a decision before crossing an enforced boundary. Do not treat every newly discovered file as an emergency; a list of suggested files and an access restriction are different things.
A failing assertion is removed. The build then passes without the original behaviour being repaired. Flag removed, skipped or weakened tests for independent review and preserve important acceptance checks outside the builder’s control. Some tests legitimately change when requirements change, so inspect the reason rather than banning all test edits.
The worker stops after a partial change. Save the diff, commands, errors and external actions. Before restarting, check whether a push or pull-request creation actually completed. A timeout does not prove that nothing happened, and taking over the lease does not undo the previous worker’s edits.
The code implements the wrong interpretation. Tests generated from the same assumption can still pass. Compare the result to the agreed request, not just to the newly written tests. Stopping to ask a specific question is the correct response when that agreement is missing.
Review these failures when evaluating the builder. Anthropic’s evaluation guide describes testing agent behaviour across tasks and outcomes. Include unsuccessful runs, review time and misleading success reports. A generic self-review note repeated on every task contributes little unless someone checks whether it identifies real risks.
IF YOU REMEMBER ONE THING
Check one behaviour before the change grows. Use an actual failure to guide another attempt. When the task needs a decision or exceeds its limits, save the work and explain what must happen next.
Questions people also ask
5 QUESTIONSHow many times should an AI coding agent retry a failed build?
Choose a limit for the task and measure whether the retries help. Three attempts is an example policy in this guide, not a proven universal optimum. Also limit elapsed time and spending. Stop earlier when the agent lacks a requirement, permission or working environment.
Should an agent generate every test before writing the implementation?
Not as a default. Agree on the important behaviours first, then implement and check one at a time. Expected results should come from an approved rule or an independently worked example, not from running the same new implementation to create both sides of an assertion.
How do you stop two agents picking up the same work item?
Use a claim operation that can succeed for only one worker at a time. Do not assume that reading and then changing an assignee provides that guarantee. For work that survives process failures, define how ownership expires and ensure a replaced worker cannot keep publishing changes.
What should an agent do when it needs to change something outside the approved scope?
Explain the dependency and request the necessary permission before making the extra change. Distinguish helpful file hints from enforced restrictions: discovering another file in the approved component is not always a new task, but crossing a prohibited service or operation boundary needs a decision.
What should we measure about a coding agent?
Measure time to a reviewable change, first-pass test results, repair attempts and the review and correction work left to people. Include failed and abandoned tasks when counting costs. Lines of generated code do not tell you whether the requested behaviour works.