AI coding and the human attention bottleneck
Keep AI-generated changes understandable: decide permissions before coding, inspect code and tests, and stop starting work when reviewers cannot keep up.
AI-generated code still needs people to decide what it should do and check whether it does it. Review important choices before coding, keep the relevant code and test results easy to find, and limit new work when reviewers cannot keep up.
Suppose an agent adds a file download. A person first decides who may use it, then checks that the code refuses everyone else. Approving the rule before coding helps, but does not prove that the finished download enforces it.
Continue the illustrative download example. A user requests an export, then loses permission before the file is ready. The team must decide whether that user may still download it. An agent can produce a plausible implementation either way; faster coding does not settle which behaviour is wanted.
A short specification can expose that choice before implementation. It might take two pages to explain a change that becomes 2,000 lines of code, but those sizes are illustrative, not a safe ratio. Reviewing the requirement does not make the resulting implementation safe to skim.
A software factory connects planning, coding, checks and delivery. When generation outruns the team’s ability to assess changes, the process must also account for the work people still do. Review is not every team’s bottleneck: unclear requirements, integration or deployment can be the limiting step instead.
Remove the work of reconstructing the requirement#
In the export example, understanding which records belong to a workspace and who may download them is necessary work. Searching five chat threads to discover which permission rule was approved is avoidable work. Put the rule beside the relevant design and code so the reviewer can assess it without first reconstructing the decision.
Research on cognitive load gives names to that distinction. Cognitive load is the demand a task places on a person’s limited capacity to process information. In Sweller, van Merriënboer and Paas’s account, intrinsic load depends on the material’s complexity and the learner’s existing knowledge. Extraneous load comes from how the material and task are presented. The theory concerns learning; applying it to software review is an engineering proposal, not a finding that this review format has been validated.
Shorter text is not automatically easier. “Use the standard secure export flow” is brief but makes the reader reconstruct everything it leaves unstated. A longer example showing an allowed request and a denied request may require less reconstruction. The aim is to remove unnecessary effort, not the reasoning needed to notice a mistake.
Try it: what has the team actually checked?
For the illustrative export, choose one review artifact. Each answer supports a narrower claim than “the whole feature is safe.”
Nor does memory research supply a maximum agent count. Cowan’s work on short-term memory capacity describes limits under particular experimental conditions, with information grouped into chunks. A chunk is not a page, a pull request or an agent. “Four items in memory” cannot be converted into “four agents per developer”.
What a two-page specification can do#
Make the illustrative request concrete: “Let workspace administrators export project records as CSV.” An agent can find the existing permission model, list unresolved cases and draft an approach. Before implementation, the reviewer needs to decide what happens when a person’s access changes between requesting the export and downloading it.
Suppose the chosen policy requires current permission at download time. The short specification should state that choice explicitly, explain its consequence for background jobs and identify the existing access-control code. It should also distinguish observations from proposals. “This endpoint checks membership” describes the current system. “Downloads must recheck membership” is a requirement that needs approval.
Implementation detail can stay behind links while the consequential choices remain visible. Shortening the document becomes harmful when it removes uncertainty. A sentence such as “Permissions follow existing behaviour” must not conceal that nobody checked how generated files are served.
There is no two-page safety threshold in the research cited here. Use a short overview when it helps a reader make a decision, then provide the necessary detail. When the relationships no longer fit into a reviewable change, reduce the scope or schedule a deeper design review. Do not shrink the explanation until the risks disappear from view.
A useful design document exposes choices while they can still change. After implementation, keeping the specification aligned with the intended behaviour is a separate maintenance responsibility.
Give each review a question it can answer#
For this workflow, give each review a question that can actually be answered at that stage. These are different views of the work, not a requirement for a separate document, meeting or agent at every step.
At the request level, decide the outcome. Who needs the export, which records belong in it, and what is deliberately excluded? The person responsible for the product answers unresolved policy questions. An agent can suggest alternatives without treating its preferred answer as permission.
At the design level, assess the mechanism. Where does the download permission check run? Can a stored file be reached through another route? What happens after access is revoked? A reviewer can reject the approach before it spreads through controllers, background jobs and tests.
At the implementation level, inspect what changed. Follow the relevant request through the code and verify that it reaches the intended check. Approval of the design cannot reveal an implementation that accidentally bypasses it. Examine tests for the failure cases, not just the successful export.
At release, check the actual result and recovery options. Confirm which revision is running and exercise the important behaviour in that environment. Reverting the application can stop a faulty export route, but it cannot retract records already downloaded. Recovery limits belong in the acceptance decision.
Between these decisions, agents can search, implement and run permitted checks. If a new dependency invalidates the approved design, pause the affected work and bring back the changed assumption. Do not force the implementation through an outdated plan, and do not restart every approval for an unrelated naming correction.
Put the requirement beside its code and checks#
An agent-generated review note should help someone find the requirement, its implementation and the check that could expose a mistake. “All checks passed” leaves too much unanswered: which checks, against which revision, and which requirement did they exercise?
Illustrative review note — evidence to request
Decision: A download requires the requester’s current permission for the workspace.
Inspect: The download route and the code that reads the stored export. Check whether another route bypasses permission enforcement.
Exercise: An authorised download, a cross-workspace attempt and a download after access is revoked. Attach actual results for the submitted revision.
Unverified: State any storage configuration or deployment route that the test environment did not cover. Missing evidence is not a pass.
This note describes requested evidence, not tests that were run for this article. In a real pull request, each reference should lead to the relevant diff, test or execution result. Show the important uncertainty in the overview; do not hide it several links deep.
Keep changes coherent enough for that inspection. Google’s engineering guidance recommends small, self-contained changes, including their related tests. It explicitly treats size as more than line count. That is engineering guidance, not a controlled experiment establishing a universal limit.
For the export, avoid mixing the new behaviour with an unrelated database refactor. But do not split the permission rule into disconnected fragments merely to meet a line quota. Review each slice in context and check the assembled feature too: individually plausible pieces can still disagree about where a boundary is enforced.
Why another agent is not proof#
Automation bias is a reason to question effortless agreement with a machine’s recommendation. Parasuraman and Manzey’s review describes both following incorrect advice and overlooking problems when an automated aid supplies no warning. It also connects complacency with competing demands for attention. These effects are not confined to novices, and simple instructions to be careful are not a sufficient safeguard.
Applied to this workflow, a green check and a fluent explanation should not decide which code the human never looks at. Ask which observation would make the proposed change unacceptable. For the export, a successful download after permission revocation contradicts the policy even if the happy-path tests pass.
The planner, coder and reviewer can share the same mistaken assumption. If all three infer that permission at export-request time is enough, their agreement does not establish the requirement. Give review a separately approved contract and tests that challenge the implementation. Another model may provide a useful second view, but a different role name does not make the evidence independent.
The larger research question is scalable oversight: how to supervise systems whose work exceeds what people can assess unaided. Bowman and colleagues studied AI-assisted human evaluation on difficult question-answering tasks. Their results support investigating assistance, not a guarantee that an AI reviewer can certify a large codebase.
Automated tests, static analysis and narrow formal checks can reduce what needs repeated manual inspection. Their coverage and assumptions still matter. A property checked against the wrong permission rule remains the wrong target. Neither a passing test suite nor a collection of model opinions resolves whether the product should behave that way.
Stop starting work when review cannot keep up#
Good review notes cannot solve a permanently overloaded queue. If work arrives faster than the team can assess and finish it, unfinished work accumulates. Adding a summarisation agent only helps that queue if it actually increases how much work the team can finish.
Use a limit on work in progress. When review is saturated, stop starting unrelated changes and help finish existing ones. Group routine questions by task, preserve the context needed to resume, and interrupt immediately only when the decision cannot safely wait. The right limit comes from the team’s observed workload, not a universal number of agents per person.
Interruptions can have a cost even when a speed metric improves. In Mark, Gudith and Klocke’s experiment on interrupted work, participants compensated by working faster but reported greater stress, frustration, time pressure and effort. This was not an AI code-review study and does not establish a fixed recovery time for every interruption.
The design goal should therefore not be to keep a reviewer at maximum mental effort all day. Leave capacity for unexpected failures, learning and a decision that turns out to be harder than it looked. A backlog that only stays small through rushed approvals is not under control.
Keep the ability to intervene#
Supervising automation also requires knowing what it is doing well enough to take over. In Endsley and Kiris’s study of a navigation task, reduced situation awareness was associated with worse takeover decision times after an automated system failed. This is the out-of-the-loop performance problem. It comes from a different domain, but it raises a useful software question: can the person asked to intervene explain the current state?
For an agent run, show the active objective, the accepted decisions, the current code revision and the unresolved failure. A stream of tool logs is not a substitute for that overview. Keep the detail available so the operator can reconstruct what happened without reading the entire conversation.
There is also a longer-term issue: how the team acquires the skills needed for future reviews. In Shen and Tamkin’s January 2026 randomised study, 52 Python programmers learned an unfamiliar library. Only four of the 52 participants reported 1–3 years of coding experience; 29 reported seven years or more. The AI-assisted group scored 4.15 points lower on the 27-point immediate quiz (p = 0.01). That raw gap is 15.4 percentage points of the available score; the paper separately calls it a “17% score difference”. Task time did not improve significantly on average.
That experiment concerns learning a new library, not years of professional skill loss. Its observed associations between particular prompting styles and learning were not causal tests of those styles. The detailed study review separates the randomised AI-access result from those exploratory usage patterns. It is a reason to measure understanding separately from task completion, not proof that AI necessarily makes developers less capable.
Margaret-Anne Storey uses cognitive debt to describe a growing gap between an evolving system and the team’s shared understanding of it. That is a useful emerging description, not a standardised measurement. More documentation alone need not close the gap if nobody uses it to reason about the system.
A practical safeguard is to retain some hands-on diagnosis and walkthroughs. Ask an engineer to trace an important request, predict a failure and explain why the test would catch it. Let AI help with explanations, then check the explanation against the code. Treat this as maintaining review capability, not as an extra approval checkbox.
Measure the whole job#
A useful question is whether comparable, valuable changes take less total human effort while remaining understandable and reliable. Count specification review, agent supervision, code review and subsequent repairs. Keep elapsed waiting time separate from active effort. Shortening the final review is not a saving if the same work reappears during an incident.
Track that effort beside escaped defects, rework and recovery events. Inspect review-queue age and ask reviewers about workload. Periodically check whether someone other than the author can explain a changed boundary. No single number captures all of this, and lines of code or pull-request counts are especially easy to increase without delivering a better outcome.
Perceived speed is not enough either. METR’s early-2025 randomised study found that 16 experienced maintainers took 19% longer with the tested AI tools across 246 tasks, despite believing they had been faster. That was a particular population, workflow and generation of tools, not an enduring estimate for AI coding.
In its February 2026 follow-up, METR warned that participation and task-selection effects, together with difficulties measuring concurrent agent work, made its newer productivity estimate unreliable. The lesson for this workflow is to measure the actual work carefully, rather than select an old study or a satisfying demo as a universal answer.
To test the proposal, compare similar changes under the existing process and a process with an early decision review plus an inspectable evidence note. Where feasible, randomise assignment and account for task difficulty and developer experience. Record all human effort and use the same follow-up period for defects. This is a proposed experiment; the article does not report a trial of the workflow.
IF YOU REMEMBER ONE THING
Use automation to reduce the work around a decision, while preserving the information and skill needed to make it. A shorter review is useful only when it leaves the team able to explain what it accepted.
The next practical question is how to make the human merge decision meaningful: what must be inspected, which risks need deeper review and when insufficient understanding should stop the change. For the reviewer’s side of the same problem, a fixed read order and checklist for AI-authored pull requests turns this into steps.
The motivating question came from the discussions with Andrej Karpathy on code agents and Dex Horthy on AI coding workflows. They are starting points for the question; the factual claims above are tied to the linked research and engineering guidance, not to unverified transcript quotations.
Questions people also ask
4 QUESTIONSIs there an established name for this approach?
There is no single name established by the research cited here. Cognitive load describes limits on processing information. Supervisory control concerns people directing automation. Scalable oversight studies how people can evaluate systems beyond their unaided abilities. Spec-driven development is one development technique, not a name for all three.
Does research establish two pages as the right specification length?
No. The studies cited here do not establish an ideal page count or a safe ratio of specification length to generated code. A short overview should expose the decisions and link to the detail needed to assess them. A complex change may need a longer review or a smaller scope.
Can an approved specification replace code review?
No. It establishes intended behaviour, not whether the implementation achieves it. Review the changed behaviour in context, inspect the relevant tests and give extra attention to harmful or hard-to-reverse failures. A generated summary is a guide to evidence, not evidence by itself.
Should every agent action require human approval?
No. In the workflow proposed here, agents can research, edit a branch and run permitted checks without asking about each mechanical step. People decide consequential requirements and accept changes. An unresolved requirement, changed assumption or missing critical check stops the affected work rather than silently becoming approval.