The human merge gate

In the supervised workflow described here, a person accepts each change before merge. They do not have to prove that no bug can exist, but they should understand the changed behaviour and the reasons for accepting it. The approval button records that judgement; it does not make the judgement.

Decide who may accept the change#

Set the approval policy explicitly. Which changes need a person, who may approve them, and which checks must pass first? Enforce the policy in the repository or release tools. A good run must not give the agent permissions it did not have at the start.

This is one factory design, not the definition of every software factory. A team can separately authorise automatic handling for a narrow kind of change. That decision still needs responsible maintainers, adequate checks and a way to handle failure.

In this supervised version, the review agent supplies findings rather than merge approval. The human reviewer examines the proposal and knows who will maintain and operate the result. The team cannot delegate that responsibility merely by saying that a model wrote the code.

Understandable code matters to that handover. When nobody on the team can explain the change, request a clearer implementation or the right specialist review. Passing tests do not show that the team will be able to diagnose the next defect.

Give consequential changes more scrutiny#

An agent can propose a risk label and identify uncertain areas, but the reviewer checks whether that assessment fits. A tiny authorisation change may need more scrutiny than a large mechanical rename.

The table is an example review policy. Adapt its categories to the system and the expertise needed; it is not a validated risk score.

Use a size limit to spot unexpectedly broad work, not as the only measure of risk. What the change permits, deletes or exposes matters more than how many lines it contains.

Understand module contracts without skipping changed code#

A deep module does substantial work behind a relatively small interface. It might be a function, class or package; it need not be one large class. Matt Pocock’s codebase-design guidance stresses that the interface includes everything callers must know, not just parameter types: errors, ordering rules and other constraints belong to that contract too.

In an illustrative CSV exporter, callers supply already-authorised rows and the required column order. The exporter owns quoting and encoding. Its contract must also explain what happens when writing fails. Callers need those promises, but should not have to coordinate each buffer operation. Internal helper functions can remain small without becoming obligations for every caller.

A gray-box approach means understanding the module’s purpose and contract while consulting its internals when needed. Here that reduces repeated investigation of unchanged details; it does not excuse skipping new code. Read the changed implementation and the surrounding code needed to judge it. Check affected callers and tests. Inspect more deeply when permissions, data loss, concurrency or a changed contract are involved.

Google’s review guidance calls for understanding assigned changes in their wider context. Do not treat ordinary AI-written application code as equivalent to mechanically generated files that may sometimes be scanned. If the implementation is too difficult to understand, ask for a smaller or clearer change, or involve a qualified reviewer.

The handover should identify the changed promise, affected callers, actual check results and remaining uncertainty. The test-first CSV example shows how to produce useful evidence for one promise. Neither a short interface nor passing examples prove that all behaviour is correct.

Explain why a change is considered low risk#

Name what could go wrong, who or what it could affect, how the team would notice and what could be restored. A label such as “low risk” should summarise those reasons rather than replace them. Multiplying invented scores does not remove uncertainty.

Compare a help-page wording correction with a change that lets one user read another user’s records. The first can be inspected in a preview. The second can disclose information before an error appears. Reverting the code cannot retract information already exposed, unsend a notification or recover every deleted record.

Give detection and recovery credit only when they exist. Name the relevant test, runtime signal and response procedure. Post-deploy verification needs to check the intended revision and behaviour, not merely that a process is alive.

A gradual rollout can reduce the number of users exposed before a problem is found. The SRE Workbook’s canarying chapter explains this approach. A small rollout still cannot make an irreversible disclosure or deletion harmless.

Resolve important questions without demanding perfection#

Inspect the changed behaviour in context and check the tests that support it. Resolve the important correctness, security and recovery questions. Then distinguish remaining blockers from optional improvements.

A naming preference need not delay a sound change. An unexplained permission check can. Google’s code-review standard favours changes that improve code health without requiring perfection; it does not justify accepting work that makes the system harder to maintain.

Record what was checked and any accepted limitation. Link approval to the relevant revision and recheck it after material changes. Faster generation of a later fix does not reduce harm that occurs before anyone notices the original defect.

Keep the review queue small enough to review#

A required approval can become a formality if too many proposals arrive for people to inspect. The repository still records approvals, but that activity alone does not establish meaningful review.

Measure waiting work and reviewer capacity. When the queue is full, pause new generation or finish the existing tasks before adding more. Review is one possible bottleneck, not inevitably the only one; preparation, tests, environments and release can also constrain delivery.

Track human review effort per accepted change, findings that required correction and defects that escaped. Under a policy requiring human approval, an unapproved merge is a policy violation to investigate. Keep that separate from whether recorded approvals were substantive.

Do not reward the agent merely for producing more pull requests or a higher share of merged code. Check whether the changes were useful and what work they left for people. Agent evaluation cases can test specific failure handling; they do not measure the product value of every proposed task.

One further risk is that the change was influenced by hostile text the agent read. Continue with prompt injection in a pipeline to see why review and restricted permissions need to account for that input too.

Questions people also ask

6 QUESTIONS

▸ Must every software factory require human approval for every merge?

No. This article describes a supervised policy that does. A team may permit some well-defined changes under an approved automatic policy. That is a separate decision about the checks, consequences and failure handling; a successful agent run does not silently change the merge rules.

▸ Does risk-based review mean skipping the code?

No. Inspect the changed behaviour and relevant surrounding code, and check that the tests can expose the important failures. Spend additional effort on harmful, hard-to-detect or hard-to-reverse mistakes. A green build and an agent summary do not replace understanding.

▸ What is a deep module, and why does it help review?

A deep module provides substantial behaviour through a relatively small interface. Callers can use that contract without reconstructing every internal step. Review still covers changed code, relevant dependencies and whether the implementation meets the contract; a short function signature does not make a module safe.

▸ Can review capacity limit a factory's output?

Yes. When proposals arrive faster than people can review them, work waits even if coding is fast. Other stages can also limit output. Measure the actual waiting and rework, then limit new tasks when the review queue is full.

▸ How do you keep human review from becoming a formality?

Keep changes understandable, supply actual check results and unresolved questions, and avoid starting more work than reviewers can absorb. Track review effort and defects as well as approval events. A recorded approval alone does not show how carefully someone read the change.

▸ Which changes need specialist attention?

In the example policy here, authentication, permissions, migrations, destructive operations, credentials and changes to the checking pipeline need an appropriately qualified person. Define the categories for your system. The consequences can be serious even when only one line changes.