Code Review in the Age of AI-Generated PRs

AI-authored pull requests can contain local mistakes as well as bugs in the connections between files. CodeRabbit’s comparison found more logic and correctness issues, alongside more readability and formatting problems. This page recommends reading structure before logic and tracing a real path through the change. That order is its own review procedure, not a measured guarantee that other reading orders fail.

Read it in this order, and here’s why#

GitHub’s own write-up on reviewing agent pull requests counts over 60 million Copilot code reviews, growth of 10x in under a year, and puts more than one in five reviews on GitHub today as involving an agent. Whatever you decide about reviewing AI-authored code, the queue in front of you already has some in it — this is GitHub’s read, published 7 May 2026.

The first thing that changes is size. AI-assisted pull requests run to 408 lines at the 75th percentile against 157 for the unassisted kind, in LinearB’s original benchmark report — about two and a half times larger, while agentic PRs in the same report run 293 lines, about 1.9 times. This is a size comparison, not a measurement of review queue capacity. Reading that top to bottom, in diff order, spends the same attention on a renamed variable as on an auth check, and the auth check did not get read first just because it needed to be.

So the order is structure before content, and it is deliberately the same order every time: shape and size, then CI and config, then duplication, then one full path traced end to end, then the security boundary, and tests last — read as evidence rather than proof. Style is left off this list on purpose. Whether to block on it is a different question, and mixing it into a correctness order slows down the one thing an AI PR needs more of.

CI and config come second, ahead of any logic, because they change what everything after them is worth. An agent that cannot get its own change to pass can delete or skip the failing test instead of fixing the code. GitHub’s own guidance on reviewing AI-generated code tells reviewers to check for new warnings and for tests that were deleted or skipped instead of fixed. If a required check changed, inspect what the green build still covers before treating it as equivalent to the earlier checks.

Duplication comes third in this procedure because an agent’s context may omit an existing helper. Repository search can reveal it; without that search, a new implementation may enforce the same rule differently. Check whether the change reuses the existing policy or introduces a second policy that maintainers would have to keep in step.

The traced path is the step that catches what none of the others can: a change that is right on every individual line and wrong where two of those lines depend on an assumption about each other that neither one states. That is also the reason it comes before the security boundary rather than after — you cannot check a boundary you have not yet followed a real request into.

Tests come last, and deliberately read as evidence rather than accepted as proof, because green tests are the thing most likely to make you stop looking. A suite that passes on an implementation it was written to match tells you the implementation is self-consistent. It does not tell you the implementation is right.

The checklist, ready to copy#

This is the order above, turned into ten checkboxes. Tick them in the page, then copy the result straight into a PR template or a review comment. Work down it in order — the order is the point, not a suggestion to reorder to taste — and treat an item you cannot check off as a comment, not a silent pass.

Try it: tick off a review as you go

Work down in order. Whatever you leave unticked is copied as an open item, so it reaches the author as a comment instead of a silent pass.

1. Shape: the file list and diff size match what the title claims — no area changed that the title doesn't mention
2. CI: no change to .github/workflows, required checks, coverage thresholds, or a skipped/quarantined test
3. Duplication: searched the repo for an existing helper, client or type before trusting a new one
4. One path: traced a full request end to end — entry, every branch, the failure case, where it writes
5. Boundaries: auth, tenancy and input validation on the touched surface are unchanged, or the change to them is the point of the PR
6. Fail state: checked what happens when the new dependency — cache, queue, external call — is unavailable
7. Deletions: every removed name has no remaining caller
8. Scope: nothing outside the stated diff moved
9. Unknowns: anything you could not verify is named in the review, not left unsaid
10. Tests: they assert the requirement, not the implementation, and fail if you revert the fix by hand

Copy as review comment

0 of 10 checked.

Two items are worth explaining before you use it. “Fail state” is not on most human-PR checklists, and it earns its place here because an agent asked to add a cache, a queue or an external call rarely gets asked what happens when that dependency is down — and a fail-open default is an easy one for it to reach for, because failing open keeps the change looking finished. That is an assessment, not a measured figure. “Unknowns” exists because an honest partial review — naming what you did not check — is worth more than a silent approval that implies you checked everything.

A worked example, step by step#

In this illustrative example, constructed for the page and not taken from a real repository, an agent opens a PR titled “Add rate limiting to /v1/export,” using a Redis-backed token bucket. It is a plausible, ordinary-looking change. Here is what the six steps above find in it.

The example has three findings: a duplicate policy, a shared anonymous bucket and an unintended fail-open path. The fail-open catch can be spotted locally; tracing the endpoint connects it to the outage behaviour and exposes the shared-key assumption. Repository search finds the duplicate. These are complementary checks, rather than proof that a diff-order read cannot find a bug.

The numbers behind each step#

The table connects this page’s recommended checks to measurements and guidance. The sources motivate those checks; they do not test this exact six-step order against another review procedure.

The size figures are PR Lens’s reading of LinearB’s benchmarks (8.1 million pull requests across 4,800-plus development teams). The duplication figures are GitClear’s, from its report on 623 million code changes, updated in 2026 — they motivate a repository search for an existing implementation before trusting a new one, since more of the changed code is a second copy of something that already exists. The logic-error, security and issues-per-PR figures are CodeRabbit’s, from 470 open-source pull requests published 17 December 2025. CodeRabbit used repository signals to classify 320 PRs as AI-co-authored and treated the other 150 as its human comparison group; it says it could not guarantee that every PR in that comparison group was written only by people. The measured differences therefore belong to those classified groups, rather than proving the authorship of every PR.

Duma et al. found no recorded review activity for 61.38% of 33,596 agent-authored PRs. Silent inspection may leave no comment, so this measures traceable interaction rather than the absence of human oversight. In the shared-repository comparison, which contains 9,616 agent PRs and 5,574 human PRs, 34.52% of human PRs also lack recorded review activity. Observable human participation is similar in those two groups: 30.12% and 30.83% respectively.

LinearB’s original report separates the PR groups at the 75th percentile. AI-assisted PRs wait 496 minutes for pickup and have a review time of 194 minutes; agentic PRs wait 1,055 minutes, about 5.3 times the unassisted pickup time, and have a review time of 382 minutes. Unassisted PRs wait 201 minutes and have a review time of 252 minutes. Review is shorter for the assisted group and longer for the agentic group; combining them into a single faster-review claim loses that distinction. The report separately gives acceptance rates of 32.7% for AI PRs and 84.4% for manual PRs.

DORA’s 2026 tension analysis includes one practitioner’s comment that reviewing another person’s code is harder than writing it. That is an experience quoted by the source, rather than a measured effect size. None of these aggregates replaces judgement about changes that need human attention regardless of size.

Nothing above requires you to build anything. If your team is drowning enough in this queue that a first automated pass would genuinely help, building a review agent is a reasonable next step — it should honour the same order, and it should never be the only reviewer a change gets. For the team-level question of where human attention runs out as AI-written changes pile up, see the human-attention bottleneck; this page is the individual reviewer’s procedure for one pull request. The same instinct that says a model should not be trusted to grade its own diff is the deeper argument in deciding which steps in a pipeline get a model at all.

Questions people also ask

6 QUESTIONS

▸ How do I review a pull request I did not write, in a codebase I don't fully know?

Review the boundary of the change, not everything around it: what it touches, what it calls, what calls it, and what it removed. Trace one path all the way through rather than trying to hold the whole file in your head, and say plainly which parts you could not judge instead of approving them by default. That gets you a real review without first needing the depth of someone who has owned the code for a year.

▸ Which parts of an AI PR are safe to skim, and which need a full trace?

You can move quickly over formatting rules that required CI demonstrably enforces, after checking that the PR did not weaken that check. Read comments for accuracy, and inspect naming when it carries a domain rule. The path you traced for the checklist, anything touching auth, tenancy or input validation, and any new external call or dependency need a full trace: those are exactly the places a change can be correct on every line and still wrong where two correct lines meet.

▸ How do I tell if an AI-generated test actually tests anything?

Break the code on purpose and see if the test fails. Revert the exact line the PR claims to fix, or comment out the new check, and run the suite; a test that still passes is asserting the implementation's shape, not the requirement. Also read what it mocks — a test that mocks the exact call it's supposed to be proving can't fail no matter what the real code does.

▸ Should I block a PR for style if the logic is correct?

Only if the style choice is actually a claim about the system, not a preference. Name the principle it rests on in one sentence: if that sentence is about what happens to the system, it's a blocker; if it's about what you would have written, it's a nit, and saying so costs you nothing. A missing error path or an unhandled failure mode isn't style at all, whatever line it's written on.

▸ Why are AI PRs bigger than human PRs, and does that matter?

The sources measure the size gap but do not establish why it exists. LinearB reports 408 vs 157 lines at the 75th percentile for AI-assisted PRs, where the developer stays in control of the workflow, about 2.6 times larger. Agentic PRs are smaller in the same report, 293 lines, about 1.9 times the unassisted size, so 2.6× is not a fact about agents. Either way a larger diff gives a reviewer more code to assess; the figure does not show whether reviewer capacity changed.

▸ How do I stop rubber-stamping PRs that look right but aren't?

Read in the fixed order below rather than top to bottom, and require yourself to name one traced path per PR before you approve it, even a short one. A PR that looks right on a diff-order read and a PR that is right are not the same claim, and the gap between them is exactly what an unordered read misses.