The Bus Factor in an AI-Written Codebase: How to Measure Who Actually Understands Your Code
Audit your AI code bus factor with five concrete comprehension checks, an interactive adjustment, and a transparent hypothetical example.
Count people who can explain and safely change each critical area—not names in Git.
Think of Git like a library book's checkout card: it tells you who carried the book, not who can explain it. AI can hand a developer a finished book while Git prints the developer's name on the card. To find who knows the story, ask them to explain a chapter, predict the ending, and repair a torn page.
For AI-written code, the useful bus factor is not how many names appear in Git. It is how many people can explain a critical area, predict how it fails, change it safely and debug it without delegating the thinking back to the tool. Measure that evidence per area, and treat the smallest count as a warning to spread knowledge.
What the bus factor measures — and why it fails with AI code#
The traditional bus factor, also called the truck factor, estimates how many people could disappear before a project could no longer continue normally. Repository tools usually approximate it from code ownership: identify the people responsible for most files, then remove the strongest owners until too much of the code has lost an owner. That is useful only because authorship used to stand in, imperfectly, for knowledge.
Even before AI, that was an estimate rather than a fact. The Applied Software Engineering Research Group behind one truck-factor tool warns that its results come from authorship measures and do not account for documentation, tests or other practices that preserve knowledge. A maintainer may understand code they never committed; an old author may have forgotten it.
AI makes the proxy weaker in a specific way. A person can prompt an agent, accept a patch and become the author recorded by Git without constructing the mental model that writing the implementation once required. A 2026 position paper calls this an invalidation of authorship-based knowledge metrics and, importantly, leaves a replacement instrument as an open measurement problem. So there is no accepted “comprehension factor” to install. There is only a question your team can test with direct evidence.
AI authorship vs. code comprehension#
Authorship answers who caused these lines to enter the repository? Comprehension answers four different questions: who can explain the intent, follow the runtime path, anticipate the dangerous failure and repair it under pressure? A commit cannot answer those.
This is not an argument that using AI necessarily destroys understanding. In a randomized study of 52 participants learning the unfamiliar Trio library, the AI-assisted group scored 4.15 points lower on a 27-point evaluation on average. Every participant reported using Python at least weekly for more than a year, having used AI coding assistance at least a few times, and never having used Trio. The authors tested one bounded task, not junior developers as a group or long-term learning in production repositories. The safe conclusion is narrow: access to generated code does not prove the operator learned it, and the way the tool is used matters.
The maintenance hand-off is real too. A study of 508 agent-generated and 508 human-generated files across 100 public repositories found that humans made 83.21% of the subsequent commits to the sampled agent-generated files. That result does not measure comprehension. It does show why “the agent can maintain it later” is not an ownership plan: in the observed sample, people still performed most later work.
A bounded code-understanding exercise#
Choose one operationally coherent area—checkout, token validation, invoice generation—not a whole monorepo. Give each candidate 45 minutes, the repository and its normal documentation, but no coding assistant. Use the same small scenario for everyone and score one point for each independently demonstrated check:
| Check | Bounded task | Evidence to keep |
|---|---|---|
| 1. Explain intent | State what the area protects or enables, plus one constraint that shaped it. | A short written explanation tied to code or an architecture decision. |
| 2. Trace behaviour | Follow one named input from entry point to durable effect. | A call-and-data path with the important boundary marked. |
| 3. Predict failure | Given one dependency timeout or malformed input, predict the visible result. | The prediction before running the scenario, then the observed result. |
| 4. Change safely | Make a small requirement change and name the tests that should fail first. | A reviewed patch and the before/after test result. |
| 5. Diagnose | Find a reversible seeded fault without being told its file. | The diagnosis, evidence used, and repair. |
Illustrative heuristic, not a validated metric: count a person for that area when they pass at least four of five checks. The threshold is deliberately strict enough to reject a fluent tour and simple enough to repeat. It has not been validated against incident recovery, team performance or project survival. Change the scenarios to fit the system; do not quietly lower the threshold after seeing the result.
Interactive calculator: adjust authorship with evidence#
Start with the authorship-derived bus factor your repository tool reports. Enter each candidate’s number of passed checks. The working result is the smaller of the Git estimate and the number of people who passed four checks. That rule can reduce a reassuring authorship estimate; it cannot inflate one from a small exercise.
ILLUSTRATIVE HEURISTIC · NOT A VALIDATED METRIC
| Person | Checks passed (0–5) | Counts as demonstrated? |
|---|---|---|
| Engineer A | Yes | |
| Engineer B | Yes | |
| Engineer C | No | |
| Engineer D | No |
2 people demonstrated understanding. The Git estimate is 3.
At 60% AI attribution, audit this area directly; do not discount the result by 60%.
The AI-attribution control does not multiply or discount the answer. There is no evidence for a universal “60% AI means multiply by 0.4” rule. Attribution tells you where the authorship proxy deserves direct checking and how much of the area your sample should cover. The result remains evidence about named people, on a named area, on the day of the exercise.
Illustrative example: a hypothetical checkout repository#
Illustrative example — hypothetical inputs, not observations: take a made-up checkout repository split into three critical areas. Assign it an authorship-derived bus factor of three and four fictional candidate scores per area. These numbers demonstrate the arithmetic only; no people or repository were audited.
| Area | Candidate scores | People at 4/5+ | Working result |
|---|---|---|---|
| Cart pricing | 5, 4, 3, 1 | 2 | min(3, 2) = 2 |
| Payment capture | 5, 3, 2, 0 | 1 | min(3, 1) = 1 |
| Refund ledger | 4, 4, 4, 2 | 3 | min(3, 3) = 3 |
For these hypothetical inputs, the repository-level warning is one because payment capture is the weakest critical area. Averaging the three results to two would hide exactly the dependency the heuristic is meant to expose. In a real audit, that result would point to a next step to test: pair a second engineer on payment incidents and changes, then repeat that area’s diagnosis and safe-change checks. Adding another CODEOWNER alone would not demonstrate that the second person understands the area.
What the number cannot tell you#
A score of two does not mean two people can handle every incident. The exercise samples one change and one fault. It can miss rare operational paths, domain knowledge outside the repository, coordination under stress and design intent that was never recorded. Rotate scenarios and keep the raw evidence; a dashboard number without the tasks behind it recreates the same false confidence as Git blame.
Do not use the result to rank individual engineers. It is a map for team risk: which area needs pairing, a runbook, an architecture decision, a game day or a second on-call maintainer? If passing becomes a performance target, people will rehearse the test instead of spreading understanding.
Related topics and internal resources#
Use the decompression test to sharpen the explanation and failure questions. Put high-risk AI changes through a human merge gate, where approval carries ownership but is not confused with proof of lasting comprehension. Record why a load-bearing choice exists with an architecture decision record, then make knowledge transfer an explicit responsibility when you decide who owns the factory.
References and sources#
- Brett Wheeler, “The Substrate Collapse” (2026): a position paper on why AI generation breaks the inference from authorship to understanding; it explicitly leaves a validated replacement open.
- Judy Hanwen Shen and Alex Tamkin, “How AI Impacts Skill Formation” (2026): randomized learning experiment, task design and limits.
- Shota Sawada et al., “To What Extent Does Agent-generated Code Require Maintenance?” (2026): sampled files, later commits and human maintenance share.
- Applied Software Engineering Research Group, “What is the Truck Factor of GitHub projects?”: explanation and cautions for authorship-derived estimates.
- Addy Osmani, “Comprehension Debt” (2026): practitioner framing for the gap between code volume and human understanding.
Questions people also ask
5 QUESTIONSDoes AI count as a contributor in the bus factor?
No. A bus factor asks how many people the project can lose before knowledge or delivery is endangered. A model is not a durable member of the team: it cannot carry accountability, retain private organisational context reliably, or be available under the same terms tomorrow. Record AI attribution so you know where to audit, but count people who demonstrate understanding.
Does a thorough code review prove that the reviewer understands the code?
Not by itself. Approval records that a decision was made, not how deeply the reviewer can explain, change, or debug the area later. Treat review as one source of candidates for the exercise, then test the operational skills directly.
Can tests replace human understanding?
No. Tests preserve specified behaviour and are essential evidence, but they cannot explain an unstated constraint or tell you whether the test suite itself encodes the right behaviour. A person still needs enough of a mental model to decide what should have been specified.
How often should a team repeat the audit?
Repeat it after a large ownership change, before an on-call handover, and when a critical area has changed substantially through agent-generated pull requests. For stable critical areas, a quarterly spot check is a practical starting cadence, not a universal standard.
Is this the same as comprehension debt?
The audit exposes one form of it: code exists, but too few people can reason about it. Comprehension debt is broader and can include forgotten design intent, weak system models, or skills that have not formed. The exercise measures neither all debt nor its cost.