What a merged change costs

The first invoice from a fleet of agents arrives as one number with nothing to attribute it to, and the instinct is to go looking at the price list — this model is a fifth of the price of that one, so switch. Sometimes that works. When it does not — when the total barely moves, or goes up — the price list is no help at all in explaining why.

What you are actually buying#

A merged change, not a run and not a token. Everything else is an input to that: the planning pass, the implementation, each review cycle it took to clear the bar, the verification afterwards, and the attempts that were abandoned along the way. A price per million tokens describes one step in that sequence, priced per unit of a thing you do not buy directly.

Three quantities drive the bill and they are not equal. First, and usually dominant, is how much of the codebase the task has to load — the context, meaning everything the agent read before it could act. An agent that explores twenty services to find where invoices are numbered pays for all twenty. Second is the number of tool calls: a reviewer that posts eleven findings as eleven separate comments has made eleven round trips, each carrying its accumulated context with it. Third is retries, which are different in kind, because they do not add to the other two. They multiply them.

That multiplication is the same arithmetic that governs reliability, and it deserves the same suspicion. A per-attempt price behaves exactly like a per-attempt success rate: correct about one attempt and misleading about a chain of them. A change that takes four steps, each of which occasionally has to be redone, does not cost the sum of four step prices. It costs the sum of four step prices times however many goes each one took, and the second number is the one nobody measured before signing off the budget.

One published comparison put figures on the spread. The same feature, given to a couple of dozen model-and-harness pairings and held to the same automated senior review, merged for anywhere between about $3 and about $33 — and what caused the spread was not how hard each model thinks on a step, but how many turns it takes and how much context it carries on each one. Changing only the command-line tool around a fixed model and task moved the cost of a merged feature by roughly 2.5 times. The token price was a minor part of the difference.

The levers, in order of size#

Model choice per agent comes first, because it is the largest single saving available and it is available this afternoon. A factory is not one agent but a dozen jobs of wildly different difficulty, and only some of them need the strongest model you can buy. Triaging a report into a category, deciding whether two crash reports are the same crash, slicing a hundred-megabyte log down to the forty lines around a stack trace — none of that is implementation or review, and running it on a frontier model is paying senior rates for filing. Route those steps down a tier and the difficult ones keep the model they need.

Caching comes second, and a factory is close to the best case for it. Every run in the fleet opens with the same shared instruction set and the same system prompt, byte for byte, because that is what makes the fleet a fleet; providers bill a cached read at a small fraction of ordinary input precisely for this shape. The saving scales with how much of the front of the request never changes, which is a design choice as much as a pricing one — a preamble assembled slightly differently per run cannot be cached at all.

Batching comes third and costs you nothing but time. Work that nobody is waiting on — the overnight verification pass, the scheduled sweep over stale items, the weekly documentation check — can go through a provider’s batch path at a discount, and the only constraint is latency tolerance. For a job that runs at two in the morning against yesterday’s merges, that tolerance is total.

The fourth lever is not about pricing at all, and it acts on the term that dominates: reduce what has to be loaded. A machine-readable map of the system — services, the paths each owns, dependencies, risk class — turns which services does this change touch from an exploration into a lookup, which is one of the quieter arguments for keeping a shared instruction set in good order. A script that extracts the relevant fragment of a log does the same for evidence, letting the model read forty lines instead of forty thousand. Most of the loading happens either side of that boundary between what a script decides and what a model decides.

One practical note, because it catches finance teams rather than engineers. Factory agents authenticate with API keys under platform usage tiers and rate limits, which is a different arrangement from the per-developer tooling the same team already pays for. A fleet therefore needs its own capacity planning and its own monthly ceiling, agreed before the first sweep runs rather than discovered from an invoice. Per-team limits matter for the same reason: without them, one team’s enthusiastic backlog clear-out can consume the month.

Where the number goes wrong#

Optimising the token price and watching the total rise is the classic version, and now the mechanism is visible it is easy to predict. Move implementation to a cheaper model, it produces work that needs a second review cycle, and you have traded a saving on the smallest term for a doubling of one of the larger ones. The saving is real. It is just being spent somewhere the price list does not show you.

The expensive failure is the repair loop with no ceiling. An agent told to keep going until the build is green will do exactly that, or keep going until something stops it — and if nothing stops it, one work item on an intractable problem bills like a week of ordinary ones. The cap belongs in the design rather than in an alert: a maximum number of attempts per item, a token ceiling per run, a daily limit per team, and a cap that hands the item to a person when it trips instead of quietly starting again.

Then there is the denominator. Measuring cost per run makes abandoned work free, which is precisely backwards: an agent that spends four dollars and produces nothing mergeable has cost four dollars, and reporting it as a cheap run hides the most useful signal you have. Divide by merged changes and the arithmetic tells the truth, including the uncomfortable truth that a run which ended in a person rewriting the change by hand belongs in the numerator too.

The environment is the omission that turns a report into fiction. Every run that builds and tests something needs a container started, a database migrated, seed data loaded, and an artifact stored somewhere; on short changes that infrastructure can rival the model spend, and it appears on a completely different invoice. A cost-per-merged-change figure that counts only tokens is not a conservative estimate. It is the wrong number with an air of precision about it.

Last, and least obvious: the word merged has to keep its meaning, or the metric improves when the gate gets worse. In the comparison quoted above every run cleared the automated senior review, yet an independent test oracle showed those accepted implementations varying widely in how much they actually passed — the author’s own case for holding agent output to something objective as well as to a model’s opinion. Soften the gate and cost per merged change falls beautifully while the thing you were buying stops arriving. That is what an honest set of factory metrics exists to catch, and why this number never travels alone.