What a merged change costs
Calculate agent costs across complete tasks, including retries, failed runs, test environments and human review. Compare savings without weakening checks.
To measure the cost of an agent's changes, count the work needed to reach acceptance: model calls, tools, test environments and human review. Include failed and abandoned attempts. A lower token price helps only if the whole task still reaches the required result at a lower cost.
A cheaper model saves a euro on writing a fix but leaves an extra hour of review and correction. The API bill went down; the work did not become cheaper overall. Compare the complete task and keep human time visible alongside the invoices.
A token price tells you what one model call costs per unit of input or output. It does not tell you how many calls a task will need, how long its test environment will run or how much correction a person will do. Measure those parts together before deciding where to save money.
Add up the complete task, including failed attempts#
Choose an outcome before calculating a unit cost. For an implementation workflow, it may be a change accepted and merged under the existing review rules. For a review-only workflow, it may be a completed review that a person judges useful. A run is an execution attempt, not necessarily either outcome.
Give each task an identifier that connects its planning calls, coding attempts, reviews and tests. Record model usage and tool charges, environment time, storage and other direct costs. Include attempts that failed or were abandoned, not only the successful final run.
The figure illustrates how costs accumulate; it is not a provider price comparison or a measured prediction for your project. Different stages can have different prices and retry counts. Add their actual costs rather than multiplying the entire task by one average retry number.
Here is a separate calculation using invented numbers. A cohort of completed tasks costs €120 in model, tool and environment charges and produces ten accepted, merged changes. Its direct cost per merged change is €120 ÷ 10 = €12. The €120 includes unsuccessful attempts associated with that cohort.
Suppose people also spend twelve hours preparing, reviewing and correcting those tasks. At an explicitly assumed €50 per hour, that adds €600 of attributed labour cost. The combined figure is (€120 + €600) ÷ 10 = €72 per merged change. Report the assumptions and human hours separately; the €600 is not another API invoice.
Shared subscriptions, platform maintenance and idle capacity need an allocation rule if included. Show direct operating cost and allocated overhead separately so a reader can reproduce the calculation. Count labour only once when allocating team costs.
No merged changes means the ratio is undefined, not zero. Report the spend and outcomes directly. For unfinished tasks, report work in progress or follow the cohort to completion; dividing this month’s starts by last month’s merges can distort the result.
Reduce the cost you actually measured#
There is no universal ranking of the largest savings. If failed setup consumes most of the bill, fix the environment. If irrelevant input is repeatedly loaded, improve retrieval and the task brief. If review dominates, investigate which defects and unclear proposals create that work.
Compare models by task type. A less expensive model may handle a routine classification well, while another task benefits from a stronger model. Use representative evaluation cases and unchanged acceptance rules. Include the cost of routing mistakes and retries, not only the selected model’s successful call.
A cheap-first cascade adds a charge to every screened input, including those passed to the expensive stage. Count screening, forwarded calls, retries and checking work together. The model-cascade walkthrough calculates a break-even point with invented prices and then changes the forwarding rate to show when the extra stage becomes more expensive.
Keep the quality question separate from that arithmetic. In a document-review workflow, a relevant passage incorrectly excluded by screening never reaches the deeper review. Checking only forwarded passages hides this loss. Measure missed material and audit excluded cases before describing the lower bill as a saving under equivalent requirements.
Remove model calls where the rule is already explicit. A script can extract a known log section or run the required formatter. The division between scripts and AI should follow the work, not a goal of putting a model in every step.
Provider caching can help with repeated input. Anthropic’s prompt-caching documentation describes matching reusable prefixes, expiry and separate cache-write and read treatment. Check actual cache usage; similar-looking requests do not necessarily match, and changing a later section does not necessarily invalidate every earlier reusable prefix.
Anthropic’s batch API is an example of discounted asynchronous processing with its own limits. A sequence that needs a tool result before choosing the next call cannot simply submit all those dependent calls at once. Check deadlines, supported features and environment waiting costs before moving work into a batch.
Prices, account terms and features change. Budget from the billing arrangement the chosen tool actually uses. Some workflows use metered APIs, others use licensed tooling or locally operated models. Do not assume a developer subscription automatically covers every unattended integration, or that every factory must use the same billing model.
Finally, enforce limits before new work starts. Track the task across replacement sessions and child agents. Allow for calls and environments already running when a threshold is reached. Combine per-task controls with appropriate team or account limits, and return saved work rather than silently restarting it.
Keep the comparison fair#
A cheaper call can produce a more expensive task if it requires extra correction. The reverse can be true too. Record what happened on comparable work instead of assuming that model price or model size determines the whole outcome.
Do not assume every tool call sends a full conversation back to a model. The runtime determines when another model call happens and what input it carries. Use recorded usage rather than estimating the bill by counting comments or shell commands.
Keep environment cost visible. Some tasks need a database and substantial integration tests; others do not. Measure startup, active use, waiting and cleanup for the environment actually provided. A smaller model does not shorten a blocked database startup.
Keep quality fixed during comparisons. Skipping a test or accepting a weak implementation can reduce cost per merge while making the software worse. Count escaped defects, reversions and later rework alongside the apparent saving.
A merge is also not a measure of product value. Generating unnecessary changes or splitting work into many tiny pull requests can improve the denominator without helping users. The measurement guide explains how to compare the complete process without mistaking activity for useful output.
IF YOU REMEMBER ONE THING
Count the whole task and report human time separately from direct charges. Reduce the measured source of cost without changing what a successful result is required to do.
Questions people also ask
4 QUESTIONSIs cost per merged change the only useful measure?
No. It is useful when merged changes are the intended outcome. Cost per run helps diagnose expensive steps, and review-only workflows need their own outcome measure. Also track quality and human effort. Merging more tiny or unwanted changes can improve the ratio without improving the work.
Does a cheaper model make the whole change cheaper?
Not necessarily. It may need more attempts or leave more review work, although it may also perform well enough at a lower price. Compare equivalent tasks under the same acceptance rules and include unsuccessful runs. Do not assume either result from the model's price alone.
When can caching or batching help?
Caching can reduce the cost of reusing eligible prompt input; the provider's matching rules, expiry and write charges matter. Batch APIs can discount supported asynchronous requests, but they have completion and feature constraints. Check the actual workload and current terms rather than assuming every overnight agent task fits.
How do you limit spending on a difficult task?
Enforce time, attempt and spending limits across the task and its child workers. Check before starting new calls and account for work already in flight. Save the unfinished work when a limit is reached. An alert or a counter reset by each new session is not a hard spending control.