Run an AI coding tools cost analysis by auditing the agent loop, because per-seat pricing misses the main cost factors. Published agent traces show that input tokens re-sent on every turn account for nearly all measured spend, and loop design controls how fast they accumulate.
TL;DR
Anthropic cost examples and arXiv agent studies show input tokens dominate agentic coding spend, often exceeding 99% of trajectory volume. Loop design, including context growth, retries, and agent fan-out, moves cost more than model choice, and routing cheap implementers under expensive reviewers cut benchmark costs by up to 14x.
A developer asks an agent to change three functions. The agent reads broad project context, retries through verbose tool output, and generates code only after the context has grown. When the bill arrives, most of the cost came from re-sent context before the agent generated code. Engineering leaders are approving AI coding budgets built on the wrong unit, because the gap between the invoice a CTO expects and the one finance receives lives inside the agent loop: context re-reading, tool output accumulation, retries, and parallel fan-out. This guide breaks down where tokens go turn by turn and why routing across models moves cost. It also covers what migrations like the $165K Bun rewrite reported and how to set a token budget that holds. A codebase index can reduce pre-search context spend by narrowing what the agent opens first. Augment Cosmos, our unified cloud agents platform for running agents in the cloud with shared context and memory that compound across the software development lifecycle, uses the Context Engine to apply semantic dependency graph analysis before an agent starts searching files.
Input Tokens Account for Most Spend
Published measurements of agentic coding sessions show the same shape: output tokens, the code the model actually writes, are small compared with the input tokens the loop re-sends. Anthropic's Claude Code costs documentation shows the pattern in a claude-sonnet-4-6 session:
| Token category | Volume | Share of total |
|---|---|---|
| Cache reads | 940,000 | ~94.5% |
| Cache writes | 50,000 | ~5.0% |
| Output | 5,300 | ~0.5% |
| Uncached input | 1,200 | ~0.1% |
| Total measured tokens | 996,500 | 100% |
Total cost for that session: $0.55. Peer-reviewed work corroborates the ratio across larger agentic coding measurements. The arXiv paper "How Do AI Agents Spend Your Money?" concludes that input token dominance holds in agentic coding even when teams use token caching, and a separate token economics study found agentic coding consumes over 1,000x more tokens than single-turn reasoning, with an input-dominant ratio exceeding 150:1. On OpenRouter, daily Claude 4 Sonnet usage hit 100 billion tokens in September 2025, and OpenRouter usage shows 99% were input tokens accumulated in agent trajectories.
The Four Categories Where Loop Tokens Go
Token spend inside an agent loop breaks into four measurable categories, and the largest one, context re-reading, exists because the model has no persistent memory between turns. Each turn re-sends the full accumulated conversation, so cost grows with session length regardless of how much new work happens. The loop categories are:
- Context re-reading. Once a tool result enters the trajectory, it stays in later turns unless the system edits or summarizes context. If the agent opens a large file or a command produces verbose output, tokens snowball in every subsequent step. Turn Summary shows which tool calls and code changes entered each response, so reviewers can find the context that repeats later.
- System prompt and instruction re-injection. System prompts, instruction files, and repository-specific guidance are part of the repeated input stream. The loop re-injects instruction files like CLAUDE.md on every turn, so even stable policy text becomes a recurring cost when the session runs long.
- File reads and tool definitions. Agents that lack a relevance index often compensate by reading broader project context before editing. Tool definitions add fixed overhead too: Anthropic's computer use tool definition costs 735 input tokens per request. The Context Engine processes entire codebases through semantic dependency graph analysis before file discovery, reducing the need for agents to open broad project context; the same failure mode appears in tools that struggle with enterprise multi-file refactoring.
- Compaction and fan-out multipliers. Compaction, summarization, and parallel agent fan-out multiply spend because each branch or rebuilt context has to carry enough prior state to continue. Fan-out can be valuable when independent agents explore separate approaches, but unmanaged parallelism turns one user request into multiple concurrent input streams. For fan-out reviews, Cosmos Sessions keep each concurrent branch replayable.
These categories explain how one user request becomes accumulated loop cost before model-unit pricing enters the calculation.
Run your software agents at scale
Cosmos gives your agents the context, tools, and feedback loops they need to get better with every workflow.
Why Loop Design Beats Model Choice
Changing the loop can move cost more than changing the model on the same task. The arXiv agent-spend study found some runs consume 30x token variance compared with others on the same task, and that accuracy often peaks at intermediate cost. In those runs, excess token expenditure frequently reflects unproductive exploration rather than deeper reasoning.
Several loop-engineering techniques have quantified savings across published sources. Anthropic has documented prompt caching savings and context editing results, while independent research quantifies gains from active context compression and codified prompting patterns:
| Technique | Documented saving |
|---|---|
| Prompt caching | Up to 90% cost reduction on long prompts |
| Context editing (clearing stale tool results) | 84% token reduction in a 100-turn evaluation |
| Active context compression | 57% savings on a SWE-bench task (4.0M to 1.7M tokens) |
| Sub-agents with isolated contexts and memory tool | 39% performance improvement over baseline |
| Codified prompting | 55-87% input token reduction |
Two caveats keep these savings from being free money. Caching correctness matters as much as caching existence, since a cache miss can make the same long prompt expensive again. Compression can also backfire when summaries hide state that later turns need. Teams should test whether their tooling identifies relevant files and stale context before relying on longer windows or constant summarization, an issue that surfaces clearly in tools that stall on large enterprise codebases. Summarization works at clear phase boundaries, while continuous compression can hide needed state.
Without hard stopping criteria, none of it holds. Agents persist through failures by design, trying variations and regenerating code while the meter runs. An agent budget should define stopping criteria for iterations, token counts, and dollar limits that force the loop to terminate.
Multi-Model Routing: Cheap Implementers, Expensive Reviewers
Many AI coding agent stacks use an expensive model for planning and review with a cheap model for mechanical execution. Most loop turns are mechanical, while plan quality determines whether the loop converges at all. Anthropic describes the Advisor tool pattern for workloads where most turns are mechanical but having an excellent plan is crucial.
Independent benchmarks quantify these routing patterns. Aider published results from its architect mode pairing R1 with Sonnet, Anthropic showed Advisor benchmark results for Opus-and-Haiku pairings, and separate arXiv studies quantify BudgetMLAgent cascades and RouteLLM routing:
| Pattern | Configuration | Result |
|---|---|---|
| Aider architect | DeepSeek R1 plans, claude-3-5-sonnet-20241022 edits | 64.0% at $13.29 vs. o1 solo at 61.7% and $186.50: 14x cheaper |
| Anthropic Advisor | Opus advises, Haiku executes | 41.2% SWE-bench, more than double Haiku's 19.7% solo |
| Anthropic Advisor | Opus advises, Sonnet executes | +2.7pp SWE-bench, ~12% cost reduction per task |
| BudgetMLAgent cascade | Free base model, GPT-4 for expert calls | $0.054/run vs. $0.931, 94.2% cheaper with higher success rate |
| RouteLLM routing (MT-Bench) | GPT-4/Mixtral-8x7B routing; a stronger-data variant sent 14% of calls to the strong model | Over 85% cost reduction at 95% of GPT-4 performance |
The Haiku-plus-Opus result matters because a cheap executor with an expensive advisor more than doubled the cheap model's solo score. Routing expands what a cheap model can do while lowering cost. Latency matters too: the R1+Sonnet pairing recorded 251.6 seconds per case in Aider versus 12.4 for a solo model.
Routing also argues against betting an agent strategy on one lab. Cosmos ships with Prism routing and BYOK across Anthropic, OpenAI, Bedrock, Vertex, and open-source models, so teams can reserve stronger models for planning and review while lower-cost models handle mechanical execution as price-performance shifts between releases. Teams weighing model providers and key-management options can also review enterprise-focused coding assistants beyond Claude Code to see which routing setups fit their workflows.
What Bun Reported About the $165K Rewrite
The cost of AI code migration now has a public, company-reported price tag, though the case study comes with material disclosures. Bun was acquired by Anthropic in December 2025, so Jarred Sumner's team now sits inside the same company that makes Claude, and the rewrite used a pre-release version of Claude Fable 5 rather than a generally available model. Readers should weigh those facts when generalizing the numbers to their own migrations.
Bun reported that its team ported 535,000 lines of Zig to Rust in 11 days (May 3 to merged May 14) across 6,778 commits, according to the Bun in Rust write-up. The team ran ~64 parallel Claude agents against a 600-line porting guide with adversarial review of each commit. Bun reported token consumption of 5.9 billion uncached input tokens, 690 million output tokens, and 72 billion cached input reads, for a total API-rate cost of approximately $165,000. Cached reads outnumbered output tokens roughly 100:1 even in a migration whose entire purpose was producing code. Jarred Sumner's manual counterfactual was three engineers with full codebase context working about a year. The rewrite has also drawn public criticism, including Zig creator Andrew Kelley calling parts of it "unreviewed slop", which points to the quality caveat below.
The Bun migration metrics show how scope, review, parallelism, and repeated context reads combine in a real agent-heavy rewrite:
| Bun-reported metric | Value | Cost relevance |
|---|---|---|
| Migration scope | 535,000 lines of Zig to Rust | Scope set the amount of code requiring translation and review |
| Calendar time | 11 days, from May 3 to merged May 14 | Parallel agents compressed elapsed time |
| Commit volume | 6,778 commits | Each commit required adversarial review |
| Parallelism | ~64 parallel Claude agents | Fan-out multiplied concurrent input streams |
| Token and cost shape | 5.9B uncached input, 690M output, 72B cached input reads, ~$165,000 | Cached reads outnumbered output tokens roughly 100:1 |
Three caveats bound the number. Treat the $165K as Bun's reported API-rate costing for a highly parallel migration, since migration pricing needs its own scope, review, and retry assumptions. The Anthropic ownership and pre-release Fable model mean cost and capability were both non-standard, so teams cannot expect the same price-performance from a generally available Claude tier. Quality also remains a separate axis from token spend, because a migration bill only measures how much loop work happened, without showing whether every translated path is production-ready.
Scope explains much of the spread between migration bills because long-tail files require repeated attempts, review, and correction. Parallelism compresses calendar time while expanding repeated context reads. Cosmos Experts define how agents behave, what tools they use, and what events they subscribe to, so teams can share porting instructions across concurrent migration agents. Migration programs evaluating governance controls can also compare enterprise review workflows against Copilot.
The "$20M in Tokens Before ROI" Claim Doesn't Trace to a Source
No primary source in this article supports the claim that companies spend $20 million in tokens before reaching ROI on AI coding agents. Leaders should not use the number as a planning benchmark without a primary source. A narrower auditable framing measures token spend against accepted engineering outcomes and avoids repeating a viral threshold.
A narrower ROI audit should connect spend to concrete engineering signals:
| ROI signal | Measurement layer | Why it matters |
|---|---|---|
| Token spend | Invoice and API usage | Shows cost mechanics by model and loop |
| Accepted work | Merged changes | Ties spend to completed engineering output |
| Rework | Reverted changes | Flags expensive changes that did not hold |
| Review burden | Review time | Captures human cost outside the model invoice |
| Quality impact | Defect escape and maintenance burden | Connects agent work to downstream business value |
The evidence shows that productivity measurement is harder than invoice measurement. METR's randomized controlled trial found experienced open-source developers were 19% slower with AI tools while estimating afterward that they had been 20% faster, a 39-percentage-point perception gap that should make any leader distrust self-reported gains. That result leaves room for positive AI coding rollouts while also showing that perceived speed and measured speed can diverge sharply.
ROI claims therefore need task-level instrumentation. Finance can see spend by invoice, and engineering needs to attach that spend to merged changes, reverted changes, review time, defect escape, and maintenance burden. Without that outcome layer, token dashboards explain cost mechanics but not business value.
How to Set a Token Budget for AI Agents
A token budget for AI agents attributes spend to teams, caps it where hard caps exist, enforces limits inside the running loop, and measures cost per accepted outcome as the planning unit. The FinOps Foundation reports FinOps AI spend management is now part of 98% of FinOps teams, up from 31% two years prior, and names token cost management as the top challenge because invoices use API keys rather than cost centers.
- Set per-developer bands from your own telemetry. Start with observed session-level token data rather than generic seat prices, then separate cached reads, uncached input, and output for each team. The right band matches task mix, model routing, and review policy.
- Attribute spend before controlling it. Structure API keys by team or cost center, which the FinOps Foundation calls the minimum viable control for allocation, and stream per-user token metrics via Anthropic's OpenTelemetry export into your observability stack.
- Use hard stops where vendors offer them, and know where they don't. GitHub Copilot provides a budget stop toggle with alerts at 75%, 90%, and 100%. OpenAI's project-level budget is explicitly a soft budget threshold, and OpenAI continues processing requests after teams exceed it.
- Enforce budgets at runtime, inside the loop. Monthly caps stop next month's bill, while agentic systems need runtime budget controls that decide mid-run whether a workflow continues, narrows, requires approval, or stops.
- Measure spend per accepted outcome. The unit of value is the task outcome: add all agent spend for a task, including failed attempts and escalation, then divide it by changes that meet a quality bar. Teams evaluating this should compare agent governance and review workflows rather than relying on a token-only dashboard.
Runtime enforcement usually requires controls beyond a single tool setting. Cosmos separates those controls across Environments, Experts, and Sessions. Environments define where agents run, Experts define how agents behave and what events they subscribe to, and Sessions turn one-off prompts into auditable, replayable workflows that finance and engineering can inspect.
Run a Loop-Level Cost Audit Before Your Next Renewal
The largest documented savings levers live below the level where procurement negotiates: caching discipline, context editing, sub-agent isolation, tiered routing, and runtime stops. Seat renegotiation addresses model-unit pricing while loop spread remains in the agent architecture. The next step is to pull one team's session-level token data for a week, split it into cached reads, uncached input, and output, then price each turn against your routing tiers. If most spend is re-read context on a frontier model doing mechanical turns, an architecture change can recover budget that a contract change leaves untouched.
Cosmos records session history, workflow state, and context so reviewers can inspect which tools ran, which context entered the workflow, and where fan-out occurred, tying agent activity to those session records for loop-level cost reviews.
FAQ
Related Resources
Written by
Paula Hingel
Paula writes about the patterns that make AI coding agents actually work — spec-driven development, multi-agent orchestration, and the context engineering layer most teams skip. Her guides draw on real build examples and focus on what changes when you move from a single AI assistant to a full agentic codebase.