Earlier this year, the conversation in software was all about maximizing agent output. Tokenmaxxing. Jensen Huang telling engineers to spend $250K a year on tokens. Every benchmark was about throughput: how much code could your AI generate per hour?
That mood has shifted fast.
Uber reportedly used its entire 2026 AI coding tools budget in four months. Microsoft started pulling back internal Claude Code licenses. Amazon just scrapped an internal AI leaderboard after employees started running up token costs to climb the rankings instead of shipping useful work. Across the industry, orgs are staring at massive AI bills and wondering where the ROI went.
In our recent research, we found a big part of the answer: teams are generating changes faster than ever but struggling to validate and ship them at the same pace. The delta between code written and code in production has never been wider, and it carries a hefty price tag.
In this issue, we’re digging into why teams are paying more than they should to move code into production. We’ll preview some new findings from an upcoming update to our State of Software Delivery report, show how moving validation into the inner loop of local, agent-driven development can dramatically reduce delivery time and cost, and give you practical steps to get started.
Merge efficiency (and why it matters)
One of the most interesting findings from the recent State of Software Delivery was that median throughput increased 15% on feature branches but declined 7% on main branches. AI makes writing code trivial, but it adds complexity downstream when those changes hit the shared codebase.
We wanted to dig into what that burden actually looks like for a typical team. How many cycles does it take to get a piece of code into production?
To find out, we looked at the ratio of feature branch runs to main branch runs across CircleCI in March 2026. This gives us an imperfect but directionally useful proxy for the total CI overhead required to ship a single change.
Here’s what we found:
For the median org, every main branch workflow is accompanied by 3.9 workflow runs on a feature branch. At the mean, that number rises to 8.6. At the p95 level, it rises to 23.7.
Using main branch workflows as a rough proxy for a merged change, that means every production deploy requires 5+ total CI runs (~4 on the feature branch and 1 on main) to complete.
Main branch workflows also fail 20-30% of the time, adding at least one more cycle when they do.
In a human-paced development context, these numbers wouldn’t be terrible. A developer pushes a change, waits a few minutes for feedback, and iterates through a couple of build-test-fix cycles before merging to main and wrapping up the workday. That’s how you get to the canonical advice that “everyone pushes to main daily.”
In an agentic world, merge efficiency matters a lot more. When full CI runs take 5-10 minutes to deliver feedback, the agent loses context. Fixing the failure means starting a new cycle: reloading context, re-examining the change, potentially redoing work that was already completed.
That not only derails productivity but also creates massive cost inefficiencies. At 5+ CI runs for every shipped change, the math gets dire quickly.
Where costs pile up
Let’s make this concrete.
Imagine an AI agent is tasked with adding a new feature to a medium-sized codebase. The full repository context, including system instructions, tool definitions, and source files, comes to about 200,000 tokens.
Modern frontier models use prompt caching to make reading massive contexts affordable. The first time the agent reads your codebase, it’s a “cold write” at a premium price. On Opus 4.7, the cost is $6.25 per million tokens. On subsequent steps, the model reads from a warm cache at a 90% discount: just $0.50 per million tokens.
But (as usual) there’s a catch: prompt caches expire after about 5 minutes of inactivity.
Here’s how that plays out in a typical delivery workflow:
1. The agent loads the codebase (200,000 tokens) and generates the code change. API cost: ~$1.28 (plus output tokens).
2. The agent pushes the change, triggering a CI pipeline. The CI platform queues the job, spins up a fresh VM, pulls dependencies, compiles the project, and runs the full test suite. The pipeline consumes about 5 minutes of billable compute across its parallel jobs. But the total round trip from push to feedback, including queue time and provisioning, stretches well past the 5-minute mark.
3. The prompt cache goes cold. By the time CI reports back, the cache window has closed. The cache is wiped.
4. The build fails on a minor syntax error. The agent needs to fix it, but the cache is gone. It pays the full cold-write price to reload the codebase, and the context has grown: the previous attempt’s output and the CI build logs have pushed it from 200,000 to roughly 230,000 tokens. API cost: ~$1.44. Then the agent regenerates the fix from scratch, burning another round of output tokens at $25 per million.
5. The agent pushes the fix. Your CI provider spins up another fresh environment and runs the full suite again. Another 5 minutes of billable compute.
The merge efficiency data shows it takes an average of 5+ total CI runs to ship a single change. And with each cycle, the context grows: the agent retains the full session history, so each retry passes the original codebase plus every previous attempt along with its CI build logs. By run 5, input context can double.
In this scenario, input tokens that should cost under $2 with a warm cache balloon to over $13 across 5 cold reloads, purely because the feedback loop is slower than the agent’s cache. Add output token rework at $25 per million and $1-3 in CI compute, and the total cost per change runs to around $25.
Now imagine this at scale. A 50-developer team at agentic pace ships about 3,000 changes a month (roughly 3 per developer per working day). At ~5 CI runs per change, that’s 15,000 pipeline runs and nearly $1 million per year in token and compute costs. At 500K tokens (common for larger production codebases), it crosses $1.5 million. And with the continued increases in change volume that our data shows, those numbers climb every quarter.
Most of that spend is going to cycles that shouldn’t exist.
Validation in the inner loop
Scenarios like this play out because all of the validation is happening in the outer loop: CI pipelines, shared infrastructure, billable compute. CI is the right place to catch integration failures, security issues, and anything that requires a full environment. But when a syntax error or a failing unit test has to round-trip through a full pipeline run before your agent gets the signal, you can quickly accumulate a ton of unnecessary costs.
What if those cheap failures could be caught earlier, before they ever reach CI? The way to do that is to move validation into the inner loop, where agents iterate locally before the push.
In a healthy system, the inner loop handles the basic checks so the outer loop only runs on code that’s ready for deeper validation. Without a quality gate before the push, your outer loop becomes a bottleneck: flooded with changes, catching failures that range from trivial to critical, burning compute and tokens on every cycle.
The path forward is to raise the quality bar in the inner loop. Validate before the push. Catch the lint failures, the broken tests, the syntax errors while the agent still has context and the fix costs almost nothing.
Introducing Chunk sidecars
Moving validation into the inner loop means having an environment that can run real checks fast enough for an agent to act on the results. To get there, we recently released Chunk sidecars: sandbox environments spun up by the Chunk CLI that mirror your CI stack.
Rather than waiting for a full pipeline run to surface a failing unit test or a syntax error, the sidecar runs the validation checks you’ve configured: linting, unit tests, build validation, or whatever your stack requires. Failure feedback comes back to the agent while it still has context. The agent fixes the issue and the hook fires again, repeating until the checks pass and the change is ready to push.
This keeps the feedback loop tight enough that agents never lose context between a failure and its fix. To measure what that’s actually worth, we took real failures from the chunk-cli CI pipeline and ran them two ways: through CI as normal, and through Chunk sidecars using microbuilds. We measured compute time and token consumption across four pipelines with warm snapshots.
Here’s how those numbers affect the costs we saw in the scenario introduced earlier:
Input tokens: $13 → under $2. The 27-second feedback time keeps the agent inside the 5-minute cache window. Five cold reloads become one cold load plus four warm cache reads.
Output tokens: 5 full regenerations → targeted fixes. The agent gets a focused failure signal instead of a full CI log dump, so it applies a small fix instead of rebuilding context and regenerating the entire change.
CI compute: 5+ runs → 1–2. Fifteen to twenty minutes of billable compute per change drops to a single pipeline run on code that has already passed its checks.
The change that was costing ~$25 in tokens and compute drops to around $6. For the 50-person team from our model, that takes annual token and compute costs from approximately $900k to around $200k. Total savings for a 50-person team: $700k or more.
Stop wasting your delivery dollars
Concerns over runaway AI spend are volume problems at their core. When you scale agentic workflows without changing the feedback architecture underneath them, costs accumulate with every additional change.
Teams who have closed this loop are already separating from the pack, and that difference is widening every quarter.
If you want more return on every token and every pipeline run, look at where your validation is actually catching failures today. If most of them are happening in CI, after a push, there’s a better way.
Chunk sidecars are available now on all CircleCI plans, including free. Install the Chunk CLI with brew install CircleCI-Public/circleci/chunk, run chunk init in your project, and start a sidecar session from your agent. The setup takes minutes, and the savings start on the first build.