Token Savings in Practice

TLDR; Do not generate what you do not need. At all.

So, Rule 0. Do not generate what you do not need. At all.

Every method for spending less on agentic development sits downstream of that line. Model choice, caching, reasoning budgets, fan-out — all of it comes after.

The biggest line on the bill is work you never should have asked for: a full spec written before the core idea even holds; the whole app coded up with no architecture, contracts, or schema behind it; a thousand lines one-shot out of a loose prompt, thrown away, and the loop starts over and does it again. One-shotting and restarting burns more than any optimization saves.

Go from small to big. Reveal the work in layers, condensed first and detailed later — architecture before any code, the contracts pinned down before the implementation, a tight plan before you let it sprawl.

The visible small case is the vision tax: a screen-watching agent that screenshots every click to rename a batch of files, a couple thousand tokens a frame, doing what a shell command does for free. The same mistake at scale is the spec, the app, the discarded thousand lines.

Shape the session first

Most of the savings I actually trust come from how a session is built.

I do the thinking up front on the strong model — the plan, the architecture, the decisions that are easy to get wrong — and then hand the execution off cheap: a fresh, small context that just walks the plan. The reasoning is already spent by then, so the window stays small and a weaker model gets through it. Hand a hard task straight to the cheap model and it fails the first pass, the window fills with its flailing, and the cost climbs past what the strong model would have run in the first place. The catch is to write the plan more verbosely than feels comfortable, because the executor is weaker and fills gaps badly.

The tier gap is what makes that split worth the trouble. Two months of my own logs — 51 sessions, 5.79 billion tokens — and the spread is lopsided: two of the five models I touched are 96% of everything I ran, both of them top-tier. The mid and cheap tiers barely show up. Two sessions made it concrete: the same work, about 630 million tokens each, and at list rates one ran roughly double the other. Nothing moved but the tier.

The handoff is messier than “hand it off” makes it sound. Sometimes the agent spawns a subagent on its own and it lands on a cheaper model; sometimes that same subagent quietly inherits the strong one and I’ve saved nothing — it comes down to the tool and the settings, and it’s mostly out of my hands. So when I actually want the cheap pass, I stop trusting the automatic route and do it by hand: open a separate chat, paste the plan, run it there. Which model spawns is the tool’s call. Splitting the work into a plan clean enough to hand off stays mine.

The rest is session hygiene:

• One task per session. Plan, implement, check, fix, PR — each stage gets its own clean context. A session that hangs onto the whole history of an argument pays to re-read it every turn.

• Carry forward clean output only. A finished plan or a working diff travels between stages; the transcript stays behind. When something goes wrong I edit the original prompt and re-run clean, because a correction chain just pays to re-read every wrong turn, and the model drifts further each round.

• Hand the agent a map. Navigation docs, a code graph — something that says where things live, so it stops reading half the tree to answer one question. I wired a code graph into a large project and it works, though whether it saves tokens I can’t prove on my own numbers yet, so I won’t quote a percentage I haven’t earned.

Caching, and how not to break it

93.8% of every token in those logs was a cache read — the model re-reading a stable context on every turn. Caching is the one optimization that paid off mechanically and hard: running without it would have cost me roughly five times as much, and it saved about 79% as it stood.

It only holds if the context is ordered right, most stable first and least stable last. Tools, system prompt, project rules go at the front. The live query and anything dynamic go at the very end. Put something that changes early in the prompt and you invalidate everything sitting after it, on every call — the cache prefix only reaches as far as the first thing that moved.

What quietly breaks it is always something small — a datetime.now() baked into the prompt, a tool list that comes back in a different order each run, a json.dumps with unsorted keys. None of it looks like a bug, which is exactly how it runs untouched for weeks. The tell is a cache-read count sitting at zero across prompts that plainly share a prefix — when you see that, something upstream is changing that has no business changing.

Savers that cost money

Some tools that advertise savings will quietly lose you money while the dashboard cheers you on. RTK is the clean example — a CLI proxy that trims the output of common dev commands and reports the difference as tokens saved. A colleague on my team ran it across 8,765 commands and showed me the summary: 46.4 million tokens saved, 92.7%. It’s a gorgeous number, and it’s gross.

Look one level down and it comes apart. Almost all of that 46.4M is a single command — rtk find, 31.9M by itself. His most-frequent one, rtk grep at 1,255 calls, saved 20.7%; rtk read saved 9.2%. And the tokens RTK strips off an output don’t leave the task — when the agent still needs what got cut, it fetches the data another way, so you pay for the trim and the re-fetch both. The real saving on the finished job comes in under the dashboard, and on the low-value commands it can go negative.

Judge any saver on one number: the total spend to finish the task. Its own dashboard is measuring the thing it was built to make look good.

The reasoning dial

Reasoning barely registered as output in my logs — 0.6% of the total. That number flatters me and shouldn’t, because the thinking is folded into the price and never broken out as its own line on the bill. I’ll be straight about what I did here: I never touched the effort dial and only looked at the reasoning spend in hindsight. Going forward I’m turning it down on simple work, where it earns nothing.

The external research pushes this harder than my own logs do. A July 2025 paper (arXiv 2507.04023) found reasoning models burning up to 18 times the tokens on basic arithmetic, and sometimes scoring lower than the plain model while they did it. On easy work the reasoning budget is mostly waste, and you pay for it at output rates whether or not it ever changed the answer.

The swarm

One session spun up 14 agents in about 13 minutes, pressure-testing a piece of writing from every angle at once. That one earned it. The counter put the run at 785,000 tokens — a number I’ve stopped trusting. Back in the raw logs, the tool’s live tally and the transcripts were off by a lot, and not in my favor; what fan-out actually costs is a measurement problem of its own, and one for another day. Either way, it was fourteen separate contexts, each one loaded and paid for. Across the whole two months, subagents were 48% of my messages and only 15% of the cost — which reads like a bargain right up until you look at the mechanics.

The hidden cost lives in the mechanics. Every agent re-reads the big shared context as a fresh cache write, and a write runs 10 to 20 times the price of a read. Anthropic has said its own multi-agent research system spends around 15 times the tokens of a normal chat, and called it a poor fit for most coding. Fan-out earns its keep on genuine breadth-first search, where the agents explore different ground. Point it at an ordinary build and it just multiplies the same context you’re already paying to hold.

The runaway case is real and the built-in brakes are soft. Cursor can switch off overages, Codex hands its autonomous mode a token budget, and cloud billing alerts show up hours late — long after the money is already gone. The only hard cap I trust is one you build yourself: a check that runs in front of the request and can refuse it before it ever reaches the model.

Which tools I’d stake a number on

People push a new token-saver at me most weeks. Worth saying first: the things that actually cut my bill — caching, matching the model to the job, one clean task per session — aren’t tools anybody sold me. They’re the habits above. The pitched tools are a shorter list, and mostly I’m still waiting on them.

• Code graph. Wired into a big project, working, still short of a savings number I’d defend. Promising, and unproven.

• Caveman. A CLI output trimmer, same box as RTK. I haven’t run it through my own tasks, so no rating — measure it the way I measured RTK, on the spend to finish the job, with the percentage it prints for itself set aside.

• Routers and fan-out packs. Model routing and multi-agent frameworks sold as a general win, usually on a leaderboard the seller built. Marketing until someone independent reproduces it.

And one that isn’t a product, just a switch left in the wrong position: reasoning turned on for work that doesn’t need it. The outside research already shows it burning tokens for nothing. Turn it down.

Cost per finished thing

The number I watch is cost per finished thing — per merged PR, per shipped document — never tokens on their own and never cost per session. A session that spends more and finishes clean beats three cheap ones that each fall over halfway. So I fail fast and cap retries at one: if the second attempt doesn’t land, the approach is wrong, and a third run just buys a pricier version of the same wrong approach.

Read the bill back and it turns into a record of how I worked. The expensive lines all have the same shape: I asked for the big thing before the small thing under it was solid — the spec before the idea held, the app before the architecture, a thousand lines before there was a plan to hang them on. Everything that came in cheaper came from building the small thing first and letting it hold before asking for anything bigger.