Save tokens & cache on every LLM — the complete guide

your monthly LLM bill: $720

$720

same work · same model · same quality

−90% WITH PROMPT CACHING

That number is real — one engineer documented cutting their API bill from $720 to $72 a month by changing nothing about what the model did. They just stopped paying full price for the same context over and over.

This guide is everything that matters about spending fewer tokens across Claude Code, OpenAI Codex, and GitHub Copilot: the billing model, prompt caching (the big one), the Batch API, the 2026 compression toolbelt, and copy-paste macOS installs. No padding. Let's go.

There are four prices, not one

Everyone thinks "tokens cost X." Wrong. There are up to four prices, and the 40× gap between the most and least expensive is where every saving in this guide lives:

output

4×

input (fresh)

1×

cache write

1.25×

cache read

0.1×

Relative cost per token (Anthropic pricing shape; output is 3–5× input depending on model).

Three facts fall out of those bars, and they drive everything else:

Output is the expensive one. Generated tokens cost 3–5× input. "Make the model write less" is a bigger lever than it sounds.
Cached input is nearly free. A cache read can cost 10% of fresh input. Re-sending the same system prompt, rules file, or codebase at full price every turn is the single most common waste.
The window is re-read every turn. In any chat or agent session, the entire conversation so far is sent again with each message. A 50-message session re-sends message #1 fifty times. That's why long sessions get expensive and slow.

Fig 1 — every request splits into three zones with wildly different prices. Optimize each zone its own way.

The big lever: prompt caching

When a model reads your prompt it builds an internal KV-cache of that computation. Normally it's thrown away. Prompt caching stores it, keyed on the exact prefix of your prompt — and if your next request starts with the same bytes, the provider skips the recompute and charges you ~10–50% of the price at a fraction of the latency.

The whole trick is in the word prefix. The cache matches from the start of the prompt to the first byte that differs. Change one token early — a timestamp, a counter — and everything after it is a miss. Every rule below follows from this one fact.

Provider	Enable	Discount	Minimum	Lifetime
Anthropic (Claude)	manual — `cache_control` markers	~90% off	1,024–2,048 tok	5 min (refreshed on read); 1-hr at 2× write
OpenAI (Codex)	automatic, free	~50% off	1,024 tok	minutes of inactivity
Google (Gemini)	implicit + explicit	~75% off	32,768 tok (explicit)	configurable TTL

On OpenAI you do nothing — prompts ≥1,024 tokens cache automatically. Your only job: don't break the prefix.
On Anthropic it's manual but deeper (~90% vs ~50%) — and Claude Code places the markers for you, caching your system prompt, CLAUDE.md, and tool definitions. Your job: keep them stable.
Gemini's 32K minimum means it's built for "query this 200-page document repeatedly," not small system prompts.

Cache writes cost extra — 1.25× input on Anthropic (2× for the 1-hour tier). Caching pays only if the prefix is read again: once for the 5-min cache, twice for the 1-hour. Cache the big stable stuff you'll reuse within minutes; never a one-off prompt.

Play with it: the cache calculator

Drag the sliders. This is the real Anthropic math — one cache write (1.25×) plus cheap reads (0.1×) versus paying full price for the context every turn:

Notice what the sliders show: the saving grows with session length, because the one-time write amortizes across more cheap reads. Long agent sessions over a big stable context are exactly where caching goes from "nice" to "90% of the bill."

Four rules that make the cache hit

Stable first, volatile last. System prompt, rules, tool definitions, project files at the top; the user's new message and fresh tool output at the end. The cache matches the longest stable prefix — front-load it.
Never edit the prefix. A timestamp, counter, or session ID near the top silently busts the entire cache on every request. Push dynamic values to the very end.
Keep it hot. Anthropic's cache lives 5 minutes, refreshed on every read. Steady requests keep it alive indefinitely; a long coffee break means paying the write again.
Reuse the exact string. Two prompts that are "basically the same" but differ in whitespace or word order are two different cache entries. Byte-identical or nothing.

And the cache-killer checklist — if your cache reads are near zero on a repetitive workload, it's one of these:

Cache killer	Why
Timestamp / "current time" near the top	changes every request → everything after it misses
Reordered files or context blocks	same content, different order = new prompt to the cache
Regenerated "basically identical" system prompt	one word or whitespace tweak = new cache key
Cold gaps longer than the TTL	entry expires; next request pays the write again

Diagnosis is mechanical: watch cache_read_input_tokens (Anthropic) or cached_tokens (OpenAI) in the usage object. Low reads on a repetitive task = something near the top is changing. Bisect from the top until the reads light up.

The second lever: Batch APIs — flat 50% off

Caching discounts repeated input. The Batch API discounts everything else: both Anthropic and OpenAI run asynchronous batches (results within hours) at 50% off input AND output — and it stacks with caching.

If a job doesn't need an answer in seconds — bulk code review, test generation, data labeling, doc generation, evals — batch it and halve it. It's the only standing discount that touches output tokens.

The third lever: the 2026 compression toolbelt

Here's the cost nobody sees: when an agent runs cargo test and gets 25,000 tokens of stack traces back, that's full-price, uncacheable input — different every run. Command output, exploration, the model's own verbosity, the ever-growing history: each is a separate token sink, and 2026 produced a specialised tool for each. They stack, because they compress different streams.

RTK−60–90%

sink: command output

Rust proxy that compresses dev-command output before the model sees it. git status: 2,000 → 600 tok. cargo test: 25,000 → 2,500. A shell hook rewrites commands transparently — the agent never knows.

brew install rtk
rtk init -g    # hook into Claude Code
rtk gain       # savings stats

Headroom−60–95%

sink: logs · files · RAG chunks

RTK's broader cousin: compresses tool outputs, logs, files, and RAG context through one layer. Ships as a library, a proxy, and an MCP server — wire it into any MCP-capable agent without code changes. Blew up mid-2026 (10K+ stars).

Caveman−65% out

sink: the model's output

Makes the agent answer terse-but-exact — output is the 3–5× token, so this hits the priciest stream. /caveman-compress also shrinks CLAUDE.md ~46%. Bonus: forced brevity improved accuracy 26 pts on some benchmarks.

curl -fsSL https://raw.githubusercontent.com/JuliusBrussee/caveman/main/install.sh | bash
# then: /caveman full

Graphify~71× cheaper

sink: code exploration

Builds a queryable knowledge graph of your repo so the agent looks up "where is auth handled?" in ~1.7K tokens instead of grepping ~123K worth of files into context.

uv tool install graphifyy  # double y!
graphify install

Mem0 / Zep−40%

sink: conversation history

For long-running assistants: replace raw ever-growing history with structured fact retrieval — serve back only what the current task needs. Overkill for a coding session; vital for a persistent agent.

Stack them by sink. Graphify for exploration input · RTK/Headroom for command and log output · Caveman for model output + memory file · a memory layer for history · prompt caching underneath it all for the repeated prefix. Each shaves a different slice; together teams routinely pass 90% total reduction.

Per-tool tactics in 90 seconds

Claude Code

It already caches your system prompt, CLAUDE.md, and tool definitions. Feed it well:

CLAUDE.md under ~200 lines — it ships on every turn, forever.
/compact at natural breakpoints, /clear on task switch. The window is re-paid every turn.
/model — Sonnet for routine work, Opus only for genuinely hard reasoning.
Plan mode (Shift+Tab) before expensive operations — review the plan before paying for execution.
Subagents for verbose work — test runs and log greps stay in the subagent's window; only the summary returns.
Be specific. "Fix validation in auth.ts:login" reads one file. "Fix the auth bug" scans the repo.

OpenAI Codex

Caching is automatic (~50% off the prefix). Keep AGENTS.md stable and lean; stable content first.
Dial reasoning effort down for routine edits — higher reasoning = hidden output tokens = cost + latency.
Exclude junk — build artifacts, lockfiles, vendored code — from what it pulls into context.

GitHub Copilot

Pricing is request/seat-based, so the lever shifts: spend premium-model requests on hard tasks only.
Keep irrelevant files closed — the inline engine sends surrounding context; tighter context is cheaper and more accurate.

Cursor, Windsurf, Cline, Aider — same economics underneath. Cline shows you the raw per-request token bill (the most honest feedback loop anywhere); Aider's /add / /drop is manual window control. The hierarchy never changes: cache the stable, prune the window, right-size the model, cap the output.

macOS installs — everything, copy-paste

# ── Claude Code ──
curl -fsSL https://claude.ai/install.sh | bash      # native (recommended)
brew install --cask claude-code                      # or Homebrew
npm install -g @anthropic-ai/claude-code             # or npm — never sudo!

# ── OpenAI Codex CLI ──
brew install --cask codex
npm install -g @openai/codex                         # SCOPED name — plain "codex" is a dead 2012 package

# ── GitHub Copilot CLI ──
brew install copilot-cli
npm install -g @github/copilot

# ── the toolbelt ──
brew install rtk && rtk init -g                      # command-output proxy
uv tool install graphifyy && graphify install        # code graph (double y!)
curl -fsSL https://raw.githubusercontent.com/JuliusBrussee/caveman/main/install.sh | bash

# sanity check
claude --version && codex --version && copilot --version && rtk --version

Two classic install failures: sudo npm install (breaks npm permissions — fix the prefix or use the native installer) and installing unscoped codex from npm (an unrelated 2012 project that installs silently and does nothing).

The whole guide in eight lines

Exploit the cache — stable-first, byte-identical, never break the prefix. 50–90% off repeated input.
Proxy your tool output — RTK/Headroom. 60–90% off the uncacheable command firehose.
Batch the non-urgent — flat 50% off input and output.
Prune the window — /compact at breakpoints, /clear on switch.
Right-size the model — cheap by default, frontier by exception. Often 5–10× per token.
Cap the output — diffs not files, summaries not transcripts. Output is the 3–5× token.
Be specific & exclude junk — name the file; keep lockfiles and build output out of context.
Measure — /cost, /context, cache_read_input_tokens. A low cache-read ratio on repetitive work means a leaking prefix.

FAQ

Does caching change the model's answers?

No. The cache stores the computation of the prefix, not a canned response. The model runs normally on the full prompt — it just skips redoing work it already did. Same outputs, lower cost, much lower latency (Anthropic measured 11.5s → 2.4s on a 100K-token prompt).

I'm on a subscription, not the API — does this matter?

Yes. Subscriptions meter you with rate and usage limits, and tokens are how you hit them. Every tactic here stretches your allowance further and keeps sessions fast — a bloated window slows the model down whoever is paying.

What's the single fastest win?

Two minutes of work: (1) make sure your big stable context sits at the top of the prompt and never changes, so it caches; (2) /clear when you switch tasks instead of dragging a stale window around. That's most of the savings for most people.

Why do long sessions get slow as well as expensive?

Same root cause: the whole window is re-read on every turn. More tokens = more compute per message = higher latency and higher cost together. Compacting fixes both at once.

References

Prompt caching · Claude API docs — cache_control, pricing, minimums
Prompt caching · OpenAI — automatic, ≥1024 tokens
Context caching · Gemini — implicit + explicit
Manage costs effectively · Claude Code docs
RTK · tool-output proxy
Caveman · output + memory compression
Graphify · code knowledge graph
$720 → $72 with prompt caching · the real before/after

Save tokens & cache on every LLM — the complete guide.