Prompt caching for agent loops: Anthropic, OpenAI and Gemini compared

Agent loops waste money on the same tokens over and over. The system prompt, the tool definitions, the repo summary, the docs you stuffed in on turn one: every later request re-sends them because the big model APIs are still mostly stateless. Prompt caching (Anthropic and OpenAI) and context caching (Gemini) are the discounts that attack that repetition. They do not change the answer the model writes. They change what you pay, and how long you wait, for the part of the prompt that did not change.

This post compares how the three APIs work as of November 6, 2026: what you have to turn on, what a write costs versus a hit, how long a cache entry lasts, and what breaks a prefix in an agent loop. All prices come from the linked official pages fetched that day. They move, so check them before you budget.

Why agents care more than chatbots

A one-shot chat pays for its prompt once. A twelve-turn tool loop pays for the shared prefix twelve times unless something caches it. That is the history snowball in a nutshell, and it is why input usually dominates an agent bill even when the model is only writing short tool calls. Hosted search and fetch make it worse by dumping large result blobs into that history, which is why hosted tool fees and caching belong in the same design conversation.

Caching only helps when the bytes at the front of the prompt stay identical. Move a timestamp into the system prompt, reshuffle the tools array, or rewrite earlier turns, and the prefix hash changes. Then you are back to full price, plus whatever write surcharge your provider charges for the new entry.

Anthropic: opt-in breakpoints, 5-minute or 1-hour TTL

Claude's prompt caching docs make caching an explicit choice on the Messages API. You either set a top-level cache_control field for automatic caching, which moves the breakpoint forward as the conversation grows, or you put cache_control on individual content blocks when you want a fixed prefix such as tools and system text. Managed Agents keeps caching on by default. The cacheable prefix covers tools, then system, then messages, in that order.

Default lifetime is five minutes, refreshed on every hit at no extra charge. A one-hour TTL is available by setting "ttl": "1h" on the cache control; that write costs more. Anthropic's pricing page lists the multipliers: a five-minute write is 1.25× base input, a one-hour write is 2×, and a cache read is usually 0.1×. Claude Sonnet 5.5 and Claude Opus 5.5 are cheaper on reads at 0.05×. In dollars for Sonnet 5.5 that is $2 per million base input, $2.50 per million for a five-minute write, $4 per million for a one-hour write, and $0.10 per million for hits.

Minimum cacheable length on Sonnet 5.5 is 512 tokens. Shorter prompts marked for caching are processed without caching and without an error, so you have to read cache_creation_input_tokens and cache_read_input_tokens in the usage block to know whether anything landed. Automatic caching is the right default for a growing agent transcript. Use an explicit breakpoint on the last *stable* block when the last block is something that changes every request, like a timestamp or the fresh user message, because writes only happen at the breakpoint and lookback only finds entries earlier requests already wrote.

OpenAI: on by default, with write fees on GPT-5.6 and later

OpenAI's prompt caching guide starts from the opposite place: caching is enabled by default for supported models. Cached input is discounted by up to 95 percent versus ordinary input. For GPT-5.6 and later, including GPT-6.1 Sol, a cache write is no longer free. In the Responses API the default "30m" lifetime writes at 1.25× ordinary input, and you can choose "12h" at 2.5×. Reads are 0.1× on most of those models and 0.05× on GPT-6.1 Sol and GPT-6.1 Terra.

OpenAI's pricing table puts GPT-6.1 Sol at $2.00 input, $0.10 cached input, $2.50 for a 30-minute write, $5.00 for a 12-hour write, and $10.00 output per million tokens on short context. Those write rates are the full rate for those tokens, not a surcharge stacked on top of ordinary input. Minimum cacheable prefix on GPT-5.6 and later is 1,024 visible input tokens.

You can leave breakpoint placement to OpenAI (implicit mode) or switch to explicit breakpoints with prompt_cache_options.mode set to "explicit" and prompt_cache_breakpoint on content blocks. Explicit-only mode is useful when the suffix of the prompt is a one-off user question you do not want to pay a write fee for. OpenAI also exposes prewarm via prompt_cache_options.prewarm: true, which writes the prefix without generating output, and a diagnostics path for explaining misses on GPT-5.6 and later. Older models use prompt_cache_retention (in_memory or 24h) and have no separate write charge.

Gemini: implicit by default, explicit only on older models

Google's context caching docs describe two mechanisms. Implicit caching is on for Gemini 2.5 and newer. You do not configure it; if the request hits a cache, the discount shows up. Minimum input size is 4,096 tokens on Gemini 4 Argon and the Gemini 3.x Flash / Pro models listed on that page, and 2,048 on Gemini 2.5 Flash and Pro. Google's wording on the caching page is careful: implicit caching has no cost-saving *guarantee*. When a hit does land on Gemini 4 Argon, the model overview and pricing page both price cached input at $0.10 per million tokens through November 30, 2026 (95 percent off the $2.00 introductory input price), rising to $0.20 when the introductory rate ends on December 1.

Explicit caching, where you create a cachedContents object and pass it into later generateContent calls, still exists for Gemini 3 and 2.5 models under the v1beta caching API. It is not supported on Gemini 4 Argon and newer; those models are implicit-only. On models that still allow explicit caches, Gemini pricing adds a storage line. Gemini 3.8 Flash, for example, lists cached input at $0.075 per million tokens through December 31, 2026, plus $0.50 per million tokens per hour of storage for the TTL you set. That storage charge is the part people forget when they pin a large corpus for a whole day.

One agent-specific detail on Gemini 4: the caching docs say the full previous request is cached only when the new request adds at least 2,050 tokens (model output, thought and function results all count). Smaller additions re-process the last 2,050 tokens of the previous request at the standard input rate. Most tool turns clear that bar, but a tiny "ok, continue" turn might not.

A worked 12-turn prefix

Take the same shape of task on all three APIs so the rate cards are comparable. A coding agent has a 50,000-token stable prefix: system instructions, tool schemas and a compressed repo map. It runs twelve requests in a few minutes, well inside every provider's short TTL. Ignore the growing transcript for a moment and price only that prefix, using Sonnet 5.5, GPT-6.1 Sol and Gemini 4 Argon at their November 2026 list rates.

Uncached, twelve reads of 50,000 tokens at $2 per million is 600,000 tokens, or $1.20.

On Claude Sonnet 5.5 with a five-minute write: one write at $2.50 per million is $0.125, and eleven hits at $0.10 per million are $0.055, for $0.18 total. On GPT-6.1 Sol with a 30-minute write the arithmetic is the same dollars: $0.125 to write, $0.055 to read eleven times, $0.18 total. On Gemini 4 Argon, if all eleven follow-ups hit implicit cache, the first request pays the ordinary $0.10 for 50,000 tokens and the eleven hits pay $0.055, for about $0.155, with no separate write fee on the implicit path.

So for a hot agent loop that keeps the prefix identical, all three land in the same neighborhood: roughly an 85 percent cut on the repeated prefix versus paying full input every turn. The gaps show up in the failure modes. Claude and OpenAI charge you up front to write; a loop that breaks the prefix every turn burns write fees and never sees the cheap reads. Gemini Argon will not let you pin an explicit cache for a shared corpus the way Flash still can, and its implicit hits are not contractually guaranteed the way a Claude breakpoint or an OpenAI explicit write is.

Add the growing transcript back in and the same pattern continues, as long as each request appends rather than rewrites. Automatic caching on Claude and implicit mode on OpenAI both exist for that case. Compaction is the rival lever: it shrinks history for quality and sometimes for cost, but it also changes the prefix and can destroy cache hits. That trade-off is the whole point of our context compaction post.

What actually breaks the cache in an agent

Keep tools, system text and earlier turns byte-for-byte stable. Prefer appending messages over editing old ones. Put volatile bits (clocks, request ids, per-user junk) after the breakpoint, not before it. On Claude, changing tool definitions, toggling web search or citations, or changing thinking / effort settings can invalidate part or all of the cached prefix; the caching docs have the full invalidation table. On OpenAI, model, tools, parallel_tool_calls, structured-output format, reasoning effort and verbosity all sit in the rendered prefix, and compaction replaces earlier content so reuse drops after a compact. On Gemini, put the large shared content at the front and send similar prefixes close together if you are relying on implicit hits.

Log the usage fields every turn: Anthropic's cache_read_input_tokens / cache_creation_input_tokens, OpenAI's cached_tokens / cache_write_tokens, Gemini's total_cached_tokens. A dashboard that only charts total input will hide a cache that stopped hitting last Tuesday.

If you want a systems-level treatment of serving trade-offs and inference optimization beyond the vendor rate cards, AI Engineering by Chip Huyen is the book we point people at on the cost side of production LLM work.

Practical picks

For a tight agent loop that finishes in minutes, Claude's five-minute automatic caching and OpenAI's default 30-minute writes are the straightforward knobs, and Sonnet 5.5 / GPT-6.1 Sol currently price that pattern almost identically at $0.10 per million on hits. Use Claude's one-hour TTL or OpenAI's 12-hour TTL when a side agent or a human approval gate regularly sits idle longer than the short window; you pay more to write and less to miss. On Gemini 4 Argon, design for implicit caching: stable prefixes up front, turns big enough to clear the 2,050-token add threshold, and no dependence on cachedContents. Keep explicit caches for Gemini 3.8 Flash-style workloads where you own the TTL and are willing to pay storage.

Caching will not fix a bloated tool result or a prompt that changes every call. It will stop you from paying full freight for the part that did not change. In an agent, that is usually most of the bill.

Sources

Anthropic: Prompt caching, Anthropic: Claude API pricing, OpenAI: Prompt caching, OpenAI: API pricing, Google: Context caching, Google: Gemini Developer API pricing, Google: What's new in Gemini 4 Argon.