Context compaction for agent loops: when trimming history actually cuts the bill

Every long-running agent eventually hits the same wall. The loop keeps appending tool calls and tool results, the prompt grows on every turn, and sooner or later you face three problems at once: the context window fills up, the model gets less sharp, and the bill climbs. Both major API providers now ship server-side features for shrinking that history mid-run. Anthropic offers tool result clearing and compaction; OpenAI offers compaction in the Responses API. This article explains what each one does, then works through the cost math, because the honest answer to "does compaction save money?" turns out to be "it depends on your price tier and your cache hit rate," and the difference can be large in either direction.

If you have not read our guide to tool-calling costs, the short version is that agent APIs are stateless, so every turn re-sends everything before it. That "history snowball" is what compaction attacks.

Why shrinking context is not only about money

Before the cost math, it is worth knowing why the providers built these features. Anthropic's engineering post Effective context engineering for AI agents describes "context rot": as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases. The post argues that context "must be treated as a finite resource with diminishing marginal returns," and names compaction as "the first lever in context engineering" for long tasks. It also calls tool result clearing "one of the safest lightest touch forms of compaction," since the agent rarely needs to see a raw result again once it has acted on it.

So even when compaction does not lower the bill, it can be the right call for quality. Keep that in mind when the numbers below come out close.

The three tools on offer

Tool result clearing (Anthropic). The context editing documentation describes the clear_tool_uses_20250919 strategy, enabled with the context-management-2025-06-27 beta header. When the prompt grows past a trigger, the API replaces the oldest tool results with placeholder text before the prompt reaches Claude. By default the trigger is 100,000 input tokens and the three most recent tool use/result pairs are kept. You can exclude specific tools with exclude_tools, and set clear_tool_inputs to also strip the tool call parameters. Your client keeps the full, unedited history; the editing happens server-side on each request. A sibling strategy, clear_thinking_20251015, does the same for old thinking blocks, which matters if you run reasoning models in the loop (see our article on reasoning models in agent loops).

Compaction (Anthropic). Instead of deleting content, compaction has Claude write a summary on the server and replaces the older turns with it. There are two flavors. Compaction at a token threshold uses the compact_20260112 edit with the compact-2026-01-12 beta header: when input tokens reach the trigger (default 150,000, minimum 50,000), the API generates a summary, returns it as a compaction block at the start of the response, and continues the turn on the shortened context. On later requests you append the response as usual and the API drops everything before the block. Compaction on demand puts you in charge of when the summary is written, which Anthropic recommends wherever it is available because it can keep recent turns word for word and can run in the background.

Compaction (OpenAI). OpenAI's compaction guide adds a context_management entry with a compact_threshold to an ordinary Responses API request. When the rendered token count crosses the threshold, the server compacts and emits an encrypted compaction item in the same response. That item is opaque and "not intended to be human-interpretable," which is a real difference from Anthropic's readable summary. There is also a standalone /responses/compact endpoint that takes a full context window and returns a compacted one you pass to the next call.

Here is what a threshold-compaction request looks like on Anthropic, with the summarization prompt told explicitly not to call tools. That instruction is there because the docs warn that when tools are defined, the model occasionally calls a tool during summarization and returns a compaction block with content: null.

response = client.beta.messages.create(
    betas=["compact-2026-01-12"],
    model="claude-sonnet-5-5",
    max_tokens=4096,
    system=[{"type": "text", "text": SYSTEM_PROMPT,
             "cache_control": {"type": "ephemeral"}}],
    tools=TOOLS,
    messages=messages,
    context_management={"edits": [{
        "type": "compact_20260112",
        "trigger": {"type": "input_tokens", "value": 100000},
        "instructions": "Summarize the transcript inside <summary></summary> tags. "
                        "Keep open tasks, IDs and file paths. Do not call any tools.",
    }]},
)
messages.append({"role": "assistant", "content": response.content})

The cache_control breakpoint on the system prompt is deliberate, and it is the subject of the next section.

The catch: compaction and prompt caching pull in different directions

Prompt caching makes re-sending history cheap, and compaction rewrites history. Every time you shrink the context, the cached prefix after the edit point is gone and has to be written again. The context editing docs say so directly: tool result clearing "invalidates cached prompt prefixes when content is cleared," which is why the clear_at_least parameter exists. If the API cannot clear at least that many tokens, it skips the edit rather than break your cache for a trivial saving. For compaction, Anthropic's threshold docs recommend a cache_control breakpoint at the end of the system prompt so that, after a compaction, the system prompt is still read from cache and only the new summary is written.

Compaction also has a direct cost. The threshold docs note that compaction "requires an additional sampling step, which contributes to rate limits and billing." The summarization pass reads the whole current context and writes a summary, and those tokens are reported in a separate usage.iterations entry. The top-level input_tokens and output_tokens do not include them, so if your cost tracking reads only the top-level fields, it will undercount every request that compacts. Sum across usage.iterations instead.

Worked example: a 40-turn research agent

Let us price a concrete loop. The agent has a 6,000-token fixed prefix (system prompt plus tool definitions). Each turn adds 4,000 tokens of history, mostly tool results, and writes 400 output tokens. After 40 turns the final prompt is 162,000 tokens. We assume turns run close enough together that the five-minute cache never expires.

We use Claude Sonnet 5.5 at the list prices on Anthropic's pricing page, which match our model table: $2 per million input tokens, $10 per million output tokens, $2.50 per million for five-minute cache writes, and $0.10 per million for cache hits (Sonnet 5.5 and Opus 5.5 cache hits are priced at 0.05 times base input). For compaction we assume a 3,000-token summary, and we price the summarization pass conservatively as uncached input, since the docs do not promise a cache discount on it.

Without caching, the run sends 3.36 million input tokens: $6.72 of input plus $0.16 of output, so $6.88. With caching and the full history kept, it writes 162,000 tokens to cache ($0.405), reads 3.198 million from cache ($0.32), and pays the same $0.16 for output, about $0.88 in total. That is the caching win our earlier guide described, and it is huge.

Now add compaction with a 50,000-token trigger. It fires three times. The loop's own requests drop to about 1.07 million input tokens, mostly cache reads, but the three summarization passes read 156,000 tokens ($0.31) and write 9,000 summary tokens ($0.09). The total comes to about $1.05, roughly 19 percent more than keeping the full history. Raising the trigger to 100,000 means one compaction and about $0.97, still more expensive. On a flat-priced model with a 0.05 times cache-hit rate, re-reading history is so cheap that summarizing it costs more than it saves on a 40-turn run.

Stretch the same agent to 100 turns and the balance shifts. The full history now peaks at 402,000 tokens and the cached run costs about $3.40, because 20.4 million cumulative input tokens add up even at $0.10 per million. Compaction at 100,000 tokens fires four times and brings it to about $2.79, an 18 percent saving, with a context that never exceeds 98,000 tokens.

Where compaction pays off hard: tiered pricing

The math changes completely on models whose price jumps with prompt length. Claude Haiku 5.5 costs $0.10 input and $0.50 output for prompts up to 100,000 tokens, but $0.50 and $2.50 above that, and Anthropic's pricing page is explicit that "a request over the threshold pays the higher prices even when part of its prompt is a cache hit." Cache hits rise from $0.01 to $0.05 per million on the same boundary.

Run the 40-turn agent on Haiku 5.5. With full history and caching, the prompt crosses 100,000 tokens on turn 25, and the last 16 turns pay five times the rate. Total: about $0.19. With compaction at 50,000 tokens, every request stays in the cheap tier and the run costs about $0.06, roughly 70 percent less. At 100 turns the gap widens to about $1.24 against $0.15, roughly 87 percent less.

OpenAI has a similar cliff. The GPT-6.1 Sol model page states that prompts with more than 272K input tokens "are priced at 2x input and cache rates and 1.5x output for the full request," and the OpenAI pricing page lists GPT-6.1 Sol at $2 input and $10 output for short context, against $4 and $15 for long. Setting compact_threshold comfortably below 272,000 keeps a long-running Sol agent on the short-context rates for its whole life.

A practical recipe

Start by trimming at the source: a tool that returns 400 tokens instead of 4,000 beats any server-side cleanup, because it never enters the history at all. Then look at your price tier. If your model has a long-context price step, set a compaction threshold below it with enough headroom for one more large tool result, and you will likely save money on long runs. If your model is flat-priced with deep cache discounts, compact for quality and context limits rather than cost, and set the trigger high enough that it fires rarely. For tool result clearing, set clear_at_least so each clear is big enough to justify the cache rewrite, and use exclude_tools for results the agent must keep, such as a plan or a spec.

Whatever you choose, measure it. Log usage.iterations on Anthropic and the compaction items on OpenAI, and compare cost per completed task, not cost per request. To estimate how prompt size moves the price across current models, try our free Token and Cost Estimator, which uses the same verified price list as our model table.

Context engineering is half craft and half systems design. AI Engineering by Chip Huyen expands on how production LLM applications manage context, evaluation and cost when loops run for dozens of turns.

If you run models on your own machine, we keep a short list of local LLM gear — GPUs, Macs, RAM and storage — with plain-text Amazon links.

Sources

Anthropic, "Effective context engineering for AI agents": https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents. Anthropic context editing: https://platform.claude.com/docs/en/build-with-claude/context-editing. Anthropic compaction overview: https://platform.claude.com/docs/en/build-with-claude/compaction. Anthropic compaction at a token threshold: https://platform.claude.com/docs/en/build-with-claude/compaction-threshold. Anthropic compaction on demand: https://platform.claude.com/docs/en/build-with-claude/compaction-on-demand. Anthropic pricing: https://platform.claude.com/docs/en/about-claude/pricing. OpenAI compaction guide: https://developers.openai.com/api/docs/guides/compaction. OpenAI GPT-6.1 Sol model page: https://developers.openai.com/api/docs/models/gpt-6.1-sol. OpenAI pricing: https://developers.openai.com/api/docs/pricing. Cost figures are our own arithmetic on the list prices above. All sources fetched and link-checked October 20, 2026.