When reasoning models help (and waste tokens) in agent loops

Agent builders discover reasoning models the same way they discover caching: after an invoice that looks nothing like the per-token rate card they used in the prototype. A planning step that used to cost a few thousand output tokens suddenly burns tens of thousands of hidden reasoning tokens. Latency stretches from a second to a minute. Quality goes up on the hard turns and stays flat on the easy ones. This guide explains what those tokens are, how the three major APIs expose them in October 2026, and a practical rule for when they belong inside an agent loop and when they are just expensive silence.

What “thinking” actually costs

Across providers, the important fact is the same: internal reasoning is billed as output (or as part of the output budget), even when you never see the raw chain of thought.

Anthropic's thinking steering and cost documentation states that you are billed for tokens Claude uses while thinking, for thinking blocks that remain in context from prior turns, and for ordinary text output. Summarized or hidden display modes change what you *see*, not what you *pay*: the billed output count is the full internal process. The response exposes the split under usage.output_tokens_details.thinking_tokens, and Anthropic notes that this breakdown is for observability while output_tokens remains the inclusive total used for billing.

OpenAI's reasoning models guide describes the same pattern for reasoning tokens: they are not returned as readable text through the API, they occupy context window space, and they are billed as output tokens. Usage reports them under output_tokens_details.reasoning_tokens. OpenAI also warns that if generation hits max_output_tokens before any visible answer is produced, you can still pay for input and reasoning tokens and receive an incomplete response.

Google's Gemini thinking documentation and pricing page treat thinking tokens as part of the output price. For Gemini 4 Argon, input, thinking, and output share a single roughly one-million-token context window, and max_output_tokens applies to the combined total of thinking and visible output, so a tight output cap can truncate an answer that spent its budget thinking.

None of this is a reason to avoid reasoning models. It is a reason to treat “turn on maximum thinking for every agent step” as a product decision with a measurable bill, not as a free quality upgrade.

How the three APIs ask the model to think

The control surfaces differ, but they all steer *how often* and *how deeply* the model reasons rather than promising a fixed token count.

Anthropic's current default path is adaptive thinking. The thinking overview and the steering page explain that Claude decides per request whether to think, and that effort (low, medium, high, xhigh, max, depending on the model) is soft guidance for how willing it should be. Manual thinking.type: "enabled" with budget_tokens still works on older Claude generations and remains useful when you need a predictable budget, but Anthropic documents it as deprecated on Claude 4.6 and unsupported on Claude 4.7 and later. For agent loops, the important adaptive feature is interleaved thinking: Claude can reason between tool calls inside one assistant turn without a special beta header on current adaptive models. Anthropic also notes that in a tool-use loop, the first request after new user input usually carries most of the reasoning, while follow-ups that only process tool results can skip thinking even at high effort.

OpenAI exposes reasoning.effort on the Responses API. The reasoning guide lists model-dependent values such as none, minimal, low, medium, high, xhigh, and max, and describes lower effort as favoring speed and cost while higher effort spends more tokens for harder work. OpenAI's reasoning best practices still frame the useful split as “planners” versus “workhorses”: use a reasoning model for ambiguous planning and high-stakes decisions, and a cheaper GPT-class model for well-defined execution steps. For multi-turn agents, the same guide family documents preserving opaque reasoning items across calls (previous_response_id, conversation state, or replaying response history) and selecting reasoning.context modes such as current_turn versus all_turns on supported models so later turns can reuse compatible reasoning without exposing the raw text.

Gemini steers with thinking_level (minimal, low, medium, high, depending on the model). The thinking docs default several current models, including gemini-4-argon and gemini-3.8-flash, to thinking on at medium, and they recommend lowering the level for simple retrieval or classification rather than starving max_output_tokens. Thought summaries are optional; signatures that keep multi-turn continuity are not a free-text dump of the private chain of thought.

A worked cost sketch for one hard turn

Take Claude Sonnet 5.5 at the list prices we track in our model table: $2 per million input tokens and $10 per million output tokens. Imagine a single agent planning turn that sends 8,000 input tokens (system prompt, tools, and a short history) and would have written about 400 tokens of tool calls or plan text without deep reasoning.

Without thinking, that turn is roughly $0.016 for input and $0.004 for output, about two cents. With adaptive thinking at high effort, suppose the model spends 8,000 thinking tokens before those 400 visible tokens. Output jumps to 8,400 tokens, or about $0.084, and the turn is suddenly closer to ten cents. That is still cheap once. Inside a twenty-turn loop where five turns “feel hard” to a max-effort setting, you are looking at roughly an extra $0.40 of thinking on a run that used to cost well under a dollar. At ten thousand runs a day, the same habit is thousands of dollars of reasoning that may not change the final tool sequence.

OpenAI's guide recommends reserving on the order of 25,000 tokens of headroom for reasoning plus visible output when you first experiment, precisely because incomplete responses can spend reasoning budget and still return nothing useful. That buffer is a latency and cost budget, not a suggestion to spend it every turn.

When reasoning earns its keep inside a loop

Reasoning helps when the turn's failure mode is *judgment*, not *typing*.

Planning and decomposition are the clearest wins. OpenAI's best-practices page explicitly calls out multistep agentic planning: a reasoning model drafts the strategy and assigns cheaper models to execute the pieces. Anthropic's older essay Building effective agents makes the complementary point that routing easy work to smaller models is the default cost lever; thinking is the other side of that coin, applied only where the plan itself is the hard part.

Ambiguous tool choice is another. If the agent has overlapping tools (search_docs, search_code, search_tickets) and weak signals about which one applies, a short thinking pass before the first tool call often prevents a cascade of wrong fetches that would have cost more in repeated history than the reasoning tokens did. Interleaved thinking between tool results is valuable for the same reason: the model can reinterpret a surprising API payload before it doubles down.

High-stakes commits belong here too. Approving a refund, merging a migration, or sending an external email is where Anthropic's effort table and OpenAI's “accuracy and reliability” column both point you toward deeper reasoning. Those steps are rare in a healthy loop; paying for them is usually cheaper than a human cleanup.

Debugging and policy-heavy review are the last cluster. Both providers document strong results on dense, ambiguous documents and on code review style tasks. An agent that reviews a diff or reconciles conflicting policy snippets once per run is a better customer for high or xhigh effort than an agent that classifies each inbound email.

When reasoning wastes tokens

Most agent turns are not those cases. Routing, extracting a field from a JSON tool result, confirming that a file write succeeded, paginating a list, or emitting a structured status update rarely improve with thousands of private reasoning tokens. Anthropic's steering docs say that at every effort level Claude may skip thinking on simple queries, and that follow-up turns that only digest tool results often skip even when effort is high. That is the behavior you want; force-max settings fight it.

Changing effort or thinking configuration mid-conversation can also erase prompt-cache savings. Anthropic documents that the resolved effort value is part of the prompt, so changing top-level effort between requests invalidates cache breakpoints the same way changing a legacy budget_tokens value does. For a caching-heavy agent, pick a stable default effort for the conversation and steer exceptional turns with per-message effort (where supported) or short natural-language instructions instead of flipping the global dial every step. Our earlier guide to tool-calling costs covers why cache stability matters; thinking configuration is another prefix you must keep still.

Finally, do not confuse “the model thought a lot” with “the agent made progress.” If your traces show large thinking_tokens or reasoning_tokens on turns that still call the wrong tool, fix the tool descriptions and schemas first. Reasoning cannot rescue a confusing interface; our craft posts on writing tool descriptions and handling tool errors are cheaper than a higher effort level.

A practical default for October 2026

Start the agent on a capable non-maximal setting: Anthropic medium effort (or the model's documented default), OpenAI medium reasoning effort on a Responses-API model you already trust for tools, or Gemini medium thinking level. Log, per turn, whether a thinking or reasoning block appeared, how many thinking or reasoning tokens were billed, which tool was chosen, and whether a human would call the turn a planning step or an execution step.

Then raise effort only on the turn types that fail. Many teams end up with a two-model loop that matches both Anthropic's routing advice and OpenAI's planner/workhorse split: a reasoning-capable model for the first plan and for rare commit steps, and a cheaper model such as Claude Haiku 5.5, GPT-6 Luna, or Gemini 3.8 Flash for the mechanical middle. Our model selection guide walks through how to pick those pairs from current Artificial Analysis scores and list prices.

If you need a hard ceiling, use the provider's output caps rather than hoping the model will be thrifty. Anthropic bounds cost with max_tokens while effort only shapes how much of that budget goes to thinking. OpenAI's max_output_tokens covers reasoning plus visible output together. Gemini's max_output_tokens does the same, which is why Google tells you to lower thinking_level instead of setting a tiny output limit when you want less reasoning.

Reasoning models are not a different product category from “agent models.” They are ordinary models with an expensive internal scratchpad. Use the scratchpad when the next tool call is genuinely uncertain. Leave it closed when the next action is obvious. That single habit usually saves more money than switching providers, and it is entirely under your control.

To see what the scratchpad is doing under the hood — attention, training objectives and how a transformer actually generates tokens — Build a Large Language Model (From Scratch) by Sebastian Raschka walks through an implementation from scratch. That background makes the effort dial feel less like a magic knob and more like a budget on internal compute.

If you run models on your own machine, we keep a short list of local LLM gear — GPUs, Macs, RAM and storage — with plain-text Amazon links.

Sources

Anthropic thinking overview: https://platform.claude.com/docs/en/build-with-claude/thinking. Anthropic thinking steering and cost: https://platform.claude.com/docs/en/build-with-claude/thinking-steering-and-cost. Anthropic extended thinking (legacy manual budgets): https://platform.claude.com/docs/en/build-with-claude/extended-thinking. Anthropic effort: https://platform.claude.com/docs/en/build-with-claude/effort. Anthropic pricing: https://platform.claude.com/docs/en/about-claude/pricing. Anthropic, “Building effective agents”: https://www.anthropic.com/engineering/building-effective-agents. OpenAI reasoning models: https://developers.openai.com/api/docs/guides/reasoning. OpenAI reasoning best practices: https://developers.openai.com/api/docs/guides/reasoning-best-practices. OpenAI pricing: https://developers.openai.com/api/docs/pricing. Google Gemini thinking: https://ai.google.dev/gemini-api/docs/thinking. Google Gemini pricing: https://ai.google.dev/gemini-api/docs/pricing. Artificial Analysis model leaderboard (for the companion selection guide): https://artificialanalysis.ai/leaderboards/models. All primary docs fetched and link-checked October 19, 2026.