Why Your Agent Costs More Than You Think: A Practical Guide to Tool-Calling Costs

The first invoice from a production agent is often a surprise. A single prompt to a model is cheap and easy to price: count the tokens in, count the tokens out, multiply by the rate card. An agent is different. It runs in a loop, calls tools, reads their results, and calls the model again, and every one of those calls carries baggage from the calls before it. This guide breaks down where the money actually goes when an agent uses tools, walks through a realistic example with current list prices, and covers the two discounts that make the biggest difference: prompt caching and batch processing.

Tool definitions are input tokens, every single turn

When you give a model tools, you send their definitions with the request: each tool's name, description, and JSON Schema for its parameters. Those definitions are not free metadata. They are part of the prompt, and you pay for them as input tokens. Anthropic's tool use documentation also notes that when tools are used, the API automatically adds a special system prompt that enables tool use, and it lists the extra token count for each model. OpenAI's function calling guide likewise explains that function definitions count against the model's context and are billed as input tokens.

This matters because definitions are re-sent on every turn of the loop. A generous set of ten well-documented tools can easily run to a few thousand tokens. If your agent takes ten turns to finish a task, you pay for those definitions ten times. The practical lesson is to give each agent only the tools it needs for its job, and to write descriptions that are clear but not padded. Our free JSON Schema to Tool Definition converter shows you exactly what a schema becomes in each provider's format, which makes bloated definitions easy to spot.

The history snowball

The second cost driver is less obvious. Most model APIs are stateless: each request must contain everything the model should know. In an agent loop, that means turn two re-sends turn one, turn three re-sends turns one and two, and so on. Every tool call the model makes and every tool result you send back joins the history and is paid for again on each later turn.

Tool results are frequently the biggest items. A web page, a database query, or a file listing can be thousands of tokens. If your agent fetches a long document on turn two and finishes on turn ten, that document is billed roughly nine times. The fix is usually not a cheaper model but a smaller result: trim tool outputs to the fields the model needs, summarize long documents with a cheap model before passing them on, and paginate rather than dumping everything at once.

A worked example

Let us price a modest agent with real numbers. Suppose it has a fixed prefix of 5,000 tokens: about 3,000 tokens of system prompt and about 2,000 tokens of tool definitions. It takes ten turns to complete a task. Each turn adds about 1,500 tokens to the history (the model's tool call plus the tool's result), and the model writes about 300 output tokens per turn.

On turn one, the input is 5,000 tokens. On turn two it is 6,500, on turn three 8,000, and on turn ten it reaches 18,500. Added up across all ten turns, the agent sends 117,500 input tokens and writes 3,000 output tokens. Notice that the task itself only generated about 15,000 tokens of new material; the other hundred thousand are repetition.

At Claude Sonnet 5.5's list price of $2 per million input tokens and $10 per million output tokens, taken from Anthropic's pricing page, that run costs about $0.235 for input and $0.03 for output, or roughly $0.27 in total. Input is nearly 90 percent of the bill. That pattern, where input dwarfs output, is typical of tool-using agents, and it is why the next section matters so much.

Prompt caching: paying less for the repetition

Because so much of each request is a repeat of the last one, providers offer prompt caching. The model provider stores the processed prefix of your prompt, and when the next request starts with the same prefix, you pay a reduced rate for those tokens.

The details vary by provider, so read the docs for the model you use. According to Anthropic's prompt caching documentation, writing to the default five-minute cache costs 1.25 times the base input price, and reading from it costs 0.1 times the base price, with cache reads on Claude Opus 5.5 and Claude Sonnet 5.5 priced at 0.05 times. A one-hour cache is also available, with writes at 2 times the base price. OpenAI's prompt caching guide states that for GPT-5.6 and later models, cache writes cost 1.25 times the uncached input rate, reads cost 0.1 times on most of those models and 0.05 times on GPT-6.1 Sol, the minimum cacheable prefix is 1,024 tokens, and a cached prefix stays eligible for reuse for 30 minutes after its last use. Google documents its own approach in the Gemini context caching guide.

Apply Anthropic's numbers to our example. On each turn after the first, the entire previous request is a cached prefix, and only the newest 1,500 tokens are fresh. Across the run that works out to 99,000 tokens read from cache and 18,500 tokens written to it. At Sonnet 5.5 rates, the reads cost about $0.01 and the writes about $0.046, so input drops from $0.235 to roughly $0.056. With output unchanged, the run falls from about $0.27 to about $0.09, a cut of roughly two-thirds, with no change in model or behavior.

Caching has rules that can quietly break it. The prefix must match exactly, so anything that changes early in the prompt, such as a timestamp in the system prompt or tools reordered between calls, invalidates everything after it. Anthropic's documentation specifically notes that modifying tool definitions invalidates the entire cache. Keep stable content first (tools, then system prompt, then history), append new content at the end, and keep the agent moving quickly enough that the cache does not expire between turns.

Batch processing for work that can wait

The second big discount is for work that does not need an answer right away. Both major providers offer batch APIs that accept many requests at once and return results later. Anthropic's Message Batches API cuts costs by 50 percent, with most batches finishing in under an hour and a 24-hour limit. OpenAI's Batch API also offers a 50 percent discount, with each batch completing within 24 hours.

Batching does not suit interactive agents that a user is waiting on, but a lot of agent work is not interactive. Nightly report generation, classifying a backlog of support tickets, enriching a product catalog, and running evaluations are all good candidates. If a job can tolerate a delay, sending it through a batch endpoint is the easiest way to halve its cost.

A cost checklist for agent builders

Pulling this together, a short routine catches most waste. Measure first: log input and output tokens per turn so you can see where the snowball builds. Trim the tool list per agent and tighten descriptions. Shrink tool results before they enter the history. Order prompts for caching and confirm in your usage reports that cache reads are actually happening. Route easy steps to a cheaper model, as covered in our guide to choosing an LLM for agents. Move anything non-urgent to batch.

None of these steps requires new infrastructure, and together they routinely turn an agent that looks too expensive to ship into one that is comfortably affordable. To estimate the cost of a specific prompt across current models before you write any code, try the free Token and Cost Estimator.

Sources

Anthropic pricing, tool use, prompt caching, and batch processing documentation; OpenAI function calling, prompt caching, and Batch API guides; Google Gemini caching guide. All linked above and checked October 15, 2026.