Handling tool errors in agent loops: retries, idempotency, and knowing when to stop

Every agent demo works on the first try. Production agents live somewhere messier. APIs time out, rate limits kick in, a database row was already deleted, and the model passes a date in the wrong format. What separates a reliable agent from a fragile one is rarely the model. It is how the loop around the model handles the moment a tool call goes wrong. This article is about that moment: how to classify tool failures, how to report them so the model can fix its own mistakes, which failures your code should retry without involving the model at all, and how to make sure a retry never does the same damage twice.

Two kinds of failure, two different audiences

The cleanest mental model comes from the Model Context Protocol tools specification, which splits errors into two mechanisms. Protocol errors are standard JSON-RPC errors for problems like an unknown tool, a malformed request, or a server fault. Tool execution errors are reported inside a normal tool result with isError set to true, and they cover API failures, input validation problems such as a date in the wrong format or a value out of range, and business logic errors. The spec explains why the split matters: execution errors contain actionable feedback that a model can use to self-correct and retry with adjusted parameters, while protocol errors point at problems with the request structure that a model is less likely to fix. Clients should pass execution errors back to the model, and may pass protocol errors, though those are less likely to lead to a recovery.

That distinction is useful even if you never touch MCP. Ask one question about every failure: who can fix this? If the model chose bad arguments, the model can fix it, so the error belongs in the conversation. If the network dropped a packet, the model cannot do anything useful about it, so the error belongs to your code. Mixing the two up is the root of most bad agent behavior around errors. Send a raw connection reset to the model and it may apologize to the user and give up on a task that one quiet retry would have finished. Swallow a validation error in your code and the model never learns that its arguments were wrong, so it makes the same call again.

Reporting errors the model can act on

Both major model APIs give you a simple way to tell the model a tool failed. In Anthropic's API, a tool_result block takes an optional is_error flag, and the guide to handling tool calls shows a network failure returned as the content "ConnectionError: the weather service API is not available (HTTP 500)" with is_error set to true. The model then works that into its reply instead of inventing a forecast. OpenAI's function calling guide takes a looser approach: the output you send back should usually be a string, and the format is up to you, whether JSON, error codes, or plain text. Either way the model reads the error as text, so the wording of that text is the whole interface.

That is why the most valuable habit is writing errors as instructions. Anthropic's documentation recommends replacing generic messages like "failed" with what went wrong and what to try next, and gives "Rate limit exceeded. Retry after 60 seconds." as an example. Its engineering post on writing tools for agents makes the same point more broadly, suggesting you prompt-engineer error responses to communicate specific, actionable improvements rather than opaque error codes or tracebacks. The MCP spec's own example of an execution error reads "Invalid departure date: must be in the future. Current date is 08/08/2025." Notice what that message does. It names the field, states the rule, and hands the model the one fact it needs to pick a valid value.

Here is the difference in practice. Imagine a create_invoice tool that rejects a due date. A weak result says {"error": "VALIDATION_FAILED", "code": 422}. A strong result says: "due_date 2026-09-30 is in the past. Use a date on or after 2026-10-17 in YYYY-MM-DD format. No invoice was created." The last sentence matters as much as the rest. Telling the model that nothing happened stops it from wondering whether a half-finished invoice is sitting somewhere, and it makes a retry obviously safe.

The same post notes that lots of tool errors for invalid parameters usually mean the tools need clearer descriptions or better examples. Treat error logs as feedback on your tool design, not just as noise. If one argument keeps failing validation, the fix is often a sharper description or a tighter schema, which we covered in our guide to writing tool descriptions and schemas.

Let the schema catch what it can

The cheapest error is the one that never happens. Both providers now offer a strict mode that forces tool arguments to match your schema. OpenAI's guide says setting strict to true makes function calls reliably follow the schema instead of being best effort, and it recommends always enabling it. Strict mode has requirements: every object needs additionalProperties set to false and every field must be listed as required, with optional fields expressed by allowing null as a type. Anthropic's handling guide points to strict tool use as the way to eliminate invalid tool calls entirely, guaranteeing that inputs always match the schema.

Strict mode removes a whole class of failures, such as missing fields, wrong types, and extra keys, but it cannot know your business rules. A schema can say due_date is a string in date format. It cannot say the date must be in the future, that the customer must exist, or that the amount must stay under a credit limit. Those checks still live in your tool, and they still need the kind of instructive error message described above. Without strict mode the model will usually recover on its own anyway: Anthropic's documentation says that when a tool request is invalid or missing parameters, Claude will retry two to three times with corrections before apologizing to the user. That is a useful safety net, but each of those retries is a full model call you pay for.

Transient failures belong to your code

Timeouts, dropped connections, HTTP 429 rate limits, and 5xx server errors are a different animal. The model's arguments were fine; the world was briefly unavailable. Retry these in your code, below the model, and only surface them to the model if the retries run out.

The model APIs themselves show the pattern. Anthropic's error reference says its official SDK automatically retries transient failures, including connection errors, rate limits, and 5xx errors, with exponential backoff, twice by default, honoring the retry-after header when it is present. Its rate limit documentation tells you to wait the number of seconds in retry-after before retrying a 429, and it also contains a useful warning: when an organization reaches its tier's monthly spend cap, the API returns a 429 with no retry-after header, and retrying fails until access resumes. Not every 429 is temporary. Your retry logic should read the response, not just the status code.

How you space retries matters more than people expect. Google's SRE book, in its chapter on addressing cascading failures, says to always use randomized exponential backoff when scheduling retries. If retries are not spread out randomly, a small blip can cause clients to retry in sync, and those waves can amplify each other. The same chapter says not to retry a given request indefinitely and suggests a process-wide retry budget, for example allowing only 60 retries per minute and failing the request once that budget is spent. The companion chapter on handling overload describes Google's own limits: a per-request budget of up to three attempts, plus a per-client cap that keeps retries to about 10 percent of requests. The book notes this second cap cuts the extra load from retries during a widespread overload from nearly three times normal down to about 1.1 times.

Agents make this more urgent, because one user request can fan out into dozens of tool calls. If every layer retries three times on its own (the SDK, your tool wrapper, and the model deciding to call the tool again), a single outage can multiply into a storm of traffic. Pick one layer to own transient retries, give it a budget, and have every other layer pass the failure up.

Retries must not repeat side effects

Retrying a read is harmless. Retrying a write can charge a card twice. The standard defense is an idempotency key, and Stripe's documentation on idempotent requests explains the mechanics well. The client sends a unique key with a request that creates or updates something. If a connection error happens, the client can repeat the request with the same key without creating a second object or applying the update twice. For API v1, Stripe saves the status code and body of the first request made with a key and returns that saved response on retries, compares parameters so a reused key with different arguments returns an error, and recommends V4 UUIDs or similarly random strings of up to 255 characters.

In an agent, the right place to create that key is your tool wrapper, not the model. Generate it once when the model first asks to perform an action, keep it in the loop's state, and reuse it for every transport-level retry of that same call. If the model later decides on purpose to do the action again with different arguments, that is a new action and gets a new key. This keeps the model in charge of what should happen and your code in charge of making sure it happens exactly once.

MCP gives clients a hint about which tools are safe to repeat. The MCP schema defines tool annotations including readOnlyHint, destructiveHint, and idempotentHint, where idempotent means calling the tool repeatedly with the same arguments has no additional effect. Those are only hints, though. The schema states plainly that they are not guaranteed to describe the tool faithfully and that clients should never base tool use decisions on annotations from untrusted servers. Use them to set sensible defaults for servers you control, and keep your own allowlist of which tools may be retried automatically.

Knowing when to stop

The last piece is a stop condition. Anthropic's essay on building effective agents describes agents as systems that gain ground truth from tool results at each step, notes that it is common to add stopping conditions such as a maximum number of iterations, and warns that agents' autonomy brings higher costs and the potential for compounding errors. Errors are exactly where loops run away. A model that keeps getting "invalid argument" may keep trying variations forever, and each attempt re-sends the whole growing conversation.

A practical setup has three limits. Cap how many times the same tool can fail in a row before the loop stops calling it and tells the model the tool is unavailable for this task. Cap total steps per task. Cap total spend or tokens per task. When any limit trips, end with an honest message to the user or a handoff to a human, saying what was attempted and what failed. The MCP spec backs this up from the security side: it says clients should set timeouts for tool calls, log tool usage for auditing, and ask the user to confirm sensitive operations, which is also the right moment to stop retrying something destructive.

Put together, the pattern is simple to state. Let the schema reject malformed calls before they run. Return business errors to the model as plain instructions that say what failed, what to do next, and whether anything changed. Retry transient failures once, in one layer, with jittered backoff and a budget. Protect every write with an idempotency key. And give the loop a firm place to stop. None of this needs a smarter model, and all of it makes the model you already have look a lot smarter.