Same tool error, different placement: a recovery layer lifts ToolMaze explicit-error success by 14.2 points
Synopsis
After a failed tool call, whether an agent retries, fixes the arguments or stops to report depends not only on the error text but on how that feedback enters the conversation. Holding the error content fixed and varying only delivery across 13 open models from 3B to 70B parameters, keeping the failed exchange and appending a runtime note with guidance in a new user turn avoided a severe stopping failure in Llama models; implemented as an opt-in recovery layer inside an inference engine, this form raised ToolMaze success on explicit errors by 14.2 points with 7–8B models and 10.5 points with 32–70B models, with no measurable gain on Loud/Quiet.
Interpretation
The same tool-error text changes what an agent does next depending on how it is delivered: retry the same call, call again with corrected arguments, or stop and tell the user. In the form the client sends as-is, five of six open 7–35B models rarely repeated a call after a transient error, with repeat rates between 0 and 37 percent and one model never repeating; correct fixes on fixable errors ranged from 2 to 80 percent. Restating the error as a runtime note made those five models act correctly almost always; even with the same guidance sentence held fixed, the restated form beat guidance appended to the tool message in four models, with no significant difference in the opposite direction.
Keeping the failed call and appending a new user turn repairs the stopping failure caused by removing that exchange. The selected history-removing form left Llama-3.1-8B calling tools after unrecoverable errors, stopping in 0 of 60 cases; even the instruction not to call any tool again did not change that. A new user message after the failed exchange restored stopping in 59 of 60 cases for that model; for Llama-3.3-70B, stopping on unrecoverable errors was 95 percent with the client's form, 2 percent with the history-removing form and 100 percent with the retained-history form.
Implemented as an opt-in recovery layer in an inference engine, this form changes agent behavior without modifying the agents. The layer rewrites the conversation before it is rendered through the chat template, touches only failures after the last assistant message, and is off by default; it detects failures from client flags, recorded agent-specific text rules and generic error prefixes, and leaves uncertain cases ambiguous with no guidance. On ToolMaze, enabling the layer raised pass rates on explicit errors by 14.2 points in the 7–8B tier and 10.5 points in the 32–70B tier, with no detectable change where there was no fault; on Loud/Quiet's 24 multi-turn tasks it showed no measurable gain, because the missing tool is classified as ambiguous and receives no guidance.
Perspective
The demonstrated scope is decision-step comparisons across 13 open models from 3B to 70B parameters, English error texts and greedy decoding, plus two public benchmarks, ToolMaze and Loud/Quiet, and a six-task Qwen Code coding-agent evaluation; the recovery layer rewrites only failures after the last assistant message and is off by default.
The gains are driven largely by Llama stopping behavior, the pooled difference shrinks markedly without the two Llama models, and Qwen2.5-7B is worse with the retained-history form; real-history results come from 19 message clusters in six sessions, and leave-one-session-out sensitivity raises the p value to as much as 0.077; the transient guidance assumes a repeated call will succeed, so tools with side effects could duplicate writes, and the current implementation has no idempotency gate.