A flipped bit no longer just ruins one generation: agents repair malformed calls, but output tokens rise 38%

Agent execution turns model-level output corruption into system-level outcomes, so a corrupted intermediate output does not necessarily become a task failure. Earlier work on hardware faults in LLMs treated the corrupted generation as the endpoint of fault propagation, measuring only wrong outputs or generation failures. On AgentDojo, the only multi-turn benchmark here, agents recover from 3.3–25.9% of malformed tool calls, while token storms are never recovered because they contain no valid action.

Recovery is not free: faulted executions consume more inference compute. Task outcomes can stay correct while execution cost rises under fault injection. On HomeBench exact-match accuracy falls only 4.2 percentage points, from 75.6% to 71.4%, while mean output tokens rise 51.3%, from 1,656 to 2,505 per execution; pooled across benchmarks output tokens rise 38.0%, and executions with token storms account for 96.0% of that increase.

Some faults do not look like obvious degeneration, producing structurally valid but semantically wrong actions. Detectable failures include malformed actions, token storms and early termination, whereas other faults keep the model fluent. Examples across all three benchmarks include an incorrect numerical interpretation, an unjustified transaction update and a misdirected escalation, each structurally valid and hard to catch from generation validity alone.

Random bitwise faults can also alter security- and alignment-relevant behavior, varying by model and scenario. After controlling for degenerate executions, AgentDojo prompt-injection susceptibility is nearly unchanged, differing by only 0.2–1.6 percentage points. Replaying one two-bit fault that caused a prompt-injection failure raises attack success for Qwen3.6-27B from 31.0% to 43.0% while task utility holds at 56.0% to 58.0%, and the Slack suite rises from 57.0% to 92.0%; the trend is not consistent for Qwen3.8-27B or Granite4.1-30B.

The study covers 218 tasks across three agent benchmarks and 6,814 executions, of which 2,400+ are clean runs and 4,300+ are runs with independently sampled two-bit weight faults; faults are injected only into model weights while the agent runtime, tools and benchmark environment stay fault-free. Qwen3.6-27B is the primary model, with Qwen3.8-27B and Granite4.1-30B used to examine model dependence. The authors propose as a next step examining how the findings generalize across different fault conditions and agentic environments, including the prevalence of subtle semantic errors and fault propagation in diverse multi-turn settings.

The authors state that the study is limited to persistent two-bit weight faults across a limited set of models and benchmarks, and that more rigorous statistical analyses would be needed to distinguish fault-induced behavioral changes from inherent execution variability. Security and alignment effects appear only for particular models and scenarios: harmful-action rates rise for Qwen3.6-27B, while the same trend does not hold consistently for Qwen3.8-27B or Granite4.1-30B, and AgentDojo prompt-injection susceptibility differs by only 0.2–1.6 percentage points after controlling for degenerate executions.