# AgentDojo

Tool-using LLM agents turn model output into real actions, making indirect prompt injection an execution-security problem. SecureGARD inserts an executable ontology between planning and tool execution, grounding heterogeneous tool calls into shared semantic actions and judging each as ready, waiting, or forbidden against the current task state. Under benign and indirect-injection settings on AgentDojo, ASB, and AgentDyn, the observed attack success rate is 0% on both backbones, with higher task utility under attack than the compared baselines.

Tool-using LLM agents turn model output into real actions, making indirect prompt injection an execution-security problem. SecureGARD inserts an executable ontology between planning and tool execution, grounding heterogeneous tool calls into shared semantic actions and judging each as ready, waiting, or forbidden against the current task state. Under benign and indirect-injection settings on AgentDojo, ASB, and AgentDyn, the observed attack success rate is 0% on both backbones, with higher task utility under attack than the compared baselines.

Tool-using language-model agents can pick the intended tool yet execute an injected argument, and existing counterfactual guards lose source sensitivity when they score with a reasoning prefix that already encodes the injected decision. BiCP removes the current-step reasoning prefix before scoring, probes both the tool-name target and the complete-call target, and routes proposals by operation risk. On a 922-case fixed replay, the full method reduces confirmed InjecAgent attacks from 83/682 to 0/682 and unsafe conditional executions from 33/240 to 0/240, while benign task success rises from 66/120 to 78/120.

Tool-using language-model agents can pick the intended tool yet execute an injected argument, and existing counterfactual guards lose source sensitivity when they score with a reasoning prefix that already encodes the injected decision. BiCP removes the current-step reasoning prefix before scoring, probes both the tool-name target and the complete-call target, and routes proposals by operation risk. On a 922-case fixed replay, the full method reduces confirmed InjecAgent attacks from 83/682 to 0/682 and unsafe conditional executions from 33/240 to 0/240, while benign task success rises from 66/120 to 78/120.

Linear injection probes could reach 0.91–0.98 AUROC on the standard evaluation using conversation position and text length alone; adding same-moment clean twins and harmless look-alikes to training cuts false alarms on injection-free conversations from 41–80% to 4–27% across six open models.

Linear injection probes could reach 0.91–0.98 AUROC on the standard evaluation using conversation position and text length alone; adding same-moment clean twins and harmless look-alikes to training cuts false alarms on injection-free conversations from 41–80% to 4–27% across six open models.

Robustness tests for tool-using agents can now check, before the agent runs, whether an edited test case still tests the same task; rescoring the same 14,336 stored trajectories under executable task contracts drops Mistral's attacked-task count from 64 to 26 under a MiniLM cosine threshold.

Robustness tests for tool-using agents can now check, before the agent runs, whether an edited test case still tests the same task; rescoring the same 14,336 stored trajectories under executable task contracts drops Mistral's attacked-task count from 64 to 26 under a MiniLM cosine threshold.

Injecting random two-bit weight faults into agent executions turns model-level corruption into system-level outcomes: agents recover from malformed tool calls and still finish tasks, yet faulted runs raise mean output tokens by 38.0%, and some faults shift security and alignment behavior.

Injecting random two-bit weight faults into agent executions turns model-level corruption into system-level outcomes: agents recover from malformed tool calls and still finish tasks, yet faulted runs raise mean output tokens by 38.0%, and some faults shift security and alignment behavior.

ASPIRE is an open-ended prompt-injection red-teaming framework for LLM agents that maintains an evolving Agent Security Behavior Graph, uses complementary Explore and Exploit experts to propose consequence-centric test skeletons, has a separate realizer build concrete payloads and trigger queries, and updates the graph and cross-run strategy memory from trajectory evidence; on AgentDojo and AgentDyn it raises attack success to as high as 54.1% while covering more consequences, injection methods, environments, and behavior paths.

ASPIRE is an open-ended prompt-injection red-teaming framework for LLM agents that maintains an evolving Agent Security Behavior Graph, uses complementary Explore and Exploit experts to propose consequence-centric test skeletons, has a separate realizer build concrete payloads and trigger queries, and updates the graph and cross-run strategy memory from trajectory evidence; on AgentDojo and AgentDyn it raises attack success to as high as 54.1% while covering more consequences, injection methods, environments, and behavior paths.

The work sizes the selection cost of picking the best single model as comparator on the evaluation data in safety-routing benchmarks: 0.003-0.030 of harm under random splits and 0.045-0.113 under held-out categories on HELM Safety, comparable to the whole deficit attributed to routing, rising seven- to ninefold on AgentDojo when suites are held out; scored honestly under shift, routing buys little, and an attacker who knows which model it faces lowers GPT-5.4's judged recognition by 19.6 points.

The work sizes the selection cost of picking the best single model as comparator on the evaluation data in safety-routing benchmarks: 0.003-0.030 of harm under random splits and 0.045-0.113 under held-out categories on HELM Safety, comparable to the whole deficit attributed to routing, rising seven- to ninefold on AgentDojo when suites are held out; scored honestly under shift, routing buys little, and an attacker who knows which model it faces lowers GPT-5.4's judged recognition by 19.6 points.