Agent policies outgrow the prompt: AgentPolicyForge trains guards that cut attack success by 40–44% relative
Synopsis
Writing tool, resource and execution policies into a task agent's prompt does not reliably prevent violations. AgentPolicyForge uses executable environments to build policy benchmarks and iteratively train policy guards, yielding PolicyGym, which spans ten policy families with 698 scenarios for training-data generation and 254 separate held-out scenarios. On PolicyGym, the final guard-based configuration achieves a 40–44% relative reduction in attack success rate and fewer benign rejections than policy prompting alone.
Interpretation
Customized agent deployments define policies over tools, resources and execution conditions, and placing those policies in the task agent's prompt does not reliably prevent violations. The prior approach put the policies into the task agent's prompt and relied on the agent to regulate itself. On PolicyGym, the final guard-based configuration achieves a 40–44% relative reduction in attack success rate and fewer benign rejections than policy prompting alone.
The approach turns policy compliance into a trainable judgment: the guard decides whether a tool call complies from the call's executed outcome. Training a guard requires examples of tool calls that comply with or violate the policies, and AgentPolicyForge generates them with executable environments. PolicyGym covers ten policy families, with 698 scenarios used for training-data generation and 254 separate held-out scenarios forming human-reviewed test sets; every violating case uses a permitted tool, so arguments, resources and context must be checked rather than tool identity alone.
Guard training is iterative: an attacker generates requests using retrieved strategies, and the guard is fine-tuned on balanced, environment-verified examples. Training improves policy enforcement even with a fixed archive, and updating the archive further reduces in-domain attack success compared with keeping it fixed. That comparison is made on in-domain attack success rate in PolicyGym, against keeping the archive fixed.
Perspective
The demonstrated scope is ten policy families in PolicyGym, with 698 training scenarios and 254 human-reviewed held-out scenarios, compared against policy prompting for task-agent self-regulation.
The material states only that updating the policy archive further reduces in-domain attack success compared with keeping it fixed, and gives no out-of-domain behavior; the attack-success reduction is reported as a relative figure and comes from summary-level information.