Skip to main content
Back to timeline
arXivSource publication:

Agent guardrails stop guessing: an LLM rewrites the rules, a verifier admits them, and coverage grows in deployment

Synopsis

VEPA writes an agent guardrail as executable symbolic rules, has an LLM propose patches from deployment feedback while a deterministic verifier admits only structurally valid, conflict-free policies, and reaches the highest F1 on events covered by its active policy across three agent safety benchmarks while expanding the scope of deterministic enforcement during deployment.

Source-provided article image: VEPA: Verified Evolving Formal Policy for Lifelong LLM Agent Guardrails
Figure 1 · arXiv

Interpretation

A guardrail's rules can be rewritten from deployment feedback while keeping allow-or-refuse decisions inspectable and reproducible. Earlier adaptive guardrails applied safety policy through LLM inference, so decisions varied with prompt context, while formal guardrails kept policies fixed before deployment or task-specific and had no mechanism to revise them from newly observed risks. Under online deployment on ASSEBench-Safety, ASSEBench-Security, and ATBench, VEPA reaches the highest F1 on events covered by its active policy, ahead of an LLM-only guardrail and adaptive guardrails built on natural-language memory.

Policy updates are split into generation, verification, and authority, so the LLM proposes candidate patches but does not directly decide enforcement. Earlier approaches either let the LLM issue the decision itself or let newly learned memory take effect immediately, without a step that checks structural validity and rule conflicts before a rule reaches deployment. Agent events are parsed into typed records, and rules are executable symbolic predicates over those records; a deterministic verifier admits only structurally valid, conflict-free candidates, and an evidence-aware authority layer activates a rule only after it accumulates enough supporting evidence, revoking authority when later feedback contradicts it.

Perspective

The method targets runtime protection of deployed agents: an agent action is first mapped to a typed event under a fixed schema, after which active rules return allow, refuse, or abstain. The determinism guarantee covers only the post-parsing enforcement layer, where an active policy gives a reproducible symbolic decision for a given event. The authors treat interactive benchmarks as instances of the same framework, invoked at each action or tool-call boundary with the current environment state included in the event context.

The authors state that the upstream LLM parser remains a neural component that may make extraction errors or be affected by adversarial inputs, so they do not claim end-to-end determinism from raw interactions to guardrail decisions; decisions from the LLM fallback also lie outside the formal guarantee. The policy language currently works over a fixed event schema and binary allow-or-refuse decisions, which may not capture richer temporal, relational, or multi-step requirements. The authority criterion is described as an operational Bayesian criterion for the selected rule rather than a selection-adjusted frequentist guarantee, and it provides no simultaneous or time-uniform guarantee over the full deployment horizon.