Skip to main content
Search

# Intent-Aware benchmark

1 related articles

Subscribe to updates
  1. arXiv

    Agent security now checks deployment purpose: four-class macro-F1 rises from 45% to 79%, out-of-scope F1 from 8% to 76%

    Existing agent defenses mainly detect prompt injection, harmful outputs or suspicious behavior, but do not determine whether an action still fits the purpose for which the agent was deployed. Intent-Aware Security extracts organizational, role and developer intent into an agent-specific Intent Profile that serves as the reference object for security decisions, and builds four protections on it: pre-execution detection, post-execution detection, input alignment and system-prompt hardening. On a benchmark of 887 samples across 35 agent configurations, profile-based pre-execution detection raises four-class macro-F1 from 45% with raw context to 79%, and Out-of-Scope F1 from 8% to 76%.

You’re all caught up