Bhavik Mangla

The work introduces VoxParity, a benchmark of 183 scenarios from 14 sectors in which one transcript stays fixed while the audio changes (a coaching voice, a medical monitor beeping, a mayday under a radio check, noise over a drug name, a child's voice placing a bet, a frightened whisper) and the correct executable tool call changes with it; across 206 cue-bearing cells, all 28 systems (including nine production realtime agents) execute the routine request on protective calls at 41% against 12% over-triggering on clean calls, and only 11 of the 23 systems with a transcript path pass the words-only null test, with passes coming almost entirely from items that state the rule.

The work introduces VoxParity, a benchmark of 183 scenarios from 14 sectors in which one transcript stays fixed while the audio changes (a coaching voice, a medical monitor beeping, a mayday under a radio check, noise over a drug name, a child's voice placing a bet, a frightened whisper) and the correct executable tool call changes with it; across 206 cue-bearing cells, all 28 systems (including nine production realtime agents) execute the routine request on protective calls at 41% against 12% over-triggering on clean calls, and only 11 of the 23 systems with a transcript path pass the words-only null test, with passes coming almost entirely from items that state the rule.