Voice-agent testing moves off manual dialing: LiveKit's Agent Simulations runs whole tasks with LLM users and fails the build
LiveKit moves voice-agent testing from turn-by-turn dialogue to task-level simulation: an LLM-driven simulated user plays out a scenario with a defined persona, goals and behavior, and an LLM judge grades the transcript against freeform expectations with a pass or fail verdict and a reason. Scenarios run in parallel, and results, transcripts and traces land in the LiveKit Cloud dashboard; text mode is the default, paced into spare capacity at low priority, and a failed scenario returns a non-zero exit code from the CLI. Serve AI used simulations to validate a full agent rewrite against a hundred real production calls before cutting any traffic over.