# Word Error Rate

The authors built VTR-Bench, a benchmark that embeds prescribed text and its carriers into 300 prompts (1,202 text blocks) across advertising, science, user interfaces, culture, and daily life, scores text fidelity by carrier-specific transcription and word error rate (WER) and scene and motion fulfillment by a 20-question prompt-specific chain of query, and adds a Keyframe-Guided Agentic Framework in which a Director agent coordinates image generation, motion planning, and visual feedback; across 11 state-of-the-art models they find widespread difficulty in rendering scene text, with the best model reaching an overall WER of 0.250, while the framework cuts overall WER on Minimax H3 from 0.4468 to 0.3015 (a 32.5% relative reduction) and raises Video Score.

The authors built VTR-Bench, a benchmark that embeds prescribed text and its carriers into 300 prompts (1,202 text blocks) across advertising, science, user interfaces, culture, and daily life, scores text fidelity by carrier-specific transcription and word error rate (WER) and scene and motion fulfillment by a 20-question prompt-specific chain of query, and adds a Keyframe-Guided Agentic Framework in which a Director agent coordinates image generation, motion planning, and visual feedback; across 11 state-of-the-art models they find widespread difficulty in rendering scene text, with the best model reaching an overall WER of 0.250, while the framework cuts overall WER on Minimax H3 from 0.4468 to 0.3015 (a 32.5% relative reduction) and raises Video Score.

This study constructed a multilingual clinical dictation corpus (five clinical dictation scripts spanning a complexity gradient, translated into 99 languages, rendered to synthetic speech under three acoustic conditions, and transcribed by a production ambient scribe), computed six frequency metrics, and had three independent large language model raters assess clinically meaningful error patterns using a Severity x Likelihood framework; across 59,819 genuine transcription-error occurrences, 58,329 (97.5%) were LOW risk and 251 (0.42%) CRITICAL or HIGH, none of the six frequency metrics showed a detectable association with serious clinical risk (absolute Spearman rho < 0.16), a Severity x Likelihood sum remained strongly correlated with WER (rho=0.

This study constructed a multilingual clinical dictation corpus (five clinical dictation scripts spanning a complexity gradient, translated into 99 languages, rendered to synthetic speech under three acoustic conditions, and transcribed by a production ambient scribe), computed six frequency metrics, and had three independent large language model raters assess clinically meaningful error patterns using a Severity x Likelihood framework; across 59,819 genuine transcription-error occurrences, 58,329 (97.5%) were LOW risk and 251 (0.42%) CRITICAL or HIGH, none of the six frequency metrics showed a detectable association with serious clinical risk (absolute Spearman rho < 0.16), a Severity x Likelihood sum remained strongly correlated with WER (rho=0.