AUROC alone picks the wrong warning probe: Prewarn scores alarms against the onset, and two models warn before most loops
Language-model failures often begin at a specific token, yet the usual response-level AUROC asks only whether a finished response failed, not when an alarm fired. Prewarn fixes each monitor's threshold on validation data at an error rate chosen for the application, then scores classification, detection and early warning separately against an annotated onset. On repetitive degeneration, a length-only rule reaches near-perfect AUROC but can never warn; frozen probes warn before the onset on most degenerate responses of Apertus and Llama, while on Mistral they mostly fire after it.