# Linear Probe

Language-model failures often begin at a specific token, yet the usual response-level AUROC asks only whether a finished response failed, not when an alarm fired. Prewarn fixes each monitor's threshold on validation data at an error rate chosen for the application, then scores classification, detection and early warning separately against an annotated onset. On repetitive degeneration, a length-only rule reaches near-perfect AUROC but can never warn; frozen probes warn before the onset on most degenerate responses of Apertus and Llama, while on Mistral they mostly fire after it.

Language-model failures often begin at a specific token, yet the usual response-level AUROC asks only whether a finished response failed, not when an alarm fired. Prewarn fixes each monitor's threshold on validation data at an error rate chosen for the application, then scores classification, detection and early warning separately against an annotated onset. On repetitive degeneration, a length-only rule reaches near-perfect AUROC but can never warn; frozen probes warn before the onset on most degenerate responses of Apertus and Llama, while on Mistral they mostly fire after it.

Audio language model benchmark scores can mistake language ability for acoustic understanding. Using synthetic readings and transcripts as references, the study measures the recording gain of the original audio over words across 92 tasks from 6 benchmarks, finding that 19 open models trail Gemini 3.1 Pro by 28.6 and 41.9 percentage points on global-attribute and within-recording structure tasks, with about half or more of each gap lying in the recording gain. Training LoRA adapters with a content anchor in the readout-decline layers of Gemma-4 12B and Qwen2.5-Omni-7B raises attribute and structure accuracy by 12.8 to 44.4 points while mean lexical accuracy stays within 1 point of the base models.

Audio language model benchmark scores can mistake language ability for acoustic understanding. Using synthetic readings and transcripts as references, the study measures the recording gain of the original audio over words across 92 tasks from 6 benchmarks, finding that 19 open models trail Gemini 3.1 Pro by 28.6 and 41.9 percentage points on global-attribute and within-recording structure tasks, with about half or more of each gap lying in the recording gain. Training LoRA adapters with a content anchor in the readout-decline layers of Gemma-4 12B and Qwen2.5-Omni-7B raises attribute and structure accuracy by 12.8 to 44.4 points while mean lexical accuracy stays within 1 point of the base models.

When routing reasoning models, how many tokens each query will produce is unknown in advance, and most routers price candidates at a fixed or median length. One approach reads the prompt's prefill activations from a frozen small model to predict both each candidate's success probability and its output length; on eight of nine datasets, pricing by predicted length spends 9% to 33% less than median-length pricing at matched accuracy, up to 69% of what perfect per-query cost knowledge would save.

When routing reasoning models, how many tokens each query will produce is unknown in advance, and most routers price candidates at a fixed or median length. One approach reads the prompt's prefill activations from a frozen small model to predict both each candidate's success probability and its output length; on eight of nine datasets, pricing by predicted length spends 9% to 33% less than median-length pricing at matched accuracy, up to 69% of what perfect per-query cost knowledge would save.

Manipulations in long videos often touch only a few seconds, while existing audio-visual deepfake benchmarks mostly use short talking-head clips and long-form forensics has been largely visual. AV-FABLE pairs 16,650 pristine–manipulated twins (998 hours, 27 editing configurations) with separate audio and visual timelines, so localization must report both the time boundaries and the affected modality. Reading frozen Qwen3-Omni perception features through separate audio and visual temporal decoders, NFSR reaches 91.30 in-domain AV-mAP against 84.76 for UMMAFormer on the same features, and the highest AV-mAP in all three out-of-distribution settings.

Manipulations in long videos often touch only a few seconds, while existing audio-visual deepfake benchmarks mostly use short talking-head clips and long-form forensics has been largely visual. AV-FABLE pairs 16,650 pristine–manipulated twins (998 hours, 27 editing configurations) with separate audio and visual timelines, so localization must report both the time boundaries and the affected modality. Reading frozen Qwen3-Omni perception features through separate audio and visual temporal decoders, NFSR reaches 91.30 in-domain AV-mAP against 84.76 for UMMAFormer on the same features, and the highest AV-mAP in all three out-of-distribution settings.

Linear injection probes could reach 0.91–0.98 AUROC on the standard evaluation using conversation position and text length alone; adding same-moment clean twins and harmless look-alikes to training cuts false alarms on injection-free conversations from 41–80% to 4–27% across six open models.

Linear injection probes could reach 0.91–0.98 AUROC on the standard evaluation using conversation position and text length alone; adding same-moment clean twins and harmless look-alikes to training cuts false alarms on injection-free conversations from 41–80% to 4–27% across six open models.

Using a controlled two-stage PGD attack on 200 held-out COCO images with Qwen2.5-VL-7B-Instruct, the authors show that a targeted perturbation can drive teacher-forced training loss for a fixed target caption to near zero while free generation still produces the original correct description, localize this train/inference gap to a single autoregressive step (target-token rank fixed at exactly 3,488 of 152,064 vocabulary entries with zero variance), and trace all 28 decoder layers to show the visual encoder corrupts every image by a comparable margin while the language decoder amplifies the corrupted signal for susceptible images and actively suppresses it for resistant ones (p<0.001, rank-biserial r=0.579).

Using a controlled two-stage PGD attack on 200 held-out COCO images with Qwen2.5-VL-7B-Instruct, the authors show that a targeted perturbation can drive teacher-forced training loss for a fixed target caption to near zero while free generation still produces the original correct description, localize this train/inference gap to a single autoregressive step (target-token rank fixed at exactly 3,488 of 152,064 vocabulary entries with zero variance), and trace all 28 decoder layers to show the visual encoder corrupts every image by a comparable margin while the language decoder amplifies the corrupted signal for susceptible images and actively suppresses it for resistant ones (p<0.001, rank-biserial r=0.579).

The authors introduce DP-RGMI, a framework that treats differentially private training as a structured transformation of representation space and decomposes performance degradation into representation displacement, spectral effective dimension, and a utilization gap defined as the difference between linear-probe and end-to-end AUROC; across more than 594,000 chest X-rays from four datasets and three pretrained initializations (ImageNet, DINOv3, MIMIC-CXR), with PadChest as the primary dataset (110,525 frontal images, 22,045 test images), they find that strong privacy consistently leaves linear separability largely preserved while a utilization gap persists (G = 8.0 for ImageNet at ε = 1.0, 3.4 for MIMIC at ε = 0.7, 6.1 for DINOv3 at ε = 0.

The authors introduce DP-RGMI, a framework that treats differentially private training as a structured transformation of representation space and decomposes performance degradation into representation displacement, spectral effective dimension, and a utilization gap defined as the difference between linear-probe and end-to-end AUROC; across more than 594,000 chest X-rays from four datasets and three pretrained initializations (ImageNet, DINOv3, MIMIC-CXR), with PadChest as the primary dataset (110,525 frontal images, 22,045 test images), they find that strong privacy consistently leaves linear separability largely preserved while a utilization gap persists (G = 8.0 for ImageNet at ε = 1.0, 3.4 for MIMIC at ε = 0.7, 6.1 for DINOv3 at ε = 0.