Prefill jailbreaks meet a training-free fix: restore the refusal signal the attack erodes, request by request
Synopsis
Prefill jailbreaks force an unsafe opening into the assistant turn so a model that would refuse continues instead. Tracking vocabulary-grounded signals with the Jacobian lens shows refusal weakening faster than harmfulness as the forced prefix deepens, reaching the benign baseline by free generation. J-Anchor, a training-free inference-time defense, anchors each request to its pre-attack assistant-header state and restores only the refusal components that fall below that target during prefilling and decoding, cutting attack success on fixed and deep prefills across four open-weight instruction models while keeping benign refusal and reasoning close to the undefended model.
Interpretation
The failure point of prefill attacks is located as refusal-state erosion: the longer the attacker-fixed assistant prefix, the weaker the model's internal refusal signal, while harmfulness recognition stays detectable. Earlier jailbreak work treated harmfulness representations and refusal behavior separately, or took single-position safety readouts before or during generation, without tracing how the two signals diverge as an attack prefix advances token by token. Tracking from the assistant header through each attacker-controlled prefill token into free generation, both signals weaken with prefill depth, but refusal declines to the benign baseline while harmfulness stays above it; attacks that ultimately fail show stronger refusal recovery during decoding.
J-Anchor treats the pre-attack assistant-header state as a request-specific reference and repairs only the refusal components that fall below it. Existing defenses fine-tune models to recover from harmful continuations, modify decoding probabilities, or steer hidden representations toward refusal, each requiring training or an extra classifier. The assistant-header state reflects the model's response to the original request, and its harmfulness and refusal readouts form the anchor; requests the model does not register as risky are gated out and generated without intervention.
Restoration happens in the Jacobian lens's vocabulary-grounded semantic space, axis by axis along the refusal subspace. The Jacobian lens maps intermediate hidden states to the output vocabulary through averaged Jacobians, giving a vocabulary-level semantic readout rather than applying the output projection directly. Refusal and harmfulness lexicons are built per model tokenizer and calibrated on 500 prompts from the benign WildJailbreak subset with their reference responses, so scores are zero at the benign mean; intervention layers span 40% to 85% of model depth.
Across four open-weight instruction-tuned models, fixed and deep prefill attack success drops sharply while benign refusal and reasoning are largely preserved. Without defense, fixed prefilling raises attack success to 54.1% to 89.0%, and deep prefilling pushes it to 83.0% to 94.0%. J-Anchor limits fixed-prefill attack success to at most 3.8%, with 0% in five of nine settings, and deep-prefill success to at most 2.5% on three backbones; over-refusal on XSTest and OKTest stays close to the clean model, as do GSM8K and MATH500 scores.
Perspective
Evaluation covers four open-weight instruction-tuned models, Llama-3.1-8B-Instruct, Gemma-4-E4B-IT, Qwen3.5-4B and Qwen3-8B in the appendix, with fixed and deep prefill attacks, XSTest's safe subset and OKTest for benign behavior, and GSM8K and MATH500 for reasoning; Qwen3 models run with thinking disabled and all experiments use greedy decoding.
Attack success is judged by HarmBench-Llama-2-13b-cls and over-refusal by WildGuard, with a DeepSeek-Flash recheck in the appendix; against the ADA defenses the safety-utility trade-off varies by model, which the authors describe as model-dependent.