# Jailbreaking

Safety alignment in open-weight models is highly sparse: pruning a small set of safety-critical neurons or ablating a single refusal direction can make a model produce harmful content. ABS-Align gradually prunes those identified safety-critical neurons during preference finetuning, forcing refusal behavior to be re-established through other parts of the network, and restores the pruned neurons afterward. Across Gemma-2-2B-IT, Llama-3-8B-Instruct, Llama-3.2-3B-Instruct and Qwen-3-4B, attack success generally falls against TwinBreak, NeuroStrike, refusal-direction ablation and ten SORRY-Bench black-box attacks, with reductions of about 90% for some attacks, while TruthfulQA improves and general capabilities change little.

Safety alignment in open-weight models is highly sparse: pruning a small set of safety-critical neurons or ablating a single refusal direction can make a model produce harmful content. ABS-Align gradually prunes those identified safety-critical neurons during preference finetuning, forcing refusal behavior to be re-established through other parts of the network, and restores the pruned neurons afterward. Across Gemma-2-2B-IT, Llama-3-8B-Instruct, Llama-3.2-3B-Instruct and Qwen-3-4B, attack success generally falls against TwinBreak, NeuroStrike, refusal-direction ablation and ten SORRY-Bench black-box attacks, with reductions of about 90% for some attacks, while TruthfulQA improves and general capabilities change little.

Across 18 Base–Instruct model pairs, harmfulness is more decodable from attention updates while refusal evidence is written mainly by MLPs, and post-training strengthens that writing without improving recognition.

Across 18 Base–Instruct model pairs, harmfulness is more decodable from attention updates while refusal evidence is written mainly by MLPs, and post-training strengthens that writing without improving recognition.

The work introduces the Perturbed Embedding Vector (PEV) jailbreak: adding independent Gaussian noise to a prompt's embedding and resampling it repeatedly, with no gradient computation and no weight modification, achieves a 100% jailbreak rate on all 100 JailbreakBench prompts across six open-weight LLMs, with the first successful jailbreak typically arriving within one minute and at lower compute cost than the compared prior methods.

The work introduces the Perturbed Embedding Vector (PEV) jailbreak: adding independent Gaussian noise to a prompt's embedding and resampling it repeatedly, with no gradient computation and no weight modification, achieves a 100% jailbreak rate on all 100 JailbreakBench prompts across six open-weight LLMs, with the first successful jailbreak typically arriving within one minute and at lower compute cost than the compared prior methods.

LADE extracts model-agnostic latent safety signals from dark knowledge in the first-token output probability distribution by contrasting harmful and benign queries, maps the selected tokens across tokenizers, and filters harmful queries via kNN distance before generation; across Llama-2-7B-Chat, Llama-3-8B-Instruct, Qwen2-7B-Instruct, Qwen3-8B, Gemma-7B-it, and Mistral-7B-Instruct-v0.3, it attains the lowest or near-lowest average compliance under multiple jailbreak attacks and the best safety-utility average on six of eight models.

LADE extracts model-agnostic latent safety signals from dark knowledge in the first-token output probability distribution by contrasting harmful and benign queries, maps the selected tokens across tokenizers, and filters harmful queries via kNN distance before generation; across Llama-2-7B-Chat, Llama-3-8B-Instruct, Qwen2-7B-Instruct, Qwen3-8B, Gemma-7B-it, and Mistral-7B-Instruct-v0.3, it attains the lowest or near-lowest average compliance under multiple jailbreak attacks and the best safety-utility average on six of eight models.