A shared target for cross-lingual jailbreaks: a safety kernel of 0.12% of neurons lets masked fine-tuning cut attack success in many languages
Synopsis
Large language models refuse the same harmful request in one language but may comply when it is rephrased in another. Researchers first identified safety-related neurons independently within each language, then took the intersection of the candidate sets from six languages to obtain a cross-lingually shared "safety kernel," and tested its functional role with activation suppression and activation-state transfer. Building on it, SK-Masked LoRA restricts fine-tuning updates to the feed-forward network output dimensions corresponding to the safety kernel, lowering attack success rates across several models and multilingual safety benchmarks, with some effect on languages excluded from kernel discovery.
Interpretation
Cross-lingual safety behavior maps onto a small set of neuron coordinates shared across languages, which the researchers call the safety kernel. Earlier multilingual safety methods improved behavior mainly through external alignment signals such as multilingual safety data, preference optimization, cross-lingual consistency constraints or safety-capability transfer, without explicitly localizing a shared internal target for cross-lingual safety intervention. In Qwen3-8B and Llama-3.1-8B the safety kernel occupies only 0.12% and 0.15% of the analyzed feed-forward output coordinates, versus 0.57% and 0.55% for the corresponding language-wise safety-neuron sets; zeroing the kernel coordinates raises average attack success rate by 52.65 and 45.13 points, while masking a matched number of random coordinates leaves it near the default level.
Restricting safety fine-tuning updates to the safety-kernel coordinates lowers attack success rates in multiple languages. Standard LoRA updates all dimensions and reaches the lowest attack success rate but causes the largest degradation in general capability; against a size-matched English-masked variant, kernel masking lowers attack success rate from 8.31 to 7.52 on Qwen3-8B and from 6.77 to 5.30 on Llama-3.1-8B. On AdvBench-x and MultiJail, average attack success rate falls from 28.22 to 1.20 and from 29.44 to 4.35 for Llama-3.1-8B-it, and from 3.42 to 0.34 and from 10.16 to 1.35 for Gemma-2-9B-it; on Vietnamese, Javanese, German, Japanese and Russian, none used for kernel discovery, average attack success rate falls from 14.41 to 8.95 on Qwen3-8B and from 18.67 to 4.76 on Llama-3.1-8B-it.
The safety-kernel coordinates come from intersecting independently selected per-language sets rather than anchoring on English. Earlier work on shared safety neurons used an English-centered formulation, treating neurons shared between English and each target language as bridges for safety transfer. Each language ranks coordinates by a safety-contrast-with-activation-responsiveness score computed from multilingual parallel safe and unsafe prompts, and the top candidates are intersected across six languages; recomputing the intersection after excluding each discovery language in turn raises the relative kernel size only modestly on average, with per-language increments ranging from the smaller Italian value to the larger Bengali value.
Perspective
Experiments cover three instruction-tuned models, Qwen3-8B, Llama-3.1-8B-it and Gemma-2-9B-it; the safety kernel is discovered from English, Chinese, Italian, Korean, Thai and Bengali; safety is evaluated on AdvBench-x and MultiJail and general capability on MMLU and MGSM; all experiments run on a single NVIDIA A100 GPU.
The authors describe the safety kernel as a shared neuronal subset functionally associated with cross-lingual safety behavior and amenable to intervention, not merely a statistical overlap; for languages excluded from discovery they report a measurable degree of generalization without claiming those languages contain identical language-specific safety-neuron sets. The masking effect stabilizes as the number of retained candidates grows, and the authors treat that setting as stable rather than universally optimal.