# Emergent Misalignment

Undesired behaviors in language models can re-emerge during later fine-tuning, and standard post-hoc suppression had not been compared on immediacy, durability and capability together. PEARL adds adversarial prefixes, continuations from intermediate states of cached misaligned trajectories, to GRPO's on-policy rollouts so the policy practices recovering from partly wrong states. Across Qwen3-4B and GPT-OSS-20B and two organism types, PEARL suppresses strongly while preserving task accuracy, with the clearest gains on the broad-trigger organism and GPT-OSS-20B; on Qwen3-4B with the narrow-trigger organism, SFT is more durable under heavy targeted retraining.

Undesired behaviors in language models can re-emerge during later fine-tuning, and standard post-hoc suppression had not been compared on immediacy, durability and capability together. PEARL adds adversarial prefixes, continuations from intermediate states of cached misaligned trajectories, to GRPO's on-policy rollouts so the policy practices recovering from partly wrong states. Across Qwen3-4B and GPT-OSS-20B and two organism types, PEARL suppresses strongly while preserving task accuracy, with the clearest gains on the broad-trigger organism and GPT-OSS-20B; on Qwen3-4B with the narrow-trigger organism, SFT is more durable under heavy targeted retraining.

The work presents a dynamic second-order geometric study of emergent misalignment (EM): tracking training trajectories shows directional Hessian curvature concentrates on semantic pivot tokens, Grassmannian projections show the harmful-safe gap widens mainly because safe-gradient overlap declines rather than through rotation toward a new malicious circuit, and on this basis it introduces a parameter-level Geometric Mitigation Framework that orthogonally projects the empirical harmful gradient subspace out of LoRA updates, suppressing free-generation EM by up to 80.0% on Qwen2.5-14B-IT and, across four open-weight instruction-based model families (3B–20B), teacher-forced evaluation shows the same harmful subspace controls the conditional support of frozen EM responses.

The work presents a dynamic second-order geometric study of emergent misalignment (EM): tracking training trajectories shows directional Hessian curvature concentrates on semantic pivot tokens, Grassmannian projections show the harmful-safe gap widens mainly because safe-gradient overlap declines rather than through rotation toward a new malicious circuit, and on this basis it introduces a parameter-level Geometric Mitigation Framework that orthogonally projects the empirical harmful gradient subspace out of LoRA updates, suppressing free-generation EM by up to 80.0% on Qwen2.5-14B-IT and, across four open-weight instruction-based model families (3B–20B), teacher-forced evaluation shows the same harmful subspace controls the conditional support of frozen EM responses.