Skip to main content
Search

# LLM Unlearning

4 related articles

Subscribe to updates
  1. aclanthology.org

    FOM-UL unlearns by editing only a few layers: under 4-bit quantization its residual memorization stays below whole-model baselines with higher retain utility

    Unlearning a large language model no longer requires diffuse updates across the whole model. FOM-UL scores each transformer layer by the ratio of its forget-set gradient norm to its retain-set gradient norm, updates only the few high-scoring layers, and adds layers one at a time when forgetting falls short. Across TOFU, KnowUnDo and MUSE-style evaluations it leaves less residual memorization than GA, NPO and KLD baselines while keeping retain-set utility close to the vanilla model, and it holds a favorable forgetting-utility trade-off under 8-bit and 4-bit post-training quantization, though residual memorization does rise under 4-bit relative to full precision.

    Oak Ridge National LaboratoryFlorida International University
    FOM-ULLLM UnlearningPost-Training QuantizationAdversarial Robustness
  2. arXiv

    Nullify achieves training-free LLM unlearning via null-space activation steering, raising TOFU forget10 FQ to 0.58 while holding MU at 0.62, matching Retrain

    Nullify introduces a training-free, weight-preserving inference-time activation steering method for LLM unlearning: it builds a forget direction from IDK-contrastive activations and confines every correction to the subspace orthogonal to retained-knowledge activations via a null-space projector, matching or surpassing baselines in forget quality on TOFU and MUSE while keeping model utility near-lossless at a small fraction of their compute.

    Zhejiang UniversityTianjin UniversityFudan UniversityKing Abdullah University of Science and TechnologyPeking University
    Wei Zhai（First author）
    NullifyLLM UnlearningActivation SteeringNull-Space Projectionknowledge unlearning
  3. arXiv

    Representation geometry explains why forgotten knowledge is easily relearned, motivating an MCU unlearning method that targets minor components

    This work studies, from a representation geometry perspective, why unlearned models rapidly recover forgotten knowledge under relearning attacks, finding that existing unlearning methods mainly optimize along dominant components while leaving minor components largely unchanged, that modifications in dominant components are easily reversed during relearning whereas minor components resist such reversal, and proposes Minor Component Unlearning (MCU), which explicitly targets minor components and is validated on three datasets to substantially improve resistance to relearning attacks.

    Alibaba Group (China)Beihang UniversityShanghai University of Finance and EconomicsSouthern University of Science and Technology
    Zeguan Xiao（First author）
    LLM UnlearningMinor Component Unlearning (MCU)Relearning Attack
  4. arXiv

    UDS measures LLM unlearning depth via activation patching, reporting top faithfulness and robustness across 150 models and 20 metrics

    The work introduces the Unlearning Depth Score (UDS), which first uses a retain-model baseline to identify layers encoding target knowledge and then measures via activation patching how much of that knowledge the unlearned model has erased on a 0-1 scale; in a meta-evaluation across 20 metrics on 150 unlearned models spanning 8 methods, UDS is reported to achieve the highest faithfulness and robustness, and case studies show it uncovers residual knowledge obscured from observational metrics by representational shifts, with erasure depth varying across prompt types.

    Sungkyunkwan University
    Jaeung Lee（First author）
    Unlearning Depth ScoreActivation PatchingLLM Unlearning

You’re all caught up