# knowledge unlearning

Nullify introduces a training-free, weight-preserving inference-time activation steering method for LLM unlearning: it builds a forget direction from IDK-contrastive activations and confines every correction to the subspace orthogonal to retained-knowledge activations via a null-space projector, matching or surpassing baselines in forget quality on TOFU and MUSE while keeping model utility near-lossless at a small fraction of their compute.

Nullify introduces a training-free, weight-preserving inference-time activation steering method for LLM unlearning: it builds a forget direction from IDK-contrastive activations and confines every correction to the subspace orthogonal to retained-knowledge activations via a null-space projector, matching or surpassing baselines in forget quality on TOFU and MUSE while keeping model utility near-lossless at a small fraction of their compute.

Addressing the look-ahead bias that arises when LLMs are applied to financial predictive tasks because they were trained on long time-series data, and the prohibitive cost of retraining frontier models from scratch with a specific knowledge cutoff, the work introduces a fast, effective, and low-cost alternative that guides generation at inference time by adjusting the logits of a large base model using a pair of smaller specialized models -- one fine-tuned on information to be forgotten and one on information to be retained -- and reports that the method effectively removes both verbatim and semantic knowledge, corrects biases, and outperforms prior methods.

Addressing the look-ahead bias that arises when LLMs are applied to financial predictive tasks because they were trained on long time-series data, and the prohibitive cost of retraining frontier models from scratch with a specific knowledge cutoff, the work introduces a fast, effective, and low-cost alternative that guides generation at inference time by adjusting the logits of a large base model using a pair of smaller specialized models -- one fine-tuned on information to be forgotten and one on information to be retained -- and reports that the method effectively removes both verbatim and semantic knowledge, corrects biases, and outperforms prior methods.