MIRROR turns LLM personalization from imitation to preference internalization via reference-revealed on-policy self-distillation, leading three benchmarks with less forgetting
MIRROR treats a user's reference text as hindsight information and aligns the model's next-token distributions along its own on-policy trajectories with those of its reference-conditioned self, internalizing preferences instead of copying wording; MIRROR-F adds selective anchoring on informative, context-grounded reference tokens. Across three personalized generation benchmarks, two model scales, and reference-based plus LLM-judge evaluations, both achieve leading personalization and text quality with less forgetting than SFT baselines on three unseen tasks.