LLMs keep learning after deployment: sleep-style offline consolidation writes new facts into weights without erasing old skills
Deployed large language models stay frozen at a fixed knowledge cutoff, continual parameter updates cause severe forgetting of earlier tasks, and in-context learning loses information once the session ends. Google Research's sleep framework alternates active interaction with offline consolidation, in which the model opens pre-allocated Mixture-of-Experts memory blocks and updates weights using self-generated practice examples. In multi-stage continual translation into unseen languages such as Manchu and Kalamang, performance dropped near zero, while in-context learning, Cartridges and supervised fine-tuning suffered severe forgetting.