Frozen models accumulate external expertise during deployment: up to 34.2% online gain on medical tasks across six benchmarks, with zero-step transfer to other models
The work presents a model-agnostic framework that lets frozen LLMs and VLMs keep learning during deployment through three forms of external expertise: a Skill guiding reasoning and tool use, a Knowledge Memory storing reliable facts, and a Multimodal Knowledge Base that keeps visual cases and guides the model to compare each retrieved case with the current image; an update is kept only if it helps on new cases without degrading earlier ones. Across six benchmarks and four base models, the framework improves online medical performance by up to 34.2% over the base model, generalizes to unseen cases, and transfers to other models without further optimization.