STAVE replaces in-context demos with two task vectors, beating per-layer interventions on six multimodal models at zero-shot inference cost
The work proposes STAVE, which replaces in-context demonstrations with two task-specific vectors added to the input embeddings of a frozen LLM or LMM, a readout vector updating the answer-producing tokens and a context vector updating the other structural token groups, both trained with answer labels on prompts with and without demos; across six LMMs and five LLMs, STAVE matches or outperforms per-layer interventions with far fewer task parameters, surpasses 15-shot ICL and prior task vectors on 18 text tasks, and runs at zero-shot inference cost.