Sheridan Feucht

Across four vision-language models (Qwen3-VL-2B, Qwen3-VL-8B, Molmo2-7B, Llava-Next-34B), the work identifies attention heads causally necessary for OCR and shows they are general-purpose verbalization heads; summing their output-value matrices yields a verbalization lens that decodes image-token hidden states into interpretable semantic words at any layer, including layer 0, and whose pseudo-inverse provides concept vectors for editing image representations (e.g., replacing a tractor with a revolver), supporting the view that image representations are aligned with language space from early layers.

Across four vision-language models (Qwen3-VL-2B, Qwen3-VL-8B, Molmo2-7B, Llava-Next-34B), the work identifies attention heads causally necessary for OCR and shows they are general-purpose verbalization heads; summing their output-value matrices yields a verbalization lens that decodes image-token hidden states into interpretable semantic words at any layer, including layer 0, and whose pseudo-inverse provides concept vectors for editing image representations (e.g., replacing a tractor with a revolver), supporting the view that image representations are aligned with language space from early layers.