Skip to main content
Search

# Visual Attention Heads

1 related articles

Subscribe to updates
  1. arXiv

    Visual-head discovery scores shift with where attention is read: necessity tracks the query span, while steering at default strength is matched by random heads

    The internal choices inside a visual-head discovery score reshape which heads it returns in vision-language models. On Qwen3-VL-8B-Instruct, fully crossing query type, discovery corpus and the token positions whose attention is read, and testing all 12 head-sets with steerability and necessity interventions, only necessity follows the attention source: query-span head-sets are more necessary than last-token ones, and the two referring plus query-span sets lie 17–19 standard deviations above 250 layer-matched random sets. At the default strength, no discovered head-set steers better than its layer-matched random sets.

You’re all caught up