Multimodal rotary position encoding split apart: text-image spatial bias barely helps, and removing it sharpens grounding
Synopsis
Multimodal Rotary Position Embedding (MRoPE), widely used in vision-language models, assigns height and width coordinates to text tokens that have no inherent spatial structure, adding positional preferences to text-vision attention. The study separates intra-visual spatial encoding from cross-modal spatial effects, probing official pretrained checkpoints without training and running controlled training at 2B and 4B from identical initial parameters. Cross-modal spatial effects provide little extra benefit as spatial references and can even impair performance, while intra-visual spatial encoding remains useful for fine-grained grounding.
Interpretation
In vision-language models, the spatial positional preference between text and image patches can be switched off while the two-dimensional spatial encoding among image patches stays intact. Earlier MRoPE variants share one coordinate layout that binds text and visual coordinates into the same space, so cross-modal spatial effects and intra-visual spatial encoding cannot be controlled separately. On the official Qwen3-VL-2B, 4B and 8B checkpoints, changing only the coordinate assignment at inference without updating parameters leaves overall general-benchmark performance within 0.4% after removing text-vision spatial effects; by contrast, removing intra-visual spatial encoding drops grounding ACC@0.5 from 82.4%, 87.1% and 86.1% to 18.5%, 16.3% and 5.3% for the 2B, 4B and 8B models.
The method gives each image patch two coordinate options: one keeps its original height and width position, the other places all patches on the trajectory followed by text coordinates. This attention-specific coordinate assignment lets text-vision attention and vision-vision attention each choose original or neutral coordinates, yielding four comparable configurations. Four configurations are trained on Qwen3-VL-2B and 4B from identical initial parameters with the same data and optimization settings; under MRoPE-I, removing text-vision spatial effects raises average spatial understanding by 1.2% and 1.3% and grounding ACC@0.9 by 1.2% and 1.2%.
The same trend appears under different coordinate layouts and frequency allocations, including VideoRoPE, HoPE and IL-RoPE. The earlier comparison was built only on MRoPE-I, so it could not show whether the result came from one particular positional design. On Qwen3-VL-2B and 4B, removing text-vision spatial effects raises ACC@0.9 by 2.8% and 1.4% for VideoRoPE and by 9.1% and 7.3% for HoPE; across the four model-variant combinations the average ACC@0.9 gain is 5.2%, compared with 2.8% for preserving intra-visual spatial encoding.
Perspective
The demonstrated scope is training-free probing on Qwen3-VL-2B, 4B and 8B, plus controlled training on Qwen3-VL-2B and 4B from identical initial parameters, covering MRoPE-I, VideoRoPE, HoPE and IL-RoPE variants across general, spatial-understanding and referring-expression grounding benchmarks.
Whether these findings generalize to the substantially larger training regimes used in large-scale vision-language models remains to be investigated; pretrained models were trained with these positional preferences throughout, so they may have learned to adapt to their influence.