# Multimodal Rotary Position Embedding

Multimodal Rotary Position Embedding (MRoPE), widely used in vision-language models, assigns height and width coordinates to text tokens that have no inherent spatial structure, adding positional preferences to text-vision attention. The study separates intra-visual spatial encoding from cross-modal spatial effects, probing official pretrained checkpoints without training and running controlled training at 2B and 4B from identical initial parameters. Cross-modal spatial effects provide little extra benefit as spatial references and can even impair performance, while intra-visual spatial encoding remains useful for fine-grained grounding.

Multimodal Rotary Position Embedding (MRoPE), widely used in vision-language models, assigns height and width coordinates to text tokens that have no inherent spatial structure, adding positional preferences to text-vision attention. The study separates intra-visual spatial encoding from cross-modal spatial effects, probing official pretrained checkpoints without training and running controlled training at 2B and 4B from identical initial parameters. Cross-modal spatial effects provide little extra benefit as spatial references and can even impair performance, while intra-visual spatial encoding remains useful for fine-grained grounding.