# Visual Grounding

Multimodal Rotary Position Embedding (MRoPE), widely used in vision-language models, assigns height and width coordinates to text tokens that have no inherent spatial structure, adding positional preferences to text-vision attention. The study separates intra-visual spatial encoding from cross-modal spatial effects, probing official pretrained checkpoints without training and running controlled training at 2B and 4B from identical initial parameters. Cross-modal spatial effects provide little extra benefit as spatial references and can even impair performance, while intra-visual spatial encoding remains useful for fine-grained grounding.

Multimodal Rotary Position Embedding (MRoPE), widely used in vision-language models, assigns height and width coordinates to text tokens that have no inherent spatial structure, adding positional preferences to text-vision attention. The study separates intra-visual spatial encoding from cross-modal spatial effects, probing official pretrained checkpoints without training and running controlled training at 2B and 4B from identical initial parameters. Cross-modal spatial effects provide little extra benefit as spatial references and can even impair performance, while intra-visual spatial encoding remains useful for fine-grained grounding.

ARC-AGI-3 asks agents to clear uninstructed interactive games with as few actions as possible, and handing a reasoning model the board as a raw integer matrix makes it collide with walls repeatedly. Sarbloh computes geometric facts such as reachability, enclosure, midpoint tethers and dihedral symmetry in host code with no learned parameters, keeps role labels as guesses each carrying a one-action test, and lets the agent commit multi-step paths from inside its Python kernel via await play(). On the official hidden set its score rose from an initial baseline of 0.27 to 16.53; the source describes that gain as a 13.7 efficiency improvement over the first Milestone winner, while the same table gives that winner a score of 1.21.

ARC-AGI-3 asks agents to clear uninstructed interactive games with as few actions as possible, and handing a reasoning model the board as a raw integer matrix makes it collide with walls repeatedly. Sarbloh computes geometric facts such as reachability, enclosure, midpoint tethers and dihedral symmetry in host code with no learned parameters, keeps role labels as guesses each carrying a one-action test, and lets the agent commit multi-step paths from inside its Python kernel via await play(). On the official hidden set its score rose from an initial baseline of 0.27 to 16.53; the source describes that gain as a 13.7 efficiency improvement over the first Milestone winner, while the same table gives that winner a score of 1.21.

When a mobile manipulator is told to put the toys away in the toy box, it usually relies on a task-agnostic semantic map that may lack the attributes, affordances or concepts the instruction needs. Feijoa has a language-model agent query a posed RGB-D scan after the instruction arrives, inspect the returned candidates and rewrite its queries until the reference resolves. On 103 open-ended grounding queries it reaches 94.2% exact match, against 68.0% for the best prebuilt representation, TextMap; on 50 real Spot trials it completes 35 tasks, 17 of them after recovering from a failed tool call or verification check.

When a mobile manipulator is told to put the toys away in the toy box, it usually relies on a task-agnostic semantic map that may lack the attributes, affordances or concepts the instruction needs. Feijoa has a language-model agent query a posed RGB-D scan after the instruction arrives, inspect the returned candidates and rewrite its queries until the reference resolves. On 103 open-ended grounding queries it reaches 94.2% exact match, against 68.0% for the best prebuilt representation, TextMap; on 50 real Spot trials it completes 35 tasks, 17 of them after recovering from a failed tool call or verification check.

Medical imaging foundation models have traded generality against pixel-level precision, and most return a label without evidence. EVLF-FM feeds both disease-level and pixel-level visual features into a language model, first with supervised fine-tuning and then with reinforcement learning on verifiable rewards, spanning six specialties and 10 imaging modalities. On internal diagnostic validation it reached the highest mean accuracy among evaluated models, 0.895, and averaged 0.842 Acc@0.5 for medical visual grounding across nine modalities.

Medical imaging foundation models have traded generality against pixel-level precision, and most return a label without evidence. EVLF-FM feeds both disease-level and pixel-level visual features into a language model, first with supervised fine-tuning and then with reinforcement learning on verifiable rewards, spanning six specialties and 10 imaging modalities. On internal diagnostic validation it reached the highest mean accuracy among evaluated models, 0.895, and averaged 0.842 Acc@0.5 for medical visual grounding across nine modalities.

TiCO replaces coordinate regression in thinking-with-images with a probe token that samples evidence over visual tokens, reaching 78.0% overall accuracy on HR-Bench 4K while generating only 6.2 tokens per query.

TiCO replaces coordinate regression in thinking-with-images with a probe token that samples evidence over visual tokens, reaching 78.0% overall accuracy on HR-Bench 4K while generating only 6.2 tokens per query.

Infographics must satisfy many text, numeric, chart and layout constraints in one image, and existing iterative refinement suffers a scale mismatch: local edits cannot fix global composition, while full regeneration destroys regions that are already correct. Stride separates constraint judgment from spatial grounding, and a lead editor agent routes each revision to a composer for global redraw or a retoucher for local text repair, accepting a candidate only when the verifier score strictly improves. On BizGenEval, Stride with the open-weight Infographic-V2 scores 69.5, above Nano-Banana-2.0 at 68.5 and Seedream-5.0-Pro at 65.7.

Infographics must satisfy many text, numeric, chart and layout constraints in one image, and existing iterative refinement suffers a scale mismatch: local edits cannot fix global composition, while full regeneration destroys regions that are already correct. Stride separates constraint judgment from spatial grounding, and a lead editor agent routes each revision to a composer for global redraw or a retoucher for local text repair, accepting a candidate only when the verifier score strictly improves. On BizGenEval, Stride with the open-weight Infographic-V2 scores 69.5, above Nano-Banana-2.0 at 68.5 and Seedream-5.0-Pro at 65.7.

Across five vision-centric tasks, adding labeled demonstrations to the context sharply raises accuracy on counterfactual examples that conflict with training priors while leaving prior-aligned accuracy nearly unchanged, indicating the models already have the visual capability and were being overridden by learned regularities.

Across five vision-centric tasks, adding labeled demonstrations to the context sharply raises accuracy on counterfactual examples that conflict with training priors while leaving prior-aligned accuracy nearly unchanged, indicating the models already have the visual capability and were being overridden by learned regularities.

Jag replaces coordinate-by-coordinate generation in visual grounding with direct regression of a bounding box in a single forward pass. Autoregressive multimodal models write a box as a coordinate sequence, so sequential decoding adds computation on top of the image-text processing that localization already needs. Jag attaches a lightweight regression head to Qwen3.5-0.8B, reads the last valid input-token state, and predicts all four continuous coordinates together under L1 and GIoU losses that jointly adapt the language backbone, visual merger, and head. On RefCOCO, RefCOCO+, and RefCOCOg it improves accuracy over its base model on every test split, with 65.97 ms mean single-request latency.

Jag replaces coordinate-by-coordinate generation in visual grounding with direct regression of a bounding box in a single forward pass. Autoregressive multimodal models write a box as a coordinate sequence, so sequential decoding adds computation on top of the image-text processing that localization already needs. Jag attaches a lightweight regression head to Qwen3.5-0.8B, reads the last valid input-token state, and predicts all four continuous coordinates together under L1 and GIoU losses that jointly adapt the language backbone, visual merger, and head. On RefCOCO, RefCOCO+, and RefCOCOg it improves accuracy over its base model on every test split, with 65.97 ms mean single-request latency.

Revisiting visual attention sinks in vision-language model decoders, this work finds a layer-dependent structure in which early and final decoder layers repeatedly collapse onto the same few image regions across prompts (Prompt-Invariant Sinks, PIS) while mid layers stay prompt-conditioned and drive vision-language alignment; it then proposes SAGE, a training-free intervention that applies an additive bias to visual-token attention logits using token-aligned ROI masks from standard vision backbones such as CLIP, ViT, and DINOv3, steering attention away from PIS toward query-relevant regions and improving visual grounding, reducing hallucination, and yielding gains on public benchmarks across LLaVA-1.5, Qwen2-VL, InternVL3, and NVILA.

Revisiting visual attention sinks in vision-language model decoders, this work finds a layer-dependent structure in which early and final decoder layers repeatedly collapse onto the same few image regions across prompts (Prompt-Invariant Sinks, PIS) while mid layers stay prompt-conditioned and drive vision-language alignment; it then proposes SAGE, a training-free intervention that applies an additive bias to visual-token attention logits using token-aligned ROI masks from standard vision backbones such as CLIP, ViT, and DINOv3, steering attention away from PIS toward query-relevant regions and improving visual grounding, reducing hallucination, and yielding gains on public benchmarks across LLaVA-1.5, Qwen2-VL, InternVL3, and NVILA.

The work proposes Spatial Latent Reasoning (SLR), which organizes pointing-gesture geometry and target-region appearance into an ordered sequence of continuous latent states: one spatial ray state supervised by fingertip position and pointing direction, followed by four states aligned with target-region features whose visual targets are built by parity pooling, which groups ROI tokens by row and column parity and averages each group; on EgoPoint-Ground it improves mIoU over same-backbone supervised fine-tuning by 2.8, 17.5, and 21.1 percentage points on Qwen3.5-4B, Qwen2.5-VL-7B, and Qwen3-VL-8B, with gains on both hard subsets, and reaches 77.6% precision at IoU 0.5 on YouRefIt.

The work proposes Spatial Latent Reasoning (SLR), which organizes pointing-gesture geometry and target-region appearance into an ordered sequence of continuous latent states: one spatial ray state supervised by fingertip position and pointing direction, followed by four states aligned with target-region features whose visual targets are built by parity pooling, which groups ROI tokens by row and column parity and averages each group; on EgoPoint-Ground it improves mIoU over same-backbone supervised fine-tuning by 2.8, 17.5, and 21.1 percentage points on Qwen3.5-4B, Qwen2.5-VL-7B, and Qwen3-VL-8B, with gains on both hard subsets, and reaches 77.6% precision at IoU 0.5 on YouRefIt.