Skip to main content
Back to timeline
arXivSource publication:

Visual reasoning drops coordinate regression: a probe token samples evidence over visual tokens, hitting 78.0% on 4K with about 1.2% of the tokens

Synopsis

TiCO replaces coordinate regression in thinking-with-images with a probe token that samples evidence over visual tokens, reaching 78.0% overall accuracy on HR-Bench 4K while generating only 6.2 tokens per query.

Source-provided article image: Thinking with Images without Coordinates
Figure 2 · arXiv

Interpretation

When a multimodal large language model reasons over a high-resolution image, it no longer has to emit bounding-box coordinates and crop the image first; it selects question-relevant evidence regions directly among visual tokens. Earlier thinking-with-images methods rely on explicit coordinate prediction for grounding: the model regresses numerical coordinates and then conditions further reasoning on the cropped region. On HR-Bench 4K this approach reaches 78.0% overall accuracy, above the reported 77.5% for Mini-o3, 75.1% for DeepEyes and 77.1% for TreeVGR; on HR-Bench 8K it reaches 76.3%, above 73.3% for Mini-o3 and 72.6% for DeepEyes.

The model is prompted to emit a special probe token, and the hidden state of that token is compared with every visual token to produce a similarity map over the image. The method is named TiCO, thinking with images without coordinates; it formulates evidence discovery as similarity-induced region sampling instead of numeric bounding-box regression. Evaluation spans V* Bench, HR-Bench 4K/8K, VisualProbe, MME-RealWorld and POPE, with 66.9% on MME-RealWorld and 89.1% overall on POPE, trained on Qwen-2.5-VL-7B-Instruct.

The similarity map is thresholded into a binary mask, grouped into spatially connected components, normalized into a probability distribution over those components, and then sampled by importance sampling. Unlike deterministic top-scoring selection, importance sampling keeps stochastic exploration over semantically plausible regions and avoids over-committing to potentially noisy peaks. Under identical training and rollout settings, importance sampling scores 90.6% on V* Bench and 76.3% on HR-Bench 8K, above top-scoring, random and token-wise sampling alternatives.

Perspective

The report covers TiCO's evaluation on V* Bench, HR-Bench 4K/8K, VisualProbe, MME-RealWorld and POPE, together with training and ablations based on Qwen-2.5-VL-7B-Instruct. It is aimed at researchers and engineers working on multimodal reasoning, visual grounding and inference efficiency. The source states no next-step experiment plan.

The method still relies on threshold-based binarization, and the threshold may require manual selection. It does not employ an explicit multi-step thinking process, so it removes reflective behaviors such as iterative refinement, which may limit adaptability on tasks requiring complex deliberation or multi-hop reflection. High-resolution reasoning remains sensitive to grounding quality: if the selected sub-region fails to cover the semantically relevant evidence, subsequent reasoning may struggle to recover the correct answer. On V* Bench the attribute subset and overall score are slightly below TreeVGR, which the authors attribute to semantic evidence aggregation possibly filtering out fine-grained local textures.