Visual grounding drops token-by-token coordinates: a 0.8B model returns the box in one pass at 65.97 ms

Visual grounding output changes from generating coordinates as text to regressing the box directly in one forward pass. Multimodal models previously wrote a box as a coordinate sequence and parsed the decoded text into a box, so sequential decoding added output computation on top of the image-text processing needed for localization. On the official test splits of RefCOCO, RefCOCO+, and RefCOCOg, Jag improves over Qwen3.5-0.8B without task-specific fine-tuning on every split, with gains of 9.63 to 17.51 accuracy points.

A regression head reads the last valid input-token state and emits all four continuous coordinates together. NExT-Chat regresses a continuous box from a generated trigger token, and LocateAnything predicts discrete coordinates in parallel inside box-aligned token blocks; Jag reads an existing input state and generates no trigger. Training supervises normalized centers and dimensions with an L1 loss and box overlap with a GIoU loss, first fitting the head alone and then jointly updating the language backbone, visual merger, and head; inference returns the box in one multimodal forward pass without coordinate generation or text parsing.

The work targets single-object localization in static images and is evaluated on the referring-expression benchmarks RefCOCO, RefCOCO+, and RefCOCOg, with training on 321,327 referring expressions covering 28,158 images from their official training partitions. Inference efficiency is measured on one NVIDIA A100-PCIE-40GB GPU, with five models processing the same evaluation inputs sequentially. Extending the setting to multiple referents, video, or segmentation calls for task-specific design and evaluation, and specialized-domain applications should assess performance on representative data.

At the stricter IoU thresholds 0.75 and 0.9, the fine-tuned autoregressive model is more accurate on all three datasets, so the gap in precise box placement is larger than Acc@0.5 suggests. On RefCOCOg the autoregressive model also has higher mean accuracy, and the paired difference interval favors it. Inference efficiency depends on input resolution, expression length, hardware, and deployment settings, and the reported speedups describe the measured end-to-end workloads.