# Geometric Supervision

Jag replaces coordinate-by-coordinate generation in visual grounding with direct regression of a bounding box in a single forward pass. Autoregressive multimodal models write a box as a coordinate sequence, so sequential decoding adds computation on top of the image-text processing that localization already needs. Jag attaches a lightweight regression head to Qwen3.5-0.8B, reads the last valid input-token state, and predicts all four continuous coordinates together under L1 and GIoU losses that jointly adapt the language backbone, visual merger, and head. On RefCOCO, RefCOCO+, and RefCOCOg it improves accuracy over its base model on every test split, with 65.97 ms mean single-request latency.

Jag replaces coordinate-by-coordinate generation in visual grounding with direct regression of a bounding box in a single forward pass. Autoregressive multimodal models write a box as a coordinate sequence, so sequential decoding adds computation on top of the image-text processing that localization already needs. Jag attaches a lightweight regression head to Qwen3.5-0.8B, reads the last valid input-token state, and predicts all four continuous coordinates together under L1 and GIoU losses that jointly adapt the language backbone, visual merger, and head. On RefCOCO, RefCOCO+, and RefCOCOg it improves accuracy over its base model on every test split, with 65.97 ms mean single-request latency.

The work proposes GeoScaffold, a geometric supervision framework that first learns a compact depth tokenizer on depth maps from training trajectories and freezes it, then fine-tunes the policy with a handful of learnable geometry query tokens whose hidden states reconstruct navigation-critical geometry such as depth, connectivity, and traversability, thereby internalizing geometry into the policy itself; after training the tokenizer, target generators, and reconstruction heads are discarded, leaving the backbone and action interface unchanged, and experiments show it consistently outperforms leading vision-only navigators on continuous VLN benchmarks, offering a practical paradigm for lightweight edge deployment of spatially aware embodied navigation models.

The work proposes GeoScaffold, a geometric supervision framework that first learns a compact depth tokenizer on depth maps from training trajectories and freezes it, then fine-tunes the policy with a handful of learnable geometry query tokens whose hidden states reconstruct navigation-critical geometry such as depth, connectivity, and traversability, thereby internalizing geometry into the policy itself; after training the tokenizer, target generators, and reconstruction heads are discarded, leaving the backbone and action interface unchanged, and experiments show it consistently outperforms leading vision-only navigators on continuous VLN benchmarks, offering a practical paradigm for lightweight edge deployment of spatially aware embodied navigation models.