Zero-shot language reasoning verifies nearby cross-view candidates but cannot rerank a visual retriever's mistakes

It proposes structured spatial verification, recasting candidate ranking as a pointwise LLM-as-a-judge task over pools of geographically adjacent hard negatives, reaching 21.0% Recall@1 and MRR 0.419 on ten-candidate pools, roughly double random ranking (10.0%) and above the sentence-transformer (14.5%) and raw BERT (11.7%). Unlike metric-learning retrieval that depends on paired supervision and tuned negative sampling, the pipeline is fully zero-shot, and each verdict names which structured fields agree and which conflict, something an embedding distance cannot provide. Evaluated on 600 queries whose pools contain the true tile plus the nine geographically nearest wrong tiles, the nearest lying 55 to 61 meters away on average, with all three baselines scoring the identical pools; the gap to TF-IDF at 20.0% lies inside the sampling interval, so the authors treat close orderings as ties.

A field ablation attributes most of the signal to road topology and geometry: exposing only topology and geometry fields gives 17.5% Recall@1, within sampling noise of the 20.8% all-fields condition, while orientation cues alone fall to 5.8%, at or below the random baseline. This locates the cross-view-transferable scene properties in road geometry rather than in orientation or landmark fields, indicating which content survives conversion into language. Each condition repeats the pointwise verification task on 120 queries using the same query list and pools; the authors note that at this sample size the binomial interval on Recall@1 is roughly several points, so only the orientation-only condition differs beyond noise and the remaining ordering is suggestive.

On the queries a trained visual retriever ranks wrongly, reranking is asymmetric: image-only reranking recovers substantially more than text-only, while text-only and image-plus-text stay close to chance, with the combined variant returning to the text-only level at Recall@1. This delimits where language reasoning helps, namely when candidates are geographically close but lexically confusable, and where it does not, namely when candidates are visually near-identical, and it argues against naive concatenation of the two modalities. Only 591 of 9,826 queries have an incorrect visual top-1, and the true tile is in the top-10 for about nine in ten of them, so the effective chance level is about 11% rather than 10%; the three variants ran on identical query lists with the same seed, making the comparison a clean ablation of input modality.

The descriptions are consistent and faithful across the two views but not discriminative: dominant anchors are near saturation, with "building" appearing in more than nine in ten descriptions in both views, and paired descriptions reach a high, tightly concentrated mean BERT cosine similarity, with mismatched pairs frequently scoring above correct ones. This explains why zero-shot full-pool text retrieval stays below 1% Recall@1 and attributes the failure to distinctiveness rather than to description accuracy. Based on 9,826 schema-valid pairs recovered by the repair module (Chicago 2,469, New York 2,464, San Francisco 2,450, Seattle 2,443), with three text representations compared on the identical reduced pool and descriptions generated once and cached so text methods read byte-identical inputs.

The results apply to the zero-shot setting: descriptions come from prompting, baselines receive no contrastive training, and the judge has never seen a labeled cross-view pair, so what is measured is the content of prompted descriptions rather than an upper bound on what a trained system could extract. The verification stage is positioned as a check that follows a geographic prior, suited to settings where GPS, dead reckoning or a first-stage retriever has already narrowed the search to a small neighborhood; for deployments that must retrieve directly from thousands of tiles, the text representations studied here do not apply. On interpretability, every verdict carries a field-level rationale that can be checked against the two descriptions, letting an operator reject a match when the evidence is weak, which is why the authors recommend human-in-the-loop deployment. The authors also name two follow-up directions: training a verifier on the hard negative pools, and training the description model with an objective that rewards text making the correct tile identifiable.

Generation and reasoning are not fully separated: the authors note that descriptions supporting verification fail on visually confusable candidates while images on those candidates do not, but the prompt format also changes between the two settings from pointwise to listwise, so the whole gap cannot be attributed to description content, and a planned oracle-description test would separate the two effects. The grounding audit reuses the model that wrote the descriptions, so self-preference is expected, and the authors read the grounded share as optimistic and the hallucinated share as a floor; the audit also covers only distinctive_anchors, not the topology fields that carry most of the verification signal. VIGOR panoramas are not centered on their tiles, so two faithful descriptions of one location can disagree, and the judge compares them as if they described the same content, producing avoidable false rejections. The reduced pool inflates absolute recall for every method, single runs leave intervals of roughly several points on some metrics, which is why close orderings are treated as ties, and reranking is reported on the hard subset with end-to-end figures given only as upper bounds. In addition, proper nouns such as street names appearing in satellite descriptions suggest that some share of verification accuracy may come from memorized geography; the authors expect this to be a small effect overall but recommend preferring generic scenes when reading individual successes.