VHOP-Router hands multi-step visual retrieval to the embedding model, lifting retrieval from under 5% to 76.3%

Introduces VHOP, a flexible data generation framework and benchmark with five core difficulty levels that tests both visual matching and search planning. Prior visual agentic search lacked a systematic, difficulty-controlled evaluation instrument; VHOP places difficulty tiers and two capability axes (matching and planning) in one framework. The abstract states the framework was used to study the problem systematically and to train VHOP-Router, indicating it can actually generate training and evaluation data.

Introduces VHOP-Router, an end-to-end training pipeline combining supervised fine-tuning, online imitation learning, and reinforcement learning that transforms a standard embedding model into an autoregressive multi-step retriever. Unlike current pipelines where the agent must issue text queries at every intermediate step, this model operates directly in visual latent space and retrieves linked image chains in a single tool call without intermediate text queries. The abstract explicitly names the three training stages and the single-tool-call, visual-latent-space mode of operation, and states that native LLM capabilities are left entirely intact.

Retrieval and agentic search results: retrieval performance rises from under 5% to 76.3%; task success improves by 52.7%; average token length falls 61% from 1886 to 728. The abstract notes that upgrading the agent yields only a 3.7% gain, indicating the bottleneck lies in the retrieval tool rather than the agent, and that offloading multi-step navigation to the tool yields larger gains. The abstract provides concrete percentages and token figures alongside an agent-upgrade comparison condition.

Efficiency and generalization: versus a strong baseline retrieving the top 50 results per step, it maintains superior performance while reducing in-context images by 23x and cutting cumulative API payload by 35x, and generalizes robustly to unseen difficulty levels and realistic test sets. Treats efficiency (context size, API payload) and generalization as evaluation dimensions alongside retrieval accuracy. The abstract reports multiplier-level comparisons and a generalization statement, but does not name the specific test sets or sample sizes.

The work targets agentic retrieval settings that require multi-step visual evidence chains: when visual clues are hard to describe in text, or when a single-step retriever fails to surface necessary intermediate evidence within its top results, multi-step navigation is offloaded to the retrieval tool itself. The abstract frames it for visual agentic search and states that native LLM capabilities are left entirely intact, so it can be inserted into existing agent pipelines without modifying the model itself. For researchers and engineering teams building retrieval tools and evaluating visual search planning, VHOP offers a framework for generating five-level difficulty data and VHOP-Router offers an end-to-end training path.

The abstract does not give the data size per VHOP difficulty level, the model scale or training cost of VHOP-Router, or statistical significance for the success-rate gain, nor does it list the source and composition of the 'realistic test sets'; the 23x and 35x efficiency comparisons rest on a baseline that retrieves the top 50 results per step, and whether the conclusion holds under other baselines remains to be shown in the body. The abstract also claims generalization to unseen difficulty levels without reporting per-level breakdowns, so readers judging stability across the difficulty curve would need the body's tables.