HKU-led team releases Compass 1 navigation model: trained on monocular RGB video, it reaches 79.4% open-world success against 17.8% for the strongest baseline
Synopsis
A group from the University of Hong Kong, Shanghai Innovation Institute, Archon Robotics and other institutions released Compass 1, an embodied navigation foundation model trained on monocular RGB video that adds sparse keyframes and an explicit history map to a unified understanding-and-generation backbone, letting an agent backtrack through nearly two minutes of visual history. Across zero-shot real-world trials in four scenarios (office, terrace, complex outdoor and pedestrian-dense street) and eight tasks, Compass 1 reached 79.4% overall success versus 17.8% for the strongest baseline, AwareVLN; on persistent history backtracking the baseline scored 0% and Compass 1 scored 58%.
Interpretation
For the first time, an embodied navigation agent can use visual memory from nearly two minutes earlier in real open-world settings, rather than relying on only the most recent frames. Earlier navigation architectures either fed the full uncompressed visual stream into the backbone or applied heuristic compression or fixed-rate uniform sampling, sustaining effective context over only a handful of frames, so persistent history backtracking could not be completed. Across four scenarios, eight tasks and 10 trials per task, 320 zero-shot real-world evaluations in total, Compass 1 reached 79.4% overall success against 17.8% for the strongest baseline, AwareVLN; on persistent history backtracking the baseline scored 0% and Compass 1 scored 58%.
History is handled without changing the network architecture: sparse keyframes and a map of the walked path are fed to the backbone together as image inputs. Keyframes are chosen with action guidance, sampled densely around turns and transitions and sparsely every 3 seconds on straight segments, compressing a one-minute 240-frame video to about 40 frames on average. Past keyframes are drawn with numbered trajectories on a 2D canvas, the History Representation Map, with the current pose marked by a red letter C and an arrow, and the backbone processes it alongside keyframes as native image input.
The map alone is not enough; three structured visual-question-answering supervision categories turn it into usable reasoning. When the History Representation Map is supplied as input without this supervision, the backbone ignores the guidance, so the protocol is the mechanism that makes the map effective. The three question categories cover task-stage grounding, exploration consistency and spatiotemporal target retrieval; removing spatiotemporal target retrieval renders the map input completely ineffective and severely degrades backtracking.
Perspective
Evaluation covers office, terrace, complex outdoor and pedestrian-dense street scenarios with 10 trials per task; physical validation centers on a Unitree Go2 quadruped with the camera at about 0.5 m, with cross-embodiment tests on a wheeled robot, humanoid and drone; persistent history backtracking currently has no corresponding simulation benchmark.
Persistent history backtracking remains difficult: occlusion by obstacles on the return path can make the robot lose its return heading even when it knows the approximate target location; humanoid and drone platforms show slight performance drops from pose jitter and aerodynamic turbulence; gains from web video data are limited by its quality.