Long-horizon navigation stops hoarding frames: compressed, pose-grounded memory lifts RxR-CE success by 6.5 points
Synopsis
MemoryVLN compresses long-horizon vision-language navigation history into pose-grounded spatial memories and reuses them across episodes within the same environment, raising success rate by about 2% on R2R-CE and 6.5% on RxR-CE.
Interpretation
The bottleneck in long-horizon navigation is recast as how memory is organized, not simply how large the context window can be made. Earlier continuous-environment navigation methods mostly encode accumulated observations as video or context sequences, keep slow-fast contexts, or rely on bounded histories and explicit maps, and rarely place compression, ego-aware representation, cross-context organization and retrieval in one mechanism. On the val-unseen splits of R2R-CE and RxR-CE, success rate improves by roughly 2% and 6.5% over the strongest data-comparable baselines, while training uses only standard VLN trajectories, without DAgger rollouts or additional navigation data.
Historical observations become compact, spatially grounded navigation experience that can be retrieved and reused across episodes within one environment. Long Sequence Compression samples along a forgetting-curve-inspired schedule to keep dense recent evidence and sparse representative distant observations under a fixed token budget, and Ego-Aware Visuospatial Representation binds 2D semantic tokens, 3D geometric tokens and 6D agent poses onto the same memory. A hierarchical memory bank holds in-episode experience in a FIFO short-term pool and a cosine-weighted long-term pool, while a context-level bank performs warm start at a 0.6 cosine threshold and confidence-gated write-back at 0.4 and is retained only within the associated environment scope; memory enhancement then retrieves the most relevant entries for the current state and residually fuses them with current-frame tokens.
Ablations show the four modules carry distinct roles, and compression is what makes long-horizon inference run at all. Removing Long Sequence Compression lets historical visual tokens grow with trajectory length and causes out-of-memory on both benchmarks, while removing pose encoding, the in-episode bank or the context-level bank degrades metrics. On R2R-CE, restoring pose encoding raises OS, SR and SPL by 3.16, 2.01 and 1.01 points, and the in-episode bank lifts SR and SPL from 53.34 and 49.06 to 54.76 and 50.26; on RxR-CE the in-episode bank raises SR, SPL and nDTW by 1.15, 2.29 and 1.69 points.
Perspective
The report is aimed at researchers and engineers working on embodied navigation and vision-language models, and covers the val-unseen splits of R2R-CE and RxR-CE, qualitative comparisons in Habitat simulation, and five instruction demonstrations on a Unitree GO2 quadruped in a real office. The context-level bank persists only within the associated environment scope, and the in-episode bank resets at each episode boundary, so information from one scene does not enter another scene's evaluation. The real-robot experiment is positioned as a qualitative demonstration rather than a large-scale physical-world benchmark.
On R2R the success-rate and SPL gains are not statistically significant under paired tests, while the RxR gains are; all reported results come from a single training run with seed 42, with no across-run variation reported. The failure cases show the context-level bank writing back a failed episode with a final navigation error of 5.39 m, which a later episode retrieves at a cosine similarity of 1.0000, yet disabling the bank still leaves that episode failing at 56 steps and 10.28 m error, so the bank is not the sole cause. Long instructions also show subgoal-progress loss and premature stopping.
