Skip to main content
Back to timeline
arXivSource publication:

Robot imagines the corridor before takeoff: video diffusion supplies virtual views, RRT finds its own path in the reconstructed obstacle map

Lead

From one RGB image and a task prompt, a video diffusion model generates virtual camera views that ORB-SLAM3 and aligned dense depth turn into a plannable 3D space; planning-feasible scenes rise from 13/19 at P20 to 15/19 at P81 across a 19-scene study.

Source-provided article image: HorizonNav: Geometry-Guided Video Diffusion for Anticipatory 3D Robot Navigation
Fig. 1 · arXiv

Story

A robot can start from a single RGB image and a task prompt, generate virtual camera views, and reconstruct them into a 3D space with obstacles that a path planner can search. Earlier generative world models mostly used predicted observations to score candidate actions or derive waypoints, leaving the final decision tied to the generated sequence or to a learned world representation. In a 19-scene horizon study, planning-feasible scenes rise from 13/19 at P20 to 15/19 at P81 and successful RRT trials from 62/90 to 75/90, indicating that more virtual viewpoints expose geometric evidence useful to the planner.

The generated video serves only as a source of virtual observations, not as the trajectory the robot will execute; the planner searches the reconstructed obstacle representation independently. A generated trajectory need not be safe to execute, and generated views may contain deformations or inconsistent occlusions that alter free-space connectivity. Of 32 corridor branch-prefix cases, 25 are evaluable in one common metric map; 6 generated camera trajectories violate the 0.20 m collision criterion, and independent RRT recovers a feasible alternative in 5 of those 6.

Dense depth cannot be treated as obstacle evidence merely because it is temporally smooth, and adding dense geometry can reduce planning utility. The sparse SLAM reconstruction supports 5/5 RRT successes in the S06 gate scene, while the corrected dense VDA+SLAM geometry closes that opening and yields 0/5. That comparison uses the same collision geometry, showing that local depth errors in dense predictions create false obstacles; the experiment does not support a claim that VDA fusion is more accurate than sparse SLAM.

What to watch

The report addresses researchers and engineers working on robot navigation, SLAM, and generative world models. The experiments cover indoor scenes, one evaluated video generator, fixed structured prompting without a prompt ablation, and an RRT planning layer shared by ground robots and UAVs. Next steps stated in the source include a protocol-matched single-image depth baseline, surface-level reconstruction metrics against independent 3D ground truth, generator and prompt ablations, broader robot and environment testing, and online verification that can accept, revise, or reject imagined geometry as new physical observations arrive.

The source limits its claim to generated virtual observations providing enough trackable structure for pre-motion planning in some scenes, not proof that the true unseen 3D scene has been recovered. Global scale consistency, local surface accuracy, and planning utility are distinct properties; the LiDAR diagnostic reports roughly 0.30 m mean discrepancy at selected matched structural landmarks and is not a complete surface-reconstruction benchmark. The six physical flights give limited end-to-end evidence in the tested gate configuration, not statistical validation of general reconstruction accuracy.

Sources