Lunar rover docking now driven directly by a vision-language-action policy: it beats a modular pipeline under the strongest sensor disturbances while making half the inferences
Synopsis
Autonomous lunar rover docking has relied on modular pipelines that split perception, mapping and planning, so perception errors propagate into planning. Researchers trained a vision-language-action policy with a Qwen3.5-4B backbone on safety-aware demonstrations from a privileged expert in the LuPNT lunar simulator, mapping a single monocular camera and a bird's-eye-view map directly to waypoints. Under dust and optical disturbances unseen in training, the policy achieved higher docking success in five of six disturbance conditions, with the largest advantage under the strongest perturbations; adaptive-horizon execution held 90% docking success while cutting policy inferences per successful scenario from 17.16 with a fixed 2 m horizon to 8.36.
Interpretation
Lunar rover docking can now be driven by one end-to-end policy that outputs driving waypoints directly from camera images, without separate perception, mapping and planning modules. Planetary rover autonomy has used modular pipelines with interpretable intermediate representations and explicit safety constraints, but perception errors propagate into downstream planning. In the LuPNT simulator the policy was compared with a modular baseline that uses the same Hybrid A* planner but operates on online-estimated hazard information, and it achieved higher docking success in five of six sensor-disturbance conditions, with the largest advantage under the strongest perturbations.
The policy feeds monocular camera frames and an online-built bird's-eye-view map into a vision-language model, and one inference returns ten waypoints spaced 0.8 m apart. The bird's-eye-view map is built by projecting successive camera frames onto an assumed ground plane with a homography and registering them with simulated odometry, so hazards that leave the camera view are retained. On held-out validation samples, adding the bird's-eye-view map raised execution-horizon bin accuracy from 75.51% to 81.15% and lowered the execution-horizon-truncated expected hazard contact rate from 2.13% to 1.89%.
The policy decides when to re-infer from a predicted curvature-conditioned execution horizon, executing farther in open terrain and correcting more often near hazards. A fixed horizon forces a choice between inference frequency and closed-loop safety, and longer horizons accumulate waypoint prediction errors. Adaptive horizon reached 90% docking success and 10% hazard contact versus 90% and 9% for the fixed 2 m horizon, while requiring 8.36 policy inferences per successful scenario versus 17.16, 51.3% fewer.
Perspective
All evaluations run in the Unreal Engine-based LuPNT lunar simulator, on procedurally generated scenarios using seeds unseen during training-data synthesis, with initial rover-to-lander separations of 30 to 50 m; the policy is trained only on sterile data without visual augmentation, and the disturbance conditions are out-of-distribution tests.
The policy remains sensitive to degraded observations under perturbation: hazard contact accounts for 87% (158/182) of its unsuccessful scenarios, with 42 of 100 scenarios at 50% dust opacity and 33 of 100 at High optical effect strength; the separate contributions of pretraining and architecture are not isolated; inference latency has a median of 225 ms on an RTX 5090, and a theoretical model estimates about 340 ms on a Jetson AGX Thor.