Robot policies ship as weights alone: letting a world action model replay its own generated past cuts forgetting by over 90%
Synopsis
A world action model can regenerate complete rollouts of previously learned tasks from their instructions alone, and after truncation and physical-consistency filtering, replaying them cuts the forgetting of sequential fine-tuning by over 90% on four LIBERO suites and five real-world tasks without keeping any data from previous tasks.
Interpretation
A sequentially fine-tuned world action model adapts well to a new task but loses almost everything it learned before; replaying rollouts the model generates itself for those earlier tasks makes that forgetting much smaller. Mitigating forgetting previously relied on experience replay, which stores real demonstrations of previous tasks, yet a robot policy often arrives as weights alone and its training data is not shared with it. Across the four LIBERO suites, sequential fine-tuning reaches only 12.5% average success, and replaying raw generated rollouts raises that to 70.4%, still far below experience replay on real data.
Generated rollouts are rectified before replay: motion after task completion is cut off, and rollouts whose actions disagree with their observations are discarded. Raw generated rollouts have two problems, the model does not recognize task completion and keeps generating purposeless motion, and under open-loop generation the actions gradually drift away from the generated observations. A vision-language model acts as an evaluator that locates the completion point and truncates each rollout, then the generated actions are executed in a robot-only simulator specified by the robot URDF and the resulting optical flow is compared with flow estimated from the generated observations, keeping the most consistent rollouts.
Rectified generative replay nearly matches experience replay on real data, and both rectification steps are necessary. Removing truncation drops LIBERO-Spatial success to 70.8% and removing filtering drops it to 80.8%, both below the 94.0% when both are used. Across the four LIBERO suites the average success is 92.7% with a forgetting measure of 2.33, only 1.9 percentage points below experience replay on real demonstrations; on five real-world tasks the success is 89.5% versus 91.8%.
Perspective
The report covers how a world action model behaves under data-free continual learning, evaluated on four LIBERO suites and five real-world robot tasks, with replay data generated from task instructions and cached for reuse. It is intended for researchers and engineers working on continual learning and generative replay for robot policies. Directions the source proposes next include aligning observations and actions during training, or grounding generation with an external world model.
A small gap to experience replay remains and is largest on long-horizon tasks, indicating that filtering mitigates rather than removes observation-action drift. In the real-world experiments the replay-based methods give up 4 to 5 percentage points of plasticity, which the source attributes to interference between tasks that still needs to be addressed.