Skip to main content
Back to timeline
arXivSource publication:

Robots now predict before they act: picking among four action chunks lifts RoboCasa success from 7.24% to 8.28%

Synopsis

A robot policy no longer executes generated actions directly; it first predicts the future state of each candidate and runs the one with the highest predicted reward. LY-GWM-RoboCasa-Human300 combines a Human300-fine-tuned GR00T-N1.5 action generator, an action-conditioned graph dynamics model, and a learned binary state-reward scorer, sampling four action chunks at inference. On 50 RoboCasa 1.0.1 pretrain-kitchen tasks with 50 episodes each, local success rose from 7.24% in the historical self-trained baseline to 8.28%, 26 more successes.

Source-provided article image: LY-GWM-RoboCasa-Human300: Prediction-Guided Action Selection with a Graph Dynamics Model
arXiv · Page 2

Interpretation

Actions sent to the environment now pass through a future-state prediction and scoring step, so predictions can change what the robot actually does. Previously the fine-tuned GR00T-N1.5 action generator produced actions directly, with no prediction involved in selection. On 50 RoboCasa 1.0.1 pretrain-kitchen tasks with 50 episodes each, the system recorded 207/2,500 successes (8.28%) against 181/2,500 (7.24%) for the historical self-trained GR00T-N1.5 baseline, a difference of 26 successes or 1.04 percentage points.

Selection rests on three parts: action generation, future-state prediction, and reward scoring, all frozen during evaluation. The action-conditioned graph dynamics model predicts latent features and robot state after each candidate action, and the binary state-reward scorer returns a reward logit for each candidate. At inference the policy samples four action chunks, predicts the future for each over a 16-step horizon, and executes the candidate with the highest predicted-state reward logit; none of the three components is updated during evaluation.

Perspective

The report covers two separately reported result sets: a local evaluation on 50 RoboCasa 1.0.1 pretrain-kitchen tasks with 50 episodes each, and a maintainer re-evaluation of the submission reporting 354/1,500 successes across 50 tasks. The intended reader is a researcher or engineer working on robot manipulation policies and inference-time selection. The source states no follow-up experiment plan.

Local and official results are reported separately, and their difference is not attributed to a model improvement. The local difference is 26 successes against a historical self-trained baseline, not a contemporaneous control. The atomic group improves from 141/900 to 150/900, and leaderboard rates are 48.3%, 16.3%, and 3.1% for Atomic-Seen, Composite-Seen, and Composite-Unseen, so composite unseen tasks remain low.