A2WM refines frozen imitation policies without retraining, lifting mean success from 27.18% to 59.51%
Synopsis
Small execution errors in imitation policies accumulate into failure, and earlier fixes relied on additional on-policy learning or costly online planning. A2WM learns local action sensitivity offline from action-perturbed rollouts of a frozen policy, predicting task-relevant geometry as a nominal trajectory plus a linear response to corrections, which turns online refinement into one model evaluation and one convex quadratic program solve. Across four simulated manipulation tasks it reaches the highest mean success among the evaluated baselines, 59.51% versus 27.18% for Diffusion Policy, with 3.8 ms mean refinement overhead per control step excluding base-policy sampling.
Interpretation
Refinement no longer requires updating policy weights, and a frozen imitation policy can still be improved substantially in closed loop. The unmodified Diffusion Policy averages 27.18% success across the four simulated manipulation tasks, with only 6.12% on RoundTable and 5.08% on Lamp. A2WM raises mean success to 59.51% and achieves the highest observed success on each task, tying GPC-OPT at 78.78% on PushT; it reaches 42.97% and 41.02% on RoundTable and Lamp, where the strongest competing baselines reach 15.23% and 25.91%.
Online refinement is cheap because the action response is learned directly rather than differentiated or searched over at deployment. GPC-RANK evaluates 100 action candidates, GPC-OPT runs 24 projected Adam updates, and GPC-QP spends 36.83 ms constructing predictions and Jacobians. A2WM needs one model evaluation and one quadratic program solve per step, averaging 3.80 ms of refinement latency, which is 6.78, 10.39 and 105.56 times faster than GPC-RANK, GPC-QP and GPC-OPT; its prediction and response construction takes 1.49 ms and QP processing 0.94 ms, close to GPC-QP's 1.11 ms.
The model places task-relevant geometry directly in the prediction space, making predicted task errors affine in action corrections. Assembly objectives depend on the relative pose between two parts, a nonlinear function of their individual poses, so affine raw-state predictions do not by themselves make the task cost quadratic. The model retains measured states, adds geometric quantities and learned nonlinear features, conditions on interaction history and the policy proposal, and outputs the nominal trajectory and response matrix for the whole horizon in one evaluation; training uses action-perturbed rollouts of the frozen policy, with 256 to 512 episodes collected per task and 250, 248, 256 and 466 remaining after filtering.
Perspective
The demonstrated scope is four simulated manipulation tasks: three FurnitureBench assembly tasks in Isaac Gym and planar PushT, using measured robot and object states, manually specified geometric objectives, and a fixed perturbed-rollout dataset of 256 to 512 episodes per task. The authors propose future work on refinement under state estimation and model uncertainty and evaluation on real robots.
The authors state that the current framework assumes measured robot and object states and uses manually specified geometric objectives, and that its local predictions depend on the coverage of the collected rollouts. Action-perturbation probes show A2WM captures a nearly linear response within the correction bounds but fails beyond them, and in a second PushT example the MLP better matches local curvature. The prediction-accuracy comparison is a task-averaged result, the multi-step MLP remains more accurate on Lamp and PushT, and model capacities differ.