Hasaan Ahmad

When a camera moves or a controller is replaced, the same command moves the robot in a different direction or by a different magnitude, while a frozen world model still plans under its training-time action convention, so every plan is expressed in the wrong action frame. Library of Hands (LoH) trains nothing and leaves the world model frozen: it uses the frozen predictor to infer the applied action from two successive camera images, estimates the map from commands to applied actions, and stores each identified map in a library for reuse. In closed-loop tests where hands change and come back, LoH raises a frozen planner's success from 28.6% to 62.3%, and loses only 0.6 percentage points when nothing changes.

When a camera moves or a controller is replaced, the same command moves the robot in a different direction or by a different magnitude, while a frozen world model still plans under its training-time action convention, so every plan is expressed in the wrong action frame. Library of Hands (LoH) trains nothing and leaves the world model frozen: it uses the frozen predictor to infer the applied action from two successive camera images, estimates the map from commands to applied actions, and stores each identified map in a library for reuse. In closed-loop tests where hands change and come back, LoH raises a frozen planner's success from 28.6% to 62.3%, and loses only 0.6 percentage points when nothing changes.