RoboDawn lets a frozen VLM control robots zero-shot through discrete commands, with one demonstration lifting RoboTwin 2.0 C2R success from 53.2% to 73.6%
The work introduces RoboDawn, which exposes robotic control to a frozen VLM as a compact set of discrete translation, rotation, and gripper commands in a closed observe-reason-act-reobserve loop, paired with an in-context learning scheme; it reaches 53.2% zero-shot and 73.6% one-shot success on RoboTwin 2.0 C2R, improves RoboDojo from 35.67% to 47.17%, and transfers zero-shot to a Franka robot for block-in-basket and block stacking.