Humanoid loco-manipulation drops full-body motion references: FORGE lifts boxes from wrist poses and coarse commands, hitting 90% success at 2 kg
Synopsis
Humanoid whole-body manipulation usually needs full-body human motion trajectories or wrist force sensors. FORGE splits commands into four granularities, velocity navigation, planar footholds, wrist poses and an optional force command, trains without full-body human motion references, and at deployment consumes only pelvis-relative bilateral wrist poses and coarse commands. A momentum observer plus a learned correction estimates wrist-equivalent force from joint states and reconstructed PD torques, with no wrist force/torque sensor. In a squat-and-lift task across four payloads, one diffusion policy driving FORGE succeeded in all 20 trials empty and at 0.5 kg, and in 18 of 20 at 1 kg and 2 kg.
Interpretation
A humanoid whole-body controller can now take velocity navigation, planar footholds, bilateral wrist poses and an optional wrist-force command through one policy, activated alone or together as the task demands. Earlier whole-body motion-tracking policies relied on dense full-body motion references and did not directly expose command interfaces for foot placement, wrist targets and sustained contact, while modular stacks that separate locomotion from the upper body can limit task-dependent whole-body coordination. Over 20 hardware trials each on the same robot and target sequences, FORGE showed 22 ms control latency and foot and wrist mean position errors of 5.42 cm and 4.15 cm; TWIST2 showed 197 ms, 7.87 cm and 7.76 cm, SONIC 252 ms, 7.35 cm and 6.42 cm, and HOMIE, which has no wrist-target or foothold interface, was recorded as not applicable for tracking.
During sustained contact, wrist-equivalent force is estimated from joint states and reconstructed low-level PD torques rather than a wrist force/torque sensor, and that estimate adjusts the wrist position target. An analytical momentum observer alone or a learned estimator alone both carry error, because external contact, friction, actuator behavior and whole-body coupling are entangled in proprioception. On G1 hardware, a handheld force gauge applied staged loads with 10 repeats per condition; at held-out vertical loads of 35, 40 and 50 N, the hybrid estimate's RMSEs were 8.28, 9.66 and 12.35 N, with lower planar error than learned-only estimation.
A frozen lower-body locomotion prior plus asymmetric residual actions makes mixed locomotion, pose and contact objectives trainable together. An end-to-end 29-DoF policy trained without a locomotion prior failed to learn stable behavior, and removing residual action-standard-deviation clipping caused late-stage reward collapse. In simulation, the masked and unmasked variants were compared over 20 rollouts each, capped at 20 s; without masking, early termination was more frequent and linear-velocity, foot and wrist errors were higher.
FORGE can serve as the whole-body executor for a visuomotor policy, turning multi-granularity command chunks from a diffusion policy into joint actions. Such whole-body controllers had not been tested as executors inside a cross-payload visuomotor pipeline. One cross-payload diffusion policy was trained on 80 demonstrations from each of four payload conditions, 320 in total; over 20 hardware trials per payload on the squat-and-lift-box task with empty, 0.5 kg, 1 kg and 2 kg boxes, success rates were 100%, 100%, 90% and 90%.
Perspective
The demonstrated scope is the evaluated humanoid platform: hardware covers walking, squat-and-lift and planar foot placement, plus recovery under a 30 N wrist push and a 50 cm body drag; force estimation is evaluated only in quasi-static contact, and the downstream task only covers squat-and-lift with 0 to 2 kg boxes.
The wrist-equivalent force estimate covers only the low-frequency, quasi-static translational component, does not identify unmodeled link contacts or whole-body coupling, and is not full-wrench sensing or dynamic impedance control; estimation accuracy degrades at higher vertical loads, with an RMSE of 12.35 N at 50 N.