TERRA reconstructs terrain from kinematics alone, letting a muscle-actuated body complete stairs, ramps and seats

TERRA recovers task-relevant support geometry from motion data that provides body kinematics but no scene model, and distinguishes terrain families such as ramps versus stairs. Prior work either relies on visual observations or assumes terrain or object geometry is already available, as with OmniRetarget's interaction mesh, while SceneBot merges constant-height terrain patches; TERRA instead performs structured inference over explicit terrain priors (independent support boxes, inclined ramps, staircases, seat supports) and separates continuous ramps from discrete steps using ankle-toe orientation and swing-foot clearance profiles. On datasets with known terrain labels it reports 99.9% terrain-family accuracy with sub-degree ramp grade and sub-centimeter stair height errors; removing the foot-orientation and swing-clearance cues drops family accuracy to 73.5%; it remains robust to threshold perturbations and 2/5/10 mm landmark noise, while removing 10%-30% of support intervals causes gradual, terrain-dependent degradation that is most pronounced for stools.

Adding anatomical, contact, clearance and collision residual terms to terrain-aware retargeting substantially reduces physical violations in the reference motions. TERRA augments OmniRetarget's interaction-mesh objective with target-specific anatomical and terrain-interaction terms solved by sequential quadratic programming; relative to the no-residual version, contact-timing F1 rises from 88.49% to 93.80% and terrain penetration falls from 33.45% to 0.16%. Across 6498 motions it reports retargeting coverage, contact-timing F1, tendon continuity, self-collision, penetration, skating and floating, compared against OmniRetarget, GMR, MM-MoCap-Body and TERRA+Voronoi, with TERRA best on all five physical violation metrics and contact-timing F1.

Training a single muscle-actuated policy on terrain-paired references yields the highest completion rate on held-out motions. Musculoskeletal imitation learning had remained largely confined to flat ground; TERRA brings three-dimensional terrain navigation into the behavioral repertoire of a single reference-conditioned policy trained on 9.4 hours of motion-terrain pairs drawn from five datasets. The 8068 clips are split deterministically by recorded identity with a roughly 15% test fraction, three policies per method are trained with random seeds, and each clip is evaluated with five stochastic runs; TERRA has the highest success rate on flat ground, ramps, boxes and seats and the lowest MPJPE on flat ground, ramps and boxes, while TERRA+Voronoi performs best on stairs and attains the lowest seat MPJPE.

Terrain-paired biomechanics datasets let signals produced by the learned policy be compared directly with measured human profiles for the same locomotion condition. This offers a route to viewing simulated muscle activity alongside measured EMG and vertical ground-reaction forces rather than reporting tracking error alone. On held-out recordings from Gait120, Darmstadt and Vielemeyer, for level walking, ramp descent, stair descent and sit/stand, ten stochastic rollouts are used to display vertical GRF and three representative muscles as normalized waveforms, which the authors describe as showing both similarities and visible mismatches in timing and shape.

The pipeline targets motion-capture data that provides body kinematics but no scene model, and applies to four static terrain priors: ramps, staircases, independent supports and seats, with inputs in AMASS-compatible SMPL-H as well as C3D, MAT and TRC formats. It lets researchers extend large flat-ground motion libraries into terrain-paired muscle-actuated behavior libraries and compare retargeting and policy-training choices on the same references; for biomechanics researchers it offers an entry point for viewing policy-generated EMG and vertical ground-reaction forces alongside measured waveforms. The authors state that code and data will be made publicly available, and list dynamic primitive geometry, online retargeting and moving surfaces as future work.

Terrain reconstruction is evaluated on datasets that carry terrain labels or reference meshes; on large-scale data with no scene information such as AMASS, only contact and non-collision consistency criteria can be used indirectly, and how those relate to true geometric error remains to be observed. Policy success rates differ noticeably across terrain families, with boxes clearly below ramps, indicating that supported terrain families are not equally difficult. The physiological comparison is currently a qualitative display of normalized waveforms, and the authors frame it as a tool that may serve future study of physiological processes rather than an established correspondence. In addition, this reading covers the full paper text, so details in figures and the supplementary video cannot be checked here, and judgments about specific curve shapes or hyperparameter values should defer to the original figures.