# Human-Object Interaction

COSMI composes multi-object human-object interaction data from single-object captures by extracting contact-consistent clips, mirroring them to balance the hands, and admitting only semantically and physically plausible pairings via a language model and geometric checks, yielding a dataset of 222k sequences and 275 hours with up to five objects (nearly thirty times the largest multi-object capture); it then trains a text-driven diffusion transformer whose object slots share weights and whose objects are predicted relative to the body joints that drive them, outperforming HIMO and multi-object adaptations of MDM and PriorMDM in text alignment and contact accuracy on a benchmark with an unseen object and unseen combinations, with the largest margin on the unseen object.

COSMI composes multi-object human-object interaction data from single-object captures by extracting contact-consistent clips, mirroring them to balance the hands, and admitting only semantically and physically plausible pairings via a language model and geometric checks, yielding a dataset of 222k sequences and 275 hours with up to five objects (nearly thirty times the largest multi-object capture); it then trains a text-driven diffusion transformer whose object slots share weights and whose objects are predicted relative to the body joints that drive them, outperforming HIMO and multi-object adaptations of MDM and PriorMDM in text alignment and contact accuracy on a benchmark with an unseen object and unseen combinations, with the largest margin on the unseen object.

OTRetarget jointly retargets robot and multi-object motion by representing surface interactions through signed distances, closest surface points and relative directions, transferring these quantities across human, robot and object geometries with entropic optimal transport, and enforcing them in a per-frame constrained inverse kinematics that optimizes robot and object poses jointly; on OMOMO it reaches a robot-object interaction Jaccard of 87% and a depth error of 8.7 mm versus 28% and 29.3 mm for OmniRetarget, and transfers to a physical Unitree G1 for two-handed box pick-and-place onto a table.

OTRetarget jointly retargets robot and multi-object motion by representing surface interactions through signed distances, closest surface points and relative directions, transferring these quantities across human, robot and object geometries with entropic optimal transport, and enforcing them in a per-frame constrained inverse kinematics that optimizes robot and object poses jointly; on OMOMO it reaches a robot-object interaction Jaccard of 87% and a depth error of 8.7 mm versus 28% and 29.3 mm for OmniRetarget, and transfers to a physical Unitree G1 for two-handed box pick-and-place onto a table.