Jinglianwen releases binocular EGO+EMG hand-interaction dataset, adding per-finger force labels across 1,170 real tasks

Public hand-interaction data moves from mainly vision and hand pose to also carrying muscle signals and per-finger force estimates, giving forceful contact tasks such as pressing, plugging and precision assembly a force supervision signal. Mainstream first-person datasets mostly cover only vision and hand pose, lacking joint annotation of motion, force and task semantics, so robots can replicate hand trajectories but struggle to learn real contact-interaction logic. The dataset spans 28 domains and 1,170 real tasks, all first-person operations by real people in real environments, with EMG wristbands worn on both wrists during capture.

The capture pipeline places binocular first-person vision, 16-channel surface EMG and IMU on one timeline and outputs force, pose and semantic annotations together. Per-finger force is not measured directly but inferred by the Fusion2Force fusion model from EMG and other signals, five channels per hand and ten in total, in millinewtons, with a 1 Hz contact mask and validity flags. Official pipeline figures report multimodal synchronization deviation under 3 ms, joint-angle mean absolute error of 3.37 degrees, finger-joint distance error of 4.2 mm and force-magnitude error under 20%.

The delivery side includes binocular stereo calibration files and frame-level alignment, with data provided in LeRobot v3.0 format stored as Parquet and MP4. Binocular video is split from the same frame of a 3840×1080 stitched source with zero left-right offset, and each episode carries a SHA256 fingerprint of its source file for audit and traceability. It ships 35 binocular stereo calibration files with intrinsics, distortion coefficients and extrinsic rotation-translation matrices at a baseline of about 60 mm; spot checks of 11 video streams matched declared frame counts 100%.

The dataset targets research institutions, robotics companies and algorithm teams for vision-language-action model training, hand-perception algorithm evaluation, dexterous-hand and dual-arm imitation learning, teleoperation and data-collection assessment, and embodied-intelligence benchmark construction; all data are first-person operations by real people in real environments, covering 28 domains and 1,170 real tasks.

Per-finger force is inferred by the Fusion2Force fusion model with force-magnitude error under 20%, not measured directly, so usability should be judged together with the 1 Hz contact mask and validity flags. Synchronization deviation and joint-angle error are official pipeline figures, and the material does not state their statistical sample or evaluation conditions.