Robot actions stop lining up by index: FAFM maps mixed-frequency demonstrations into frequency coefficients and lifts real-arm success by 10 to 15 points
Synopsis
A team from Lingchu Intelligence with Beihang University, Peking University and the Chinese University of Hong Kong introduces FAFM, a frequency-aware flow-matching method that represents robot actions as frequency coefficients tied to real physical time instead of fixed-length discrete sequences, and explicitly constrains the first time derivative while generating actions by flow matching. Demonstrations recorded at 5 Hz, 10 Hz and 20 Hz can then be mapped onto one physical time scale and trained together without downsampling or interpolation. On a Franka arm, integrating FAFM into π₀.₅ raised success from 80% to 95% on grasp-and-place and from 80% to 90% on multi-obstacle avoidance.
Interpretation
Robot action generation no longer treats actions only as fixed-length discrete sequences; frequency coefficients describe a continuous trajectory in physical time, so demonstrations collected at different control rates can be trained together. Generative robot policies previously emitted an action chunk and implicitly assumed that position k corresponds to a comparable physical moment across datasets, yet control rates of different robots and pipelines can differ several-fold, and one second of motion recorded at 5 Hz, 10 Hz and 20 Hz yields completely different action counts; learning by index lets the same position stand for different motion stages, so correct demonstrations at different frequencies can supply conflicting supervision. For a demonstration of fixed physical duration, coefficients extracted by the discrete cosine transform (DCT-II) correspond to a fixed real time range rather than a sampling rate, so the same physical motion keeps consistent continuous-trajectory coefficients within discrete approximation error across sampling rates, turning conflicting action-position supervision into more consistent trajectory-shape supervision.
Action generation runs flow matching in coefficient space and reconstructs the action at any physical moment through cosine basis functions, while the training objective also constrains the first time derivative of the action. Conventional flow-matching action policies predict discrete action chunks directly and fit only the value at each action point, so errors at adjacent moments can accumulate into jitter that affects grasping, obstacle avoidance and deformable-object manipulation. A cosine expansion can be differentiated analytically, so the first-derivative constraint is optimized jointly with the flow-matching loss; in frequency terms this penalizes coefficient error more strongly at higher frequencies, linking it to a Sobolev-type constraint, and the change sits at the action representation and training-objective level without adding networks or learnable parameters, usable as a standalone generative control policy or plugged into vision-language-action models such as π₀ and π₀.₅.
On a real Franka arm, integrating FAFM into π₀.₅ raised success on both a grasp-and-place task with varying target positions and a multi-obstacle avoidance task above the original π₀.₅. The original π₀.₅ reached 80% success on both tasks. With FAFM, grasp-and-place reached 95%, a gain of 15 percentage points, and multi-obstacle avoidance reached 90%, a gain of 10 percentage points; the experiments attribute the improvement to smoother trajectories reducing missed targets and obstacle collisions during execution.
Perspective
The demonstrated scope covers up to 61% task success with 12 valid trajectory modes in multi-obstacle arm avoidance, 97%, 100%, 99% and 100% success on suture threading, grasp-lift contact, bimanual tissue manipulation and loop ligation in a robotic surgical simulation, and two real-arm tasks on a Franka platform with FAFM integrated into π₀.₅; the intended readers are researchers and engineers working on robot action generation and vision-language-action models.
Consistency of coefficients across frequencies holds within discrete approximation error, not exactly; the real-arm gains come from two tasks with FAFM integrated into π₀.₅ on a Franka platform, and behavior on other embodiments and tasks is not given in the stated scope.
