Joint audio-video generation stops letting audio drag: OmniOPD steadies distillation by giving the audio teacher a cleaner video condition

OmniOPD distills two specialist teachers, one trained with video rewards and one with audio rewards, into a single joint audio-video student, so audio and video capabilities enter one model together. Prior practice optimized rewards for several modalities jointly, but audio and video objectives conflict and their gradients are imbalanced, producing cross-modal interference; optimizing with audio rewards alone was unstable and also degraded video quality. On JavisBench, OmniOPD reaches video quality VQ 3.347 and audio quality AQ 5.449, against 2.714 and 5.252 for the LTX-2.3 baseline and 2.713 and 5.577 for the joint-reward GDPO variant.

Audio supervision is sparse because audio carries far fewer tokens than video, and audio learning depends heavily on visual context, so a cleaner visual condition gives the audio teacher more informative targets. Simply increasing the audio loss weight does not fix the imbalance; it amplifies unstable audio learning signals and degrades both modalities. On JavisBench, an asymmetric-noise shift of s=1.5 gives VQ 3.332 and AQ 5.376, and s=2.0 gives VQ 3.354 and AQ 5.353, both above the matched-noise 3.174 and 5.342.

OmniOPD has three parts: asymmetric noise that gives the audio teacher a cleaner video, grouped sampling of several trajectories per prompt to widen audio supervision coverage, and timestep-wise weighting by teacher-student transition divergence. The three parts target insufficiently informative audio targets, insufficient audio supervision coverage, and unreliable supervision signals, respectively. On JavisBench, grouped sampling with G=3 gives VQ 3.315 and AQ 5.393 versus 3.174 and 5.342 at G=1; removing importance weighting gives VQ 3.174 and AQ 5.342, while a soft ratio of p=0.75 gives 3.322 and 5.381.

The demonstrated setting is on-policy distillation on the LTX-2 (22B) dual-stream joint audio-video DiT, with teachers trained separately by DiffusionNFT with LoRA on video rewards and audio rewards, evaluated on JavisBench and VBench.

Teachers supervise on states from the student's own rollouts, which may differ from the input distributions seen during teacher training; when student states deviate substantially from the teachers' supported distributions, teacher predictions may become unreliable or misleading and the distillation supervision ineffective.