Skip to main content
Back to timeline
arXivSource publication:

VA-TDM swaps four-step supervision for a velocity-field adversarial game: 45.53% human preference against the many-step teacher on 33B MiniMax-H3

Synopsis

Few-step diffusion distillation has long relied on estimating the student score, yet at intermediate steps that score is hard to track and the estimated distribution can differ from the one used to update the student. VA-TDM instead supervises velocity fields built from the student's own sampling process, uses a learned critic to compare student and teacher fields and turns their disagreement into generator feedback, unifying score distillation, Fisher matching and moment matching. On four-step SD3.5-Medium it raises GenEval from TDM's 0.61 to 0.67; on 33B MiniMax-H3 at the same 4-NFE budget it wins 61.33% of human comparisons against Fast-H3 and reaches a 45.53% preference rate against the 100-NFE teacher, versus 24.67% for Fast-H3.

AI-generated editorial illustration: VA-TDM: A Unified View for Improving Few-Step Diffusion Models by Velocity Adversarial Distillation

Interpretation

Few-step diffusion models can now improve prompt following, image quality and diversity together within four sampling steps, without relying on a hard-to-estimate student score at intermediate steps. Earlier trajectory distribution matching methods needed an online student-score estimate at intermediate transitions, and the commonly used clean-target denoising estimates a different distribution from the one the generator update requires. On four-step SD3.5-Medium, VA-TDM raises GenEval from the strongest baseline TDM's 0.61 to 0.67, with OCR at 0.59 and LPIPS diversity at 0.58, above TDM's 0.57.

The supervision signal becomes a velocity field constructed from the student's own sampling chain, and the discrepancy between student and teacher velocity fields feeds directly into generator learning. Velocity targets can be defined through paths with coupled endpoints, so they inherit the dependence between the clean prediction and the residual inside a student transition instead of estimating the score of the perturbed output around that dependence. On four-step FLUX.2-Klein-Base 4B, VA-TDM raises GenEval from the officially released four-step model's 0.78 to 0.80, OCR similarity from 0.64 to 0.69, and LPIPS from 0.47 to 0.56.

One adversarial velocity framework combines two velocity constructions with two gradient conventions, mapping onto existing score distillation, Fisher matching and moment matching updates and yielding new training objectives. The earlier MMD update was a heuristic approximate gradient, whereas this framework gives the exact gradient of its intended objective under the fixed-outer-measure convention and explains the heuristic update as a generator update in the adversarial velocity game. In the four-step SD3.5-Medium ablation, A-form alone gives GenEval 0.64 and OCR 0.55, B-form alone gives GenEval 0.61 and OCR 0.57, and combining the two games yields higher OCR and ImageReward than either endpoint.

Each intermediate transition along the student trajectory receives its own supervision, and the field network is told which transition is being supervised so that different steps get different signals. Earlier few-step methods typically covered all steps with a shared auxiliary score model, even though student transitions with different step indices generally induce distinct noisy distributions. On four-step SD3.5-Medium, removing transition-aware field estimation gives GenEval 0.58, OCR 0.53 and LPIPS 0.56, below the full configuration's 0.67, 0.59 and 0.58.

Perspective

The demonstrated scope is four-step generation: text-to-image evaluation on SD3.5-Medium and FLUX.2-Klein-Base 4B, and text-to-video and audio generation on 33B MiniMax-H3, with the human preference study using 10 randomly chosen prompts that were each manually checked for content. The intended readers are researchers and engineers working on diffusion and flow-model distillation; the source proposes no next experiment beyond this scope.

The source reports results for the four-step setting and does not give performance at other step counts; on FLUX.2 4B, LPIPS rises from 0.47 to 0.56 while PickScore is 0.8694, below the official four-step model's 0.8746; the human preference comparison is an anonymized pairwise study over 10 randomly chosen prompts.