Few-step text-to-image gains a feature-moment constraint: Z-Image-Turbo hits 94.25% on Chinese LongText-Bench at two evaluations
Synopsis
Few-step text-to-image generation must render prompt details with very few sampling steps, and distribution matching distillation (DMD) relies on learned score differences whose statistics need not align. Intrinsic Distribution Matching keeps the full DMD correction and adds a moment constraint on the means and per-coordinate standard deviations of the generator's exponential-moving-average features, with no external feature encoder. On Z-Image-Turbo 6B at two network evaluations it reaches 94.25% and 90.20% on Chinese and English LongText-Bench, the highest scores among the compared methods at that budget.
Interpretation
Few-step text-to-image training now carries a statistical constraint drawn from the generator itself: besides the existing score-difference supervision, training explicitly aligns the means and per-coordinate standard deviations of generated and reference samples in the generator's feature space. Earlier internal DMD guided the generated distribution only through learned score differences, with no separate constraint on whether statistics aligned; approaches using external encoders or teacher features needed an extra network to supply features. Under the shared training configuration, the control without the moment loss reaches 77.46 GenEval on full-parameter Z-Image at two network evaluations, versus 77.81 with it; LongText-CN rises from 93.87 to 94.25 and LongText-EN from 89.79 to 90.20, with gains on all five aggregate metrics.
The features used by the moment constraint come from the generator's own exponential-moving-average copy, so score estimation and feature extraction share one model. The compared alternatives supply features from a DINOv2 external encoder, from teacher features, or from live self-features. From the same control without the moment loss, DINOv2 raises GenEval to 77.75 and generator EMA features reach 77.81, with LongText-CN at 94.13 and 94.25 respectively, so moment supervision close to the external encoder is available without one.
Training and evaluation cover Z-Image-Turbo 6B and Qwen-Image 20B under both full-parameter tuning and low-rank adaptation. The earlier internal DMD route replaced a separate fake-score network with velocity-branch differences but added no cross-sample statistical objective on generator features. At two network evaluations, full-parameter IDM reaches 0.78 GenEval and 6.71 FID on Z-Image and 0.91 GenEval and 6.34 FID on Qwen-Image; under the same setting APEX reports 0.90 GenEval and 6.44 FID, with latencies of 0.44 and 0.45 seconds.
Perspective
Evaluation covers the Z-Image-Turbo 6B and Qwen-Image 20B backbones with full-parameter tuning and LoRA at ranks 32 and 64, reporting results at one and two network evaluations; long-text ability is measured by LongText-CN and LongText-EN, and general quality and prompt following by GenEval, FID, CLIP, DPG-Bench, WISE and OneIG.
The moment loss matches only pooled feature means and per-coordinate standard deviations, not the full feature distribution, and it neither estimates the sample-space Gaussian component nor the residual component; queue entries come from earlier model and EMA states, so the moments approximate current population statistics. Ablations also show that removing the DMD loss costs 0.91 GenEval points versus 0.35 for removing moment matching, so the score correction remains central.