Stochastic diffusion sampling drops the higher-order detour: F-DDPM evaluates at a cleaner lookahead point and lowers FID at matched NFE
Synopsis
Stochastic diffusion samplers have lagged deterministic ones, and raising approximation order does not necessarily improve their strong convergence order. F-DDPM evaluates the data-prediction model at a cleaner lookahead point built from the previous prediction and the previously injected noise, still with one model evaluation per iteration and a first-order strong convergence guarantee. On CIFAR-10, ImageNet, FFHQ and LSUN it reaches lower FID than the evaluated stochastic baselines at matched NFE budgets, and with 1.5 times the NFE it approaches or beats deterministic baselines in many settings.
Interpretation
Stochastic diffusion sampling need not speed up by raising approximation order: moving where the model is evaluated, from the current state to a lookahead state, reduces sampling error while keeping one model evaluation per step. Stochastic samplers previously needed substantially more neural function evaluations than deterministic samplers to reach comparable sample quality, and the benefit of higher-order discretization for stochastic sampling was unclear. Matching upper and worst-case lower bounds for SDE-DPMSolver++1 and SDE-DPMSolver++2 show both remain first-order strongly convergent even with exact score functions, so the higher-order approximation in SDE-DPMSolver++2 does not raise the convergence order.
F-DDPM keeps the DDPM update form and changes only where the data-prediction model is evaluated: at a cleaner time than the current step's endpoint, to account for the noise injected afterward. That lookahead point comes from a single DDIM step whose noise estimate reuses the previous model prediction and the noise injected at the previous step, so no extra model evaluation is added. On unconditional CIFAR-10 with the EDM model at 10 model evaluations, F-DDPM reaches an FID of 3.34, against 32.64 for SDE-DPMSolver++1 and 7.07 for SDE-DPMSolver++2; F-DDPM has the lowest FID among the three stochastic methods in every tested setting.
Perspective
The demonstrated scope is image generation on CIFAR-10, ImageNet-64 and ImageNet-512, LSUN bedrooms and FFHQ, covering pixel-space EDM/EDM2 models and latent diffusion models, with quality measured by FID over 50K generated samples; reference statistics for LSUN and FFHQ were computed by the authors, so those FID scores may differ from values reported elsewhere.
The convergence guarantees characterize discretization error against the continuous diffusion dynamics and do not explain why changing evaluation placement improves few-step sample quality; behavior under classifier guidance and classifier-free guidance, and interaction with measurement-consistency constraints in inverse problems such as deblurring and super-resolution, remain open.