Kwanyoung Kim

Reward guidance steers a frozen diffusion or flow model at sampling time by following the gradient of a value function, but both common estimates of that gradient, differentiating a one-step denoised estimate and a learned value model, are least accurate at high noise, where the coarse structure of a sample is decided. VaLoP evaluates the value at a less noisy lookahead state one sampler step ahead and pulls its gradient back with one vector-Jacobian product; across SD3.5-Medium, SD3.5-Large and FLUX.1-dev with three rewards, VaLoP and its amortized variant VaLoP-Fast raise the target reward above StitchVM and DPS, and VaLoP-Fast attains the highest success rates on LIBERO and RoboCasa.

Reward guidance steers a frozen diffusion or flow model at sampling time by following the gradient of a value function, but both common estimates of that gradient, differentiating a one-step denoised estimate and a learned value model, are least accurate at high noise, where the coarse structure of a sample is decided. VaLoP evaluates the value at a less noisy lookahead state one sampler step ahead and pulls its gradient back with one vector-Jacobian product; across SD3.5-Medium, SD3.5-Large and FLUX.1-dev with three rewards, VaLoP and its amortized variant VaLoP-Fast raise the target reward above StitchVM and DPS, and VaLoP-Fast attains the highest success rates on LIBERO and RoboCasa.