History Forcing teaches few-step video models to remember: a 1.3B student reaches 79.82 on 60-second VBench-Long
Synopsis
Few-step autoregressive video diffusion models gradually lose subject and scene information over long rollouts, and existing distillation targets do not constrain distant history separately. History Forcing builds a context residual from the difference between one autoregressive teacher's predictions with and without extra history, then weights it into the DMD or OPSD distillation target. A 1.3B student reaches a VBench quality score of 85.07 for 15-second generation and a VBench-Long aggregate of 79.82 for 60-second generation, and raises PlayWorld Evolution Quality from 2.35 to 2.70 when the camera leaves and returns.
Interpretation
In long video generation, subject appearance and scene structure drift as autoregressive rollouts accumulate; when the camera leaves and returns, recent frames no longer carry information about the hidden object, which the model may alter or lose. Earlier DMD distillation uses a bidirectional diffusion teacher for visual quality but does not explicitly include distant history; distilling directly from a history-conditioned autoregressive teacher instead couples history consistency with visual quality and inherits that teacher's quality limits. On 15-second VBench, adding history supervision raises the quality score from the DMD baseline's 84.44 to 84.82, with subject consistency, background consistency and dynamic degree all rising; fine-tuning the autoregressive teacher on 10-second clips lifts the quality score further to 85.07.
The method has one autoregressive teacher predict the same noisy student chunk twice, with inputs differing only in an extra span of history while the recent context stays fixed. The difference between the two predictions is the context residual, which carries only the history-induced effect and is scaled by a weight lambda and added to the bidirectional teacher's reference score to form a corrected distillation target. For long video, the extra history is the 12 latent frames immediately preceding a shared recent context of the latest 6 student-generated latent frames; for world modeling, the extra history comes from a fixed-camera teacher rollout while the student follows its prescribed camera path.
The corrected target plugs into DMD and into on-policy self-distillation OPSD without changing the existing fake-score model or DMD weighting. On 60-second VBench-Long, a 1.3B student with a 1.3B DMD teacher reaches an aggregate of 79.82, above Rolling Forcing's 79.22 and LongLive's 79.12, both using 14B teachers, and close to the 5B LongLive-2.0's 79.90. These 60-second results come from student training rollouts of at most 15 seconds; on the PlayWorld Out-of-sight subset, Evolution Quality rises from the baseline's 2.35 to 2.70, and on the MemoBench Real subset object consistency rises from 58.43 to 62.18.
Perspective
The demonstrated scope is long video generation with 1.3B teachers and students, and out-of-sight world modeling on a 5B DreamX-World backbone; the world-model training set is 200 Pexels initial scene images each paired with 4 camera trajectories that leave and return to the initial viewpoint, giving 800 samples. The intended readers are researchers and engineers working on autoregressive video diffusion distillation and world-model training.
History supervision remains limited by the underlying models' ability to capture temporal dynamics; in world modeling, fixed-camera teacher rollouts themselves show limited scene evolution, which weakens supervision of out-of-sight state changes. Some appearance and scene drift remains in the qualitative examples. The authors propose future work on larger teachers and students, longer-context training, and better simulation of physically plausible dynamics.