Video attention often confuses different scene elements: GAD distills geometry-model attention, lifting Wan2.2 5B cross-frame matching by at least 30%
Synopsis
Video diffusion models lose scene geometry when the camera moves or objects move, because cross-frame attention often spreads its response over several regions instead of locking onto the same scene element. GAD extracts correspondence targets from the attention of a frozen geometry foundation model, VGGT-, on clean videos, and routes supervision by noise timestep to the layer where scene structure is forming, training video attention with pairwise and trackwise objectives. On 15-second long-range generation with RealEstate10K and DL3DV-10K, adding GAD to Wan2.2 5B raises cross-frame matching hit rate by at least 30% relative on both datasets, with no extra inference modules or computation.
Interpretation
Video diffusion models produce temporally coherent frames yet struggle to preserve scene geometry and object structure through camera motion and object movement; cross-frame attention often disperses its response across several regions with similar appearance or semantics, even after scene structure is visible. Camera-aware positional encoding, explicit geometric conditioning, history-conditioned generation and geometry-aware feature alignment improve information retention and exchange across frames, but do not ensure that attention links observations of the same scene element. Visualizing video attention across noise levels and network depth shows responses can remain spread over competing regions after structure emerges, while probing VGGT- finds selected layers and heads keep attention concentrated on the same railing across viewpoint changes and follow a moving person's hat.
GAD distills cross-frame correspondences from a geometry model into video attention during training, so attention points to geometrically corresponding regions while scene structure is forming. Correspondence targets come from the attention of a frozen VGGT- teacher, weighted by the total attention mass a source token assigns to the target frame as a reliability cue, with Gaussian smoothing around the target to tolerate small localization errors. The supervised layer is routed by noise timestep, using layer 5 at low noise, layer 15 at medium noise and layer 25 at high noise; pairwise supervision discriminates spatial matches within frame pairs, trackwise supervision matches one source to multiple target frames for temporal coverage, and both are optimized jointly with the flow-matching loss.
Across three video backbones, GAD improves long-range scene consistency and dynamic temporal consistency while keeping the original inference procedure. Compared with Wan baselines fine-tuned on the same training data with only the flow-matching objective, GAD improves cross-view matching and camera accuracy on RealEstate10K and DL3DV-10K, with better FID, FVD and CLIP as well. On Wan2.2 5B, matching hit rate rises by at least 30% relative on both long-range benchmarks; on Wan2.1 14B, GAD gives the best results across all seven metrics on both datasets among evaluated methods, with unchanged peak GPU memory and runtime differences consistent with run-to-run variability.
Perspective
Evaluation is limited to camera-controlled video generation: 15-second long-range scenes on RealEstate10K and DL3DV-10K and 5-second dynamic scenes on OmniWorld-Game, applied to three backbones, Wan2.1 1.3B, Wan2.2 5B and Wan2.1 14B. The training corpus is 35K clips from the three datasets, with scene-disjoint training and test splits.
Correspondence target quality depends on the VGGT- teacher's matching ability and spatial resolution, and may be less accurate for ambiguous appearance, occlusion and moving objects; the current layer routing depends only on noise timestep and stays fixed throughout training, while the layers that benefit most may change as the model learns. The method is validated on Wan's bidirectional video models; autoregressive generation has not been evaluated.