# Video Diffusion Model

Video diffusion models lose scene geometry when the camera moves or objects move, because cross-frame attention often spreads its response over several regions instead of locking onto the same scene element. GAD extracts correspondence targets from the attention of a frozen geometry foundation model, VGGT-, on clean videos, and routes supervision by noise timestep to the layer where scene structure is forming, training video attention with pairwise and trackwise objectives. On 15-second long-range generation with RealEstate10K and DL3DV-10K, adding GAD to Wan2.2 5B raises cross-frame matching hit rate by at least 30% relative on both datasets, with no extra inference modules or computation.

Video diffusion models lose scene geometry when the camera moves or objects move, because cross-frame attention often spreads its response over several regions instead of locking onto the same scene element. GAD extracts correspondence targets from the attention of a frozen geometry foundation model, VGGT-, on clean videos, and routes supervision by noise timestep to the layer where scene structure is forming, training video attention with pairwise and trackwise objectives. On 15-second long-range generation with RealEstate10K and DL3DV-10K, adding GAD to Wan2.2 5B raises cross-frame matching hit rate by at least 30% relative on both datasets, with no extra inference modules or computation.

Long-horizon video generation has traded between two kinds of memory: explicit geometric caches recall a place by location but grow with the scene, while recurrent memories stay constant-size yet let old observations fade with time. Smart turns a fixed-capacity slot memory into one addressed by camera pose, so writes and reads depend on where something was seen rather than how long ago, and a diffusion generator uses the memory's confidence to decide where to trust it. On long trajectories in Memory Maze and Minecraft it keeps high-fidelity recall far beyond its training horizon at constant memory and per-step cost.

Long-horizon video generation has traded between two kinds of memory: explicit geometric caches recall a place by location but grow with the scene, while recurrent memories stay constant-size yet let old observations fade with time. Smart turns a fixed-capacity slot memory into one addressed by camera pose, so writes and reads depend on where something was seen rather than how long ago, and a diffusion generator uses the memory's confidence to decide where to trust it. On long trajectories in Memory Maze and Minecraft it keeps high-fidelity recall far beyond its training horizon at constant memory and per-step cost.

In dynamic world modeling, the same observations must condition repeated generation at different times and viewpoints, and prior pipelines rendered the reconstructed scene to RGB and re-encoded it for every request. 4LC binds visual representations of static and dynamic content from several encoders to shared geometry and motion, forming a 4D latent cache that is read directly at independently specified times and viewpoints. Under identical generation settings, conditioning falls from 14.9% to 0.25% of end-to-end generation time, and a single 4LC-Video model covers camera control, temporal control, novel-view synthesis, scene revisits and long-term generation without task-specific fine-tuning.

In dynamic world modeling, the same observations must condition repeated generation at different times and viewpoints, and prior pipelines rendered the reconstructed scene to RGB and re-encoded it for every request. 4LC binds visual representations of static and dynamic content from several encoders to shared geometry and motion, forming a 4D latent cache that is read directly at independently specified times and viewpoints. Under identical generation settings, conditioning falls from 14.9% to 0.25% of end-to-end generation time, and a single 4LC-Video model covers camera control, temporal control, novel-view synthesis, scene revisits and long-term generation without task-specific fine-tuning.

The work identifies a "uniformity trap" in which language-style token-wise MoE routing, combined with uniform expert-usage regularization, scatters neighboring patches of the same object across different experts in video diffusion, causing routing fragmentation and structural distortion; it proposes SplitMoE, which bifurcates the expert pool into semantic and generic experts and uses prototype-guided routing with pull-push regularization so tokens cluster by semantic attributes, outperforming traditional load-balanced MoE on VBench-2.0 and T2V-CompBench under an equivalent activated-parameter budget, while revealing a coarse-to-fine denoising pattern in which semantic-expert weight peaks in the early high-noise stage around steps 10-15 and then shifts toward generic experts.

The work identifies a "uniformity trap" in which language-style token-wise MoE routing, combined with uniform expert-usage regularization, scatters neighboring patches of the same object across different experts in video diffusion, causing routing fragmentation and structural distortion; it proposes SplitMoE, which bifurcates the expert pool into semantic and generic experts and uses prototype-guided routing with pull-push regularization so tokens cluster by semantic attributes, outperforming traditional load-balanced MoE on VBench-2.0 and T2V-CompBench under an equivalent activated-parameter budget, while revealing a coarse-to-fine denoising pattern in which semantic-expert weight peaks in the early high-noise stage around steps 10-15 and then shifts toward generic experts.

The work proposes SVEET, a framework that trains a control branch only on a pretrained bidirectional video diffusion model and, through a temporally independent 2D spatial-attention control branch plus Orthogonal Decoupled Training (ODT) that bridges the bidirectional-to-autoregressive feature gap, enables high-quality streaming video editing without retraining or distilling the streaming backbone, reaching 15 FPS on a single H100 GPU and outperforming several baselines on style transfer, video inpainting, and depth-to-video generation.

The work proposes SVEET, a framework that trains a control branch only on a pretrained bidirectional video diffusion model and, through a temporally independent 2D spatial-attention control branch plus Orthogonal Decoupled Training (ODT) that bridges the bidirectional-to-autoregressive feature gap, enables high-quality streaming video editing without retraining or distilling the streaming backbone, reaching 15 FPS on a single H100 GPU and outperforming several baselines on style transfer, video inpainting, and depth-to-video generation.