Dynamic world models stop re-rendering every request: a 4D latent cache cuts conditioning from 14.9% to 0.25% of generation time
Synopsis
In dynamic world modeling, the same observations must condition repeated generation at different times and viewpoints, and prior pipelines rendered the reconstructed scene to RGB and re-encoded it for every request. 4LC binds visual representations of static and dynamic content from several encoders to shared geometry and motion, forming a 4D latent cache that is read directly at independently specified times and viewpoints. Under identical generation settings, conditioning falls from 14.9% to 0.25% of end-to-end generation time, and a single 4LC-Video model covers camera control, temporal control, novel-view synthesis, scene revisits and long-term generation without task-specific fine-tuning.
Interpretation
Conditioning for video world models moves from re-rendering the reconstructed scene per request to reading a cache encoded once. Geometric conditioning pipelines keep scene information in explicit 3D reconstructions or spatial memories, yet each new generation request still renders RGB views and re-encodes them, and this repeated processing can consume a substantial share of total generation time. With the same Wan2.1 backbone and VAE encoder and 81-frame requests, NeoVerse spends 14.9% of its 18.47 s end-to-end time on condition acquisition, against 0.25% of 16.19 s for 4LC-Video.
The cache holds not one feature type but representations from several encoders, all bound to the same geometry and motion. Existing latent-reuse approaches store features in a specific generator's latent space and exclude dynamic content from the cache. The default pairs FLUX.2 VAE with V-JEPA 2.1; adding DINOv2 and VGGT raises readout latency from 162.03 to 255.20 ms and peak GPU memory from 0.763 to 1.605 GiB, still below rendering and encoding one representation.
A readout interface propagates cache units to the requested time, projects them through the target camera, and merges readouts across entries using coverage masks. Entries are organized by insertion order, and the source entry stays unchanged and available for later requests. On camera control, 4LC-Video reports rotation and translation errors of 0.037 and 0.089 versus 0.059 and 0.120 for NeoVerse; under temporal control the figures are 0.041 and 0.060 versus 0.057 and 0.080.
One checkpoint serves five generation tasks by changing source observations, spatiotemporal requests and historical reuse. Baselines tend to trade camera following against scene consistency: NeoVerse tracks cameras with lower consistency, while GEN3C and SpaceTimePilot stay consistent but close to static views with several-fold camera errors. On RealEstate10K novel-view synthesis it reaches LPIPS 0.188 and PSNR 18.94, within 0.1 dB of LiveWorld's 19.04; on WorldScore scene revisits it leads LiveWorld by 1.84 dB in PSNR; on MBench-A long-term generation it scores 85.56 Environment and 89.06 Causal, the highest among compared methods.
Perspective
The report covers the conditioning stage of dynamic world modeling: the cache stores visual representations of source observations, readout propagates geometry to the requested time and projects it through the target camera, and composition merges entries through coverage masks. Experiments use the Wan2.1-T2V-14B backbone with a VACE control branch and the NeoVerse reconstructor, train on monocular clips from SpatialVID, RealCam-Vid and OpenVid, and generate 81-frame segments at inference. Efficiency figures average 50 runs on an NVIDIA A100, with scene construction excluded from timing for all frameworks. It suits researchers and engineers working on video world models, conditioning control and inference efficiency.
4LC relies on a feed-forward 4D reconstruction model to estimate depth, camera parameters and motion, and its estimation errors can affect readout quality despite training augmentation. Long-term generation reuses only the static units of historical entries, with dynamic reuse across segments left to future work. Direct readout and re-encoding do not agree equally across features: semantic and geometric features keep stable relative structure, while VAE features show lower relative similarity, with medians of 0.85 for FLUX.2 VAE and 0.71 for Wan2.1 VAE. Performance stays stable for motion thresholds from 0.05 to 0.4 pixels per frame, but disabling motion filtering degrades it noticeably.