Jinhyeok Choi

Interactive world models have been limited by memory that grows with stream length: temporal context runs out of GPU memory, and explicit 3D memories slow down as redundant primitives accumulate frame by frame. L3G-World replaces that with a compact latent 3D Gaussian memory that spawns Gaussians only in newly observed regions, refines existing ones with residual updates, and stores a VAE latent feature in each Gaussian so the memory renders directly into the diffusion transformer's latent space without re-encoding. On 481-frame loop-closure trajectories from DL3DV and RealEstate10K, full L3G-World reaches PSNR of 16.00 and 18.36, and with few-step distillation it generates at 9.02 FPS on a single H100.

Interactive world models have been limited by memory that grows with stream length: temporal context runs out of GPU memory, and explicit 3D memories slow down as redundant primitives accumulate frame by frame. L3G-World replaces that with a compact latent 3D Gaussian memory that spawns Gaussians only in newly observed regions, refines existing ones with residual updates, and stores a VAE latent feature in each Gaussian so the memory renders directly into the diffusion transformer's latent space without re-encoding. On 481-frame loop-closure trajectories from DL3DV and RealEstate10K, full L3G-World reaches PSNR of 16.00 and 18.36, and with few-step distillation it generates at 9.02 FPS on a single H100.