Skip to main content
Back to timeline
arXivSource publication:

Interactive video world models treat old frames as false: re-dating by valid time lifts the current-state share from 0.36 to 0.62 after an undo

Synopsis

Interactive video world models draw frames from a memory bank stamped with capture times, and a frozen generator reads the newest timestamp as truth, so after an agent undoes an action the only view of the present is an older frame judged outdated. ReDate keeps every frame and re-dates retrieved tokens by valid time read from the action log, so a reverted region reads as one that has just reached a new state, with no training. On FAR's 352 reverted steps the current-state share rises from 0.36 to 0.62 and the outdated share falls from 0.27 to 0.09; on HY-WorldPlay it rises from 0.003 to 0.19.

Source-provided article image: Old Is Not False: Valid-Time Memory for Interactive Video World Models
arXiv

Interpretation

Interactive video world models show an outdated scene as current after an agent undoes an action, because the memory bank orders frames by capture time and the generator reads the newest timestamp as truth. Earlier memories for changing worlds treat a view contradicted by a newer one as false for good, deleting, masking or decaying it, or filtering out whole frames that disagree with a state table. With frames, weights and noise fixed and only timestamps rewritten, FAR on 1,478 evaluated steps of subset A draws the current state on 0.51 and the outdated state on 0.12 under kept timestamps, and 0.07 and 0.58 when reversed; HY-WorldPlay with the outdated view placed later gives 71 outdated and 1 current render, and 70 current and 5 outdated after exchanging the two rotary positions.

ReDate changes no generator weight and only changes the timestamps the generator sees, so a reverted region reads as one that has just reached a new state. It borrows the temporal-database split between when a record is stored and when its content holds, deriving each region's valid time from the agent's action log. Inside FAR's frozen diffusion transformer, the current-state share on 352 reverted steps rises from 0.36 to 0.62 and the outdated share falls from 0.27 to 0.09, higher on 54 of 141 clips and lower on 2; on 590 reverted steps of 197 held-out clips it is 0.58 against 0.34.

Moving outdated content into the past instead of deleting it keeps frames whole and gives higher image quality than leaving a hole. Deletion rules reach only 0.02 to 0.14 on reverted steps because they remove the only frame at the target pose that records the current state. On the 352 reverted steps, hiding outdated tokens costs over 4 dB of PSNR against moving them and sends over a third of the renders below the quality floor; per-token timestamps cut the outdated share from 0.15 per frame to 0.09.

The same timestamp rule works on a second generator and needs no ground-truth frame or change mask. HY-WorldPlay orders history through temporal rotary positions and under capture time almost never draws the current state. On the 352 reverted steps at seed 1 the current share rises from 0.003 to 0.188 and the outdated share falls from 0.281 to 0.091; seed 0 gives 0.208 and held-out clips give 0.181.

Perspective

The demonstrated setting is AI2-THOR room rollouts inside FAR's frozen diffusion transformer and HY-WorldPlay's 480p image-to-video configuration, plus a two-way reading on 31 undo events in real Charades homes. The method targets retrieval memories that date their frames and leaves retrieval, compression and generator weights unchanged.

The remaining errors concentrate where the target pose never stored the current state: 120 of ReDate's 134 misses on reverted steps fall in the 133 such steps, where every memory stays at or below 0.10. A relay through FAR's rollout and VGGT depth reprojection reach only 0.13 and 0.10, and the authors name rendering a valid-time view at the target pose and logging what other actors change as the next problems.