World models need not rerun per candidate: one extraction of local shape scores sixteen actions faster than the source generator

An action-conditioned world model spends a full forward pass on every candidate action, and a planner comparing many candidates cannot afford that cost. IRASim spends 25 diffusion steps on each candidate action chunk and WorldGym 40, so comparing a hundred candidates means a hundred model rollouts. On BridgeData V2, scoring sixteen candidates from a single extraction is over an order of magnitude faster than with either source generator, and the gap widens with the number of candidates.

Fixing the current state and varying only the action, the reachable latent image states lie on a low-dimensional surface that is a property of the data distribution rather than of any one model. Previously that surface could only be approached by repeatedly sampling a generator, one rollout per candidate. On Kubric, the per-unit action fields recovered by the probe match the differentiable generator's autodiff Jacobian in cosine similarity, showing the measurement captures the generator's response to the action.

The method first measures how each image region moves per unit of each action axis, then has a geometry encoder emit that action Jacobian in one forward pass, and finally adds second- and third-order terms with a correction head. Only the tangent comes from the generator; the higher-order terms and a correction to the tangent itself are fit to recorded transitions. On BridgeData V2, Action Jet with its fine-tuned decoder beats IRASim, the generator that supplied its tangent, on PSNR and LPIPS at gap 1 and is level at gap 2, at over an order of magnitude less compute.

All geometry is measured in one fixed autoencoder's latent space, so strictly the recovered object is the composite data-and-encoder manifold, and the experiments vary the generator rather than the latent. Each chart holds only at its probe scale, the advantage over the source generator holds at short horizons, and all evaluations are open-loop.

Disocclusion has no polynomial form, so even with noise injection some blur concentrates at visibility boundaries. Each chart holds only at its probe scale, and the advantage over the source generator holds at short horizons. On BridgeData V2 the chart learned from WorldGym fails to anchor on real transitions and is kept as a negative control.