Skip to main content
Search

# World Models

16 related articles

Subscribe to updates
  1. arXiv

    Duo v1 turns the world model into a two-way conversation: 17-frame chunks, it watches while it speaks, and it can be interrupted

    Duo v1 is a full-duplex world model that generates the assistant's audio and video in real time while continuously taking in the user's audiovisual input. Earlier world models largely kept intelligence outside the world, letting users steer scenes through keyboard, mouse or text prompts while nothing inside perceived the user or decided how to respond. Duo v1 cuts interaction into 17-frame chunks, about 0.71 s at 24 fps; a language intent module decides per chunk whether to speak or wait, and a causal audiovisual module generates the matching sound and picture. Of 180 checks across 30 real-time duplex cases and six binary criteria, 168 passed, including all 90 on language, speech intelligibility and visual identity, while content responsiveness reached 73.3%.

  2. arXiv

    Depression screening gets more than a label: InertiaMPDD measures departure from a healthy reference and corrects each decision

    InertiaMPDD extends depression screening beyond a single recording-level label to a measurable departure from a healthy reference: a latent predictor of the next state is trained on frozen multimodal representations, and three descriptors are read from its prediction residuals, namely reference deviation, residual decay rate and a Jacobian spectral radius, used either for standalone screening or as an individual correction to a population decision. Across five cohorts from four datasets, the fused pathway reaches 98.41 F1 on D-Vlog and 84.98 F1 on LMVD, while on AVEC2014 the reference descriptors discriminate weakly and fusion reaches 66.67 F1.

  3. arXiv

    Frozen world models skip per-step backprop: WARP-JEPA swaps planning for learned proposals, speeding long-horizon episodes 9.6–18.8×

    Hierarchical Joint-Embedding Predictive Architectures (H-JEPA) previously planned by running gradient descent on actions at test time, and the iterative solver consumed 96.6%–97.7% of episode runtime on long-horizon tasks. WARP-JEPA keeps the world models frozen and replaces online optimization with learned multi-candidate trajectory proposers, scoring candidates with the frozen predictor inside the generation loop. Across 30 frozen two-, three- and four-level models on four benchmarks, it speeds long-horizon episode inference by 9.6–18.8× (2.9–3.0× on Cube) and raises task success by up to 33.3 percentage points.

  4. arXiv

    Cellular world models' high correlations fall apart: GEARS scores 0.977 globally but only 0.076 on perturbation deltas

    On 509 unseen perturbations from the Replogle K562 screen, GEARS reaches a global Pearson correlation of 0.977 between predicted and observed transcriptomes, but only 0.076 once the control baseline is subtracted and the perturbation-induced delta is scored; a trivial baseline that predicts the same mean training delta for every perturbation reaches 0.267. The gap arises because global correlation is dominated by thousands of non-responsive genes, so delta-space evaluation is proposed as the primary benchmark, together with two post-hoc corrections that need no retraining and a retrieval upper bound that localizes the residual.

  5. World models move to closed loop: a Transfer-Auto reference run on four GPUs returns seven 720p videos and 99 LiDAR sweeps

    NVIDIA has packaged world-foundation-model training, optimization and inference into Cosmos Framework, with agent-callable skills and checkpoints that run out of the box. Developers previously had to fit a model to their own sensors, action space and data, then get it into an application, two separate rework efforts. The released Cosmos 3 Nano simulation recipes cover driving, robotics and surgery: a Transfer-Auto reference run on four GPUs returns seven 720p videos and 99 LiDAR sweeps, while Sim-Auto, distilled into a causal few-step model, generates observations step by step from driving-policy input and returns them to the policy for closed-loop evaluation.

  6. Ai2 releases ToyBlocks, a controlled robot-learning sandbox: skills trained in simulation hit 81.1% on real hardware versus 80.0% in sim

    Ai2 released ToyBlocks, an open simulation framework built around a robot arm and tabletop blocks, the first controlled world in its FoundationWorlds program. It automatically generates tasks and demonstrations, letting researchers vary training data, repeat experiments and trace why a robot fails. In physical tests, the low-level skill model ManiFlow, trained only on simulation data, reached 80.0% success in simulation and 81.1% on the real robot, while the combined high-level planner plus low-level executor succeeded in 55.5% of simulated episodes and 50.0% of real-world episodes.

  7. arXiv

    Emotion as a causal variable: AffectWorld scores five world models, and counterfactual sensitivity tops out at 0.070

    AffectWorld evaluates world models with emotion treated as a causal variable driving future world change, and five state-of-the-art models reach only 0.100–0.280 on world-state change and 0.027–0.070 on counterfactual emotion sensitivity across 1,498 everyday videos.

  8. arXiv

    Robot world models stop learning only from success: training on human egocentric video and perturbed failures aligns predicted success with real deployment

    The action-conditioned world model HF-WM trains on egocentric human manipulation video together with perturbed failure actions, reaching state-of-the-art action-conditioned video generation and predicted success rates that track real-world deployment.

  9. arXiv

    Latent planners can pick the wrong way: at an acute bend, terminal distance ranks the backward candidate first

    When a latent world model ranks candidate actions by terminal Euclidean distance to a goal latent, an acute bend at a mandatory junction can make it prefer a candidate that moves backward over one that makes real progress, even with exact rollouts.

  10. arXiv

    World-model planning drops CEM: sampling tree search is 13.8x faster in simulation and lifts real-robot block pushing by 20%

    For long-horizon planning with world models, sampling-based tree search in latent space replaces optimizers such as CEM and MPPI, cutting planning time by up to 13.8x in simulation and raising real-robot block-pushing success by 20%.