# Mixture-of-Transformers

NVIDIA's Cosmos world-model development platform adds four open-weight checkpoints, each post-trained from Cosmos 3 Nano with 16B trainable parameters, covering multi-view driving data generation, synchronized camera-LiDAR streaming driving simulation, photorealistic rendering from simulator controls, and video prediction from dual-arm robot actions. The checkpoints ship with a four-stage post-training recipe under OpenMDW1.1, which permits commercial and non-commercial use; NVIDIA also states that measuring where these models faithfully support policy learning and evaluation, and where their limits lie, remains ahead.

NVIDIA's Cosmos world-model development platform adds four open-weight checkpoints, each post-trained from Cosmos 3 Nano with 16B trainable parameters, covering multi-view driving data generation, synchronized camera-LiDAR streaming driving simulation, photorealistic rendering from simulator controls, and video prediction from dual-arm robot actions. The checkpoints ship with a four-stage post-training recipe under OpenMDW1.1, which permits commercial and non-commercial use; NVIDIA also states that measuring where these models faithfully support policy learning and evaluation, and where their limits lie, remains ahead.

YuE2 uses a single roughly 3.58B-parameter AR–NAR Mixture-of-Transformers to first write an ABC score specifying melody, chords, key, meter, tempo, and form, then expand it into semantic music tokens and render full-song audio, with companion models MERT2 and SheetSage2 constructing semantic and symbolic supervision from recordings; with the same checkpoint, experts prefer symbolic planning 49.3% to 34.6% for overall quality, YuE2 scores 6.7316 on SongBench Global Avg on WildSongBench (6.9632 with best-of-8), MERT2 surpasses previous best results on 14 of 15 MARBLE metrics, and SheetSage2-AR leads 12 of 15 benchmark–metric pairs.

YuE2 uses a single roughly 3.58B-parameter AR–NAR Mixture-of-Transformers to first write an ABC score specifying melody, chords, key, meter, tempo, and form, then expand it into semantic music tokens and render full-song audio, with companion models MERT2 and SheetSage2 constructing semantic and symbolic supervision from recordings; with the same checkpoint, experts prefer symbolic planning 49.3% to 34.6% for overall quality, YuE2 scores 6.7316 on SongBench Global Avg on WildSongBench (6.9632 with best-of-8), MERT2 surpasses previous best results on 14 of 15 MARBLE metrics, and SheetSage2-AR leads 12 of 15 benchmark–metric pairs.

Mira-Scene is a compositional single-image 3D scene reconstruction framework that pairs a pixel-aligned Canonical Coordinate Map (CCM) with a scene-space Point Cloud Map (PCM) from monocular geometry estimation to form dense, bounded correspondences, recovers object transformations through robust geometric alignment, and jointly generates object geometry and CCMs with a multimodal diffusion transformer; across indoor, outdoor, synthetic, and in-the-wild scenes it improves layout accuracy over strong baselines, achieving relative gains of 39.8% in 3D-IoU and 16.5% in 2D-IoU over SAM3D on BlendSwap while using limited open-source training data.

Mira-Scene is a compositional single-image 3D scene reconstruction framework that pairs a pixel-aligned Canonical Coordinate Map (CCM) with a scene-space Point Cloud Map (PCM) from monocular geometry estimation to form dense, bounded correspondences, recovers object transformations through robust geometric alignment, and jointly generates object geometry and CCMs with a multimodal diffusion transformer; across indoor, outdoor, synthetic, and in-the-wild scenes it improves layout accuracy over strong baselines, achieving relative gains of 39.8% in 3D-IoU and 16.5% in 2D-IoU over SAM3D on BlendSwap while using limited open-source training data.

InternW0-Δ couples a pretrained video expert and an action expert through a directed Mixture-of-Transformers, uses a frozen VLM for scene semantics, learns future-relevant scene changes via Causal Imprint from training-only future supervision, and distills 4D geometric and motion priors from a Track4World teacher at training time only; pretrained on a corpus of over 20K hours unifying robot demonstrations, UMI data, egocentric human demonstrations, and Ego2Robot data under a canonical state-action representation, it reaches 92.8% on LIBERO-Plus, 71.9% on RoboTwin 2.0 Clean2Random, an overall score of 66.0 on EBench, and an average success rate of 23.91% on RoboDojo, with deployment on four real-robot platforms (two gripper-based, two dexterous-hand).

InternW0-Δ couples a pretrained video expert and an action expert through a directed Mixture-of-Transformers, uses a frozen VLM for scene semantics, learns future-relevant scene changes via Causal Imprint from training-only future supervision, and distills 4D geometric and motion priors from a Track4World teacher at training time only; pretrained on a corpus of over 20K hours unifying robot demonstrations, UMI data, egocentric human demonstrations, and Ego2Robot data under a canonical state-action representation, it reaches 92.8% on LIBERO-Plus, 71.9% on RoboTwin 2.0 Clean2Random, an overall score of 66.0 on EBench, and an average success rate of 23.91% on RoboDojo, with deployment on four real-robot platforms (two gripper-based, two dexterous-hand).