Alignment without scale: good views let a 109M text JEPA beat 72B language models on the Platonic representation benchmark
Synopsis
The Platonic Representation Hypothesis, which holds that separately trained models converge on a shared picture of the world, becomes a theorem for JEPAs (joint-embedding predictive architectures): if each modality's positive pairs link the same world states, the encoders agree up to one rotation. The resulting good-view versus bad-view distinction predicts alignment, and a 109M-parameter text JEPA reaches .184 on the PRH benchmark, above every language model evaluated, up to Qwen2.5-72B at .154.
Interpretation
There is now a provable sufficient condition for when separately trained encoders of different modalities learn the same representation: they must agree on which world states are views of one another. The Platonic Representation Hypothesis had been an observation, with the closest proof covering embedded deep linear networks trained by SGD and no proof or prediction for the self-supervised objectives used in practice. In an idealized setting of a finite world, injective renderings, population optima and minimizers unique up to rotation, any number of JEPAs trained separately agree up to one rotation each; when positives differ, misalignment is bounded by the mismatch of the kernels their positives induce, and on 300 random finite worlds the error stayed below that bound, at most 18% of it.
What decides alignment is the nature of the positive pairs, not model size: pairing two observations of one scene makes an encoder spend dimensions on the world, while pairing two corruptions of one observation makes it spend them on its modality's nuisance. Scaling models and data had been the main lever for cross-modal alignment. On Flickr30k with identical texts and encoders, the good-view JEPA reaches mutual 10-NN .062 against .047 for the bad-view JEPA and .042 for next-token prediction, and the scene-explained fractions are .804, .655 and .608, matching the order of the theoretical error bounds.
Next-token prediction is shown to be a bad-view JEPA, with a prefix and its continuation in one document as positives and the softmax log-partition as its anti-collapse term. This offers a reading of why language models align with vision only partly and mostly through scale. On the same corpus, the good-view JEPA reaches the alignment that next-token prediction reaches with 900M samples after 21.6M samples, 42 times fewer, and at matched samples a 33M encoder scores .108 against .071.
Alignment can be used as a tool: one fitted rotation joins a frozen image encoder to a decoder trained on text alone, and the rotation can even be found with no pairs at all. Earlier text-only captioners decoded the embeddings of jointly trained encoders built on hundreds of millions of image-text pairs. The captioner reaches CIDEr .838 on COCO, and under the unpaired protocol of Schnaus et al. the 109M text JEPA reaches 64.4% zero-shot ImageNet-100 top-5, 16.8 points above a rerun of Qwen3-8B under identical conditions.
Perspective
The theorem is stated for a finite world with injective renderings, matched marginals and population optima; uniqueness is proved for second-order anti-collapse, not for LeJEPA's full-distribution constraint, and the experiments use LeJEPA with crops as positives on most of the image side. Good views require knowing which texts describe the same scene: Flickr and COCO group human captions by image, and two of each Recap-DataComp image's three texts are written by an image captioner.
The authors note that real modalities are lossy and that finite-sample training relies on the network's inductive bias; the scaling law overestimates good-view runs at 320M samples and has no term for repeated data, its exponents are poorly identified, and it summarizes one recipe. At the 940M scale WIT is sensitive to the SIGReg weight, and the four seeds average .168, still above every language model evaluated.