Earth embeddings stop being a tangle of mixed dimensions: sparse autoencoders pull out nameable geographic features, though a random dictionary already earns part of the score
Synopsis
Earth-observation foundation models such as AlphaEarth and Tessera compress each 10-metre pixel into a dense vector whose individual dimensions often mix several land-surface concepts. Sparse autoencoders expand that vector over an overcomplete dictionary and keep only a few atoms active, yielding sparse, approximately monosemantic features. At matched sparsity, three sparse-autoencoder variants reconstruct better than PCA, ICA, NMF and raw dimensions; yet a control whose decoder is a frozen random dictionary also shows more specific features and finer spatial scales, so those two properties follow from sparsity itself, and only a learned dictionary cuts AlphaEarth's unexplained variance eightfold.
Interpretation
An Earth embedding can be split into a set of sparse, approximately monosemantic geographic features, each tied to a land-surface concept a person can name, rather than raw dimensions that mix several unrelated concepts. These embeddings were largely opaque: intrinsic dimensionality is far below vector length, a few dimensions correlate with specific land-surface properties, but most mix several concepts, which hinders both interpretation and assessment of robustness under distribution shift. On AlphaEarth, the large majority of live Matryoshka sparse-autoencoder features align with at most one concept group, some with exactly one and the rest with none, whereas a raw dimension aligns with a median of several concept groups and a PCA component likewise, so rotating the coordinates alone does not remove polysemanticity.
The sparse code spreads embedding variance onto finer spatial scales, so high-frequency variation such as fields and roads gets dedicated features. Raw dimensions and PCA, ICA and NMF components have median correlation lengths of tens to hundreds of kilometres and change mainly between distant places, so fine-scale variation is outweighed. On AlphaEarth, sparse-autoencoder features have median correlation lengths of a few to about a dozen kilometres and a larger share of variance within a 320-pixel patch than raw dimensions; the frozen-random-decoder control spreads over scales similarly, so this property comes from sparsity rather than the learned dictionary.
Only a learned dictionary reconstructs the embedding accurately, and that is where sparse autoencoders genuinely gain over the random-dictionary control. Because the random-dictionary control can choose among its random atoms, it already explains more AlphaEarth variance than the first principal components, so interpretability scores alone cannot separate it from a trained model. At matched active features per pixel, all three sparse-autoencoder variants beat every baseline on both AlphaEarth and Tessera; on AlphaEarth, trained sparse autoencoders leave eight times less variance unexplained than the frozen-decoder control.
Sparse features show promise in two applications: discovering sub-concepts inside coarse land-cover classes, and flagging out-of-distribution pixels through area-of-applicability estimation. Neither was a routine use of sparse codes, and existing practice relies on distances in dense representations, where a dimension mixing concepts can place genuinely unrelated pixels close together. Three hand-picked Matryoshka features concentrate in tropical rainforest, hot semi-arid and subarctic climates respectively; for canopy height, the ridge-weighted distance in a BatchTopK code ranks errors better than the raw distance on most hold-outs, but it does not help for elevation, and on AlphaEarth an untrained random sparse code gains as much.
Perspective
The experiments run on two pixelwise Earth embeddings, AlphaEarth and Tessera, sampled as 320-pixel patches from the MajorTOM grid, with all numbers reported on test patches; the protocol is designed to apply to any sparse autoencoder, and code and a data recipe are released.
Single-feature statistics such as concept-alignment counts and spatial scale do not separate a trained sparse autoencoder from a frozen random decoder, so they cannot alone show that the dictionary learned meaningful features; the applicability estimate targets covariate shift only, and its hold-outs are synthetic, so whether it helps must be checked against held-out errors for each new target.