Sparse autoencoders trained on 909,873 CT/MRI slices decompose medical foundation-model embeddings into language-describable concepts, recovering 87.8% of downstream performance with 10 features
Trained on 909,873 2D CT and MRI slices from the TotalSegmentator dataset, this study fits Matryoshka sparse autoencoders with BatchTopK sparsification on frozen embeddings from BiomedParse (biomedical) and DINOv3 (general-purpose) foundation models alongside a random-weight baseline, finding that sparse features reconstruct original embeddings with R2 up to 0.941, recover 87.8% of downstream performance with only 10 features (99.4% dimensionality reduction), preserve 97.7% of dense retrieval quality with five-feature fingerprints, correspond to monosemantic concepts expressible in language as verified by an independent LLM judge, and enable zero-shot language-driven image retrieval on a single clinical text query.