A whole table can become one vector: frozen tabular foundation models already carry dataset-level structure, and contrastive post-training makes retrieval robust

An entire supervised dataset can now be encoded as one fixed-dimensional vector, reusable for dataset-level tasks such as dataset retrieval, benchmark construction, and distribution-shift monitoring. Earlier approaches either relied on hand-crafted meta-features, whose representation is fixed by design and cannot adapt to new notions of dataset similarity, or trained dedicated dataset encoders from scratch without using representations from large-scale predictive pretraining. On real datasets unseen during training, the two TabMF encoders take the two strongest aggregate ranks across the six primary tasks, with median ranks of 1.5 and 2.0, and lead numerically on five of the six individual tasks.

The route is to turn a tabular foundation model that already predicts into an encoder of whole datasets, rather than training a separate network from scratch. TabMF attaches an architecture-aware dataset-level readout to a pretrained backbone and then adapts the whole model end-to-end with contrastive post-training on stochastic views of synthetic supervised datasets. Training views are generated on the fly from the TabICLv2 graph-structured prior, resampling rows with replacement and subsampling features without replacement while applying noise and Fourier transformations to 30% of selected columns with labels unchanged; the two encoders use 45,000 optimization steps on two NVIDIA H100 GPUs, about 315 and 273 GPU-hours.

Predictive pretraining by itself already leaves usable dataset-level structure inside the model, with no dataset-level training at all. Simply mean-pooling the internal states of a frozen backbone yields strong dataset embeddings, rather than requiring a purpose-trained dataset encoder. Frozen TabICL ICL-mean beats Dataset2Vec and the Neural Statistician on retrieval and stays competitive with learned encoders and classical meta-features on dataset-property prediction; on epiplexity prediction, frozen representations are stronger than their TabMF counterparts under the primary linear probe.

The evaluation covers classification datasets and two pretrained tabular foundation model backbones, with encoders post-trained on synthetic data and transferred to unseen real datasets. The intended readers are machine-learning researchers and engineers who need dataset retrieval, benchmark subset selection, or distribution-shift monitoring; at inference one dataset maps to a fixed-dimensional vector in a single forward pass. Post-training uses the TabICLv2 graph-structured prior, and extending the framework to regression and other architectures, or to broader synthetic priors and mixtures of synthetic and real datasets, is the next direction the source proposes.

The representations are explicitly supervised: changing the designated target can change the representation even when the feature table is fixed. On epiplexity prediction, frozen representations are stronger under the primary linear probe, and the authors note that linear results characterize readily accessible structure; with more expressive nonlinear probes, neither TabMF encoder is significantly separated from its corresponding frozen representation. The TableShift evaluation contains only nine task-level observations, so the authors do not interpret the numerical ordering as evidence of pairwise superiority. Both variants fall below PyMFE under quantization.