# TabFM

3 related articles

Subscribe to updates

arXiv Oct 6, 2026
Tabular foundation models can drop 90% of context: one linear alignment layer, 10% of data matches two to four times more
Tabular foundation models do in-context learning by carrying every training example through each forward pass, so shrinking the context saves compute but costs accuracy. Activation alignment instead uses the full context to teach a data-constrained student: a single linear transformation is fitted on synthetic unlabeled data to pull the student's intermediate activations toward the teacher's. On 38 TabArena classification datasets with TabPFN-3 and TabFM, the aligned student beats the unaligned baseline across context budgets and recovers nearly half the teacher's advantage in low-data regimes.
Tabular foundation models do in-context learning by carrying every training example through each forward pass, so shrinking the context saves compute but costs accuracy. Activation alignment instead uses the full context to teach a data-constrained student: a single linear transformation is fitted on synthetic unlabeled data to pull the student's intermediate activations toward the teacher's. On 38 TabArena classification datasets with TabPFN-3 and TabFM, the aligned student beats the unaligned baseline across context budgets and recovers nearly half the teacher's advantage in low-data regimes.
Yoel Zeldes（First author, Corresponding author）
Activation Alignment Tabular In-Context Learning TabPFN-3 TabFM Tabular Classification
arXiv Sep 30, 2026
TabFM-Auto pairs an LLM agent with frozen TabFM to evolve data pipelines, lifting Elo from 1785 to 2013 across all 51 TabArena datasets
TabFM-Auto pairs the frozen tabular foundation model TabFM with a language-model coding agent that iteratively rewrites four pipeline stages—data cleaning, feature engineering, context selection, and post-processing—guided by dataset metadata and validation feedback; across all 51 TabArena datasets five configurations take the top five overall positions, the best (Codex with Opus 5) raises TabFM from 1785.3 to 2013.0 Elo, the discovered pipelines transfer without further search to TabPFN-3, TabICLv2, and EXAONE-Tabular (+69 to +143 Elo), and TabFM-Auto ranks first overall among MLE agents on the 8 tabular competitions of MLE-Bench.
TabFM-Auto pairs the frozen tabular foundation model TabFM with a language-model coding agent that iteratively rewrites four pipeline stages—data cleaning, feature engineering, context selection, and post-processing—guided by dataset metadata and validation feedback; across all 51 TabArena datasets five configurations take the top five overall positions, the best (Codex with Opus 5) raises TabFM from 1785.3 to 2013.0 Elo, the discovered pipelines transfer without further search to TabPFN-3, TabICLv2, and EXAONE-Tabular (+69 to +143 Elo), and TabFM-Auto ranks first overall among MLE agents on the 8 tabular competitions of MLE-Bench.
The University of Texas at Austin University of Southern California Harvard University
Deqing Fu（First author）
TabFM-Auto TabFM Tabular Foundation Models LLM agent Tabular Prediction
arXiv Sep 30, 2026
Google's 400M-parameter TabFM tops all 51 TabArena datasets zero-shot, and TabFM-Auto adds Elo with frozen weights
TabFM is a 400M-parameter tabular foundation model that formulates supervised tabular prediction as in-context learning, is pretrained entirely on synthetic tables generated from structural causal models, and produces calibrated zero-shot predictions in a single forward pass; across all 51 TabArena datasets (38 classification, 13 regression) zero-shot TabFM ranks first among default tabular foundation models and outperforms tuned AutoML pipelines, while its two extensions, TabFM+ (multi-view feature expansion, ensembling, and post-hoc calibration) and TabFM-Auto (LLM-guided, dataset-specific data processing and feature engineering), improve both tracks over the same frozen weights.
TabFM is a 400M-parameter tabular foundation model that formulates supervised tabular prediction as in-context learning, is pretrained entirely on synthetic tables generated from structural causal models, and produces calibrated zero-shot predictions in a single forward pass; across all 51 TabArena datasets (38 classification, 13 regression) zero-shot TabFM ranks first among default tabular foundation models and outperforms tuned AutoML pipelines, while its two extensions, TabFM+ (multi-view feature expansion, ensembling, and post-hoc calibration) and TabFM-Auto (LLM-guided, dataset-specific data processing and feature engineering), improve both tracks over the same frozen weights.
Google Research
Weihao Kong（First author）
TabFM Tabular Foundation Model In-Context Learning Tabular Prediction TabFM-Auto

You’re all caught up

Tabular foundation models can drop 90% of context: one linear alignment layer, 10% of data matches two to four times more

TabFM-Auto pairs an LLM agent with frozen TabFM to evolve data pipelines, lifting Elo from 1785 to 2013 across all 51 TabArena datasets

Google's 400M-parameter TabFM tops all 51 TabArena datasets zero-shot, and TabFM-Auto adds Elo with frozen weights