# Tabular Classification

TabOmics is an open tabular foundation model with 7.3M parameters, pretrained only on synthetic tables, that processes all features retained after preprocessing jointly in a single forward pass instead of subsampling or compressing them. Across 46 classification tasks from three high-dimensional, low-sample-size benchmarks, the eight-member TabOmics v2.5 reaches a mean AUROC of 0.943, above 0.942 for eight-member TabPFN-3.5; the single-member version reaches 0.941 with no significant paired difference, at roughly five times the runtime per forward pass.

TabOmics is an open tabular foundation model with 7.3M parameters, pretrained only on synthetic tables, that processes all features retained after preprocessing jointly in a single forward pass instead of subsampling or compressing them. Across 46 classification tasks from three high-dimensional, low-sample-size benchmarks, the eight-member TabOmics v2.5 reaches a mean AUROC of 0.943, above 0.942 for eight-member TabPFN-3.5; the single-member version reaches 0.941 with no significant paired difference, at roughly five times the runtime per forward pass.

Tabular foundation models do in-context learning by carrying every training example through each forward pass, so shrinking the context saves compute but costs accuracy. Activation alignment instead uses the full context to teach a data-constrained student: a single linear transformation is fitted on synthetic unlabeled data to pull the student's intermediate activations toward the teacher's. On 38 TabArena classification datasets with TabPFN-3 and TabFM, the aligned student beats the unaligned baseline across context budgets and recovers nearly half the teacher's advantage in low-data regimes.

Tabular foundation models do in-context learning by carrying every training example through each forward pass, so shrinking the context saves compute but costs accuracy. Activation alignment instead uses the full context to teach a data-constrained student: a single linear transformation is fitted on synthetic unlabeled data to pull the student's intermediate activations toward the teacher's. On 38 TabArena classification datasets with TabPFN-3 and TabFM, the aligned student beats the unaligned baseline across context budgets and recovers nearly half the teacher's advantage in low-data regimes.

The authors introduce PrivTab, a tabular foundation model that embeds a differential-privacy mechanism in its architecture: pretrained on simulated datasets, it uses in-context learning to compress sensitive rows into compact, provably private summaries, outperforming DP-LR and DP-MLP on 33 TabArena classification datasets across six privacy regimes under moderate-to-strong privacy, maintaining well-calibrated predictions, and reducing fitting time from minutes to a single forward pass of about 10 milliseconds.

The authors introduce PrivTab, a tabular foundation model that embeds a differential-privacy mechanism in its architecture: pretrained on simulated datasets, it uses in-context learning to compress sensitive rows into compact, provably private summaries, outperforming DP-LR and DP-MLP on 33 TabArena classification datasets across six privacy regimes under moderate-to-strong privacy, maintaining well-calibrated predictions, and reducing fitting time from minutes to a single forward pass of about 10 milliseconds.