Tabular foundation models stop learning only sampled hard labels: mean rank improves across three architectures, with 62.1% of paired comparisons won
Tabular foundation model pretraining has scored query predictions against sampled hard labels, so a fixed input keeps supervision from only one class and sampling variability stays in pretraining. GIST designs synthetic task generation and query supervision together: on tractable branches it averages over the conditional label distribution using generator states retained during synthesis, and an entropy budget calibrates the noise scale of the category encoding. Under matched pretraining schedules, TabICLv2, TabSwift and TabPFN-2.5 all improve mean rank on TabArena and TALENT without changing architecture or inference, winning 62.1% of paired dataset-level comparisons across these benchmarks and architectures against the corresponding hard-label controls.