# DiagScale

This work systematically studies how test-time compute affects predictions of pretrained tabular foundation models along three axes—adaptation, aggregation, and context construction: it introduces DiagScale, a diagonal query-key similarity update training only 0.003-0.03% of parameters that matches full fine-tuning gains across three independently pretrained backbones; on a broader pool of 96 configurations, greedy selection reduces error by 2.4% relative to the default predictor while uniform averaging increases error; attention-guided retrieval improves TabPFN-3 on some large tables, whereas the context expansion methods tested yield no consistent improvement.

This work systematically studies how test-time compute affects predictions of pretrained tabular foundation models along three axes—adaptation, aggregation, and context construction: it introduces DiagScale, a diagonal query-key similarity update training only 0.003-0.03% of parameters that matches full fine-tuning gains across three independently pretrained backbones; on a broader pool of 96 configurations, greedy selection reduces error by 2.4% relative to the default predictor while uniform averaging increases error; attention-guided retrieval improves TabPFN-3 on some large tables, whereas the context expansion methods tested yield no consistent improvement.