PowerMH must decide each accept/reject before it can generate the next proposal; PreSTO prefetches several future proposals in one batched call, giving a median 1.71× speedup over PowerMH across seven datasets and five base LLMs while leaving its transition kernel unchanged.
This work systematically studies how test-time compute affects predictions of pretrained tabular foundation models along three axes—adaptation, aggregation, and context construction: it introduces DiagScale, a diagonal query-key similarity update training only 0.003-0.03% of parameters that matches full fine-tuning gains across three independently pretrained backbones; on a broader pool of 96 configurations, greedy selection reduces error by 2.4% relative to the default predictor while uniform averaging increases error; attention-guided retrieval improves TabPFN-3 on some large tables, whereas the context expansion methods tested yield no consistent improvement.
The work introduces Parallel Power Tempering (PPT), which adapts classical parallel tempering (replica exchange) to sequence-level power-sharpened LLM sampling: multiple replicas at different sharpening levels run in parallel so that low-power replicas explore diverse reasoning trajectories while high-power chains exploit higher-likelihood responses, with swaps carrying promising traces to the sharpest output rung, and a fixed-horizon construction removes a structural truncation bias identified in early-stopped power samplers; across MATH500, GPQA, HumanEval, GSM8K, AIME 24&25 and LiveCodeBench v5 on Qwen3-4B, Qwen3-8B and Qwen3.5-9B, PPT attains best or tied-best accuracy in all model-benchmark settings, outperforming Power Sampling, PowerSMC and GRPO, and on Qwen3.
You’re all caught up