MCMC and SMC samplers for the power distribution fail to converge under realistic compute budgets, and their reasoning gains are largely recreated by simple Best-of-N selection; Power-RLOO, which fine-tunes directly toward the power distribution, helps only when the base model is already aligned with the task.
The work introduces Power-SMC, a training-free sampling method that maintains multiple candidate sequences in parallel, scores each with an importance weight measuring how well it matches the sequence-level power distribution, and periodically prunes low-scoring candidates in favor of high-scoring ones, matching or exceeding Metropolis-Hastings sampling in accuracy on MATH500, GSM8K, GPQA and HumanEval, preserving output diversity, and accelerating inference by up to 17.6x.
The work introduces Parallel Power Tempering (PPT), which adapts classical parallel tempering (replica exchange) to sequence-level power-sharpened LLM sampling: multiple replicas at different sharpening levels run in parallel so that low-power replicas explore diverse reasoning trajectories while high-power chains exploit higher-likelihood responses, with swaps carrying promising traces to the sharpest output rung, and a fixed-horizon construction removes a structural truncation bias identified in early-stopped power samplers; across MATH500, GPQA, HumanEval, GSM8K, AIME 24&25 and LiveCodeBench v5 on Qwen3-4B, Qwen3-8B and Qwen3.5-9B, PPT attains best or tied-best accuracy in all model-benchmark settings, outperforming Power Sampling, PowerSMC and GRPO, and on Qwen3.
You’re all caught up