# Latent Reasoning

HybridAgent lets a single agent move back and forth between continuous latent reasoning and text interaction within one generation, so part of its thinking leaves token-by-token decoding while tool use stays intact. Text agents previously had to write every step as tokens, while latent-reasoning models lacked native tool access and were hard to inspect. Across 11 benchmarks in mathematics, science, medicine, search and agentic reasoning at 4B to 14B scale, it reports a 10.4% average gain over frontier text and latent agents, with 35.5%–54.7% fewer tokens and 1.8–2.6 times faster inference.

HybridAgent lets a single agent move back and forth between continuous latent reasoning and text interaction within one generation, so part of its thinking leaves token-by-token decoding while tool use stays intact. Text agents previously had to write every step as tokens, while latent-reasoning models lacked native tool access and were hard to inspect. Across 11 benchmarks in mathematics, science, medicine, search and agentic reasoning at 4B to 14B scale, it reports a 10.4% average gain over frontier text and latent agents, with 35.5%–54.7% fewer tokens and 1.8–2.6 times faster inference.

Typed decision models used to return option probabilities in a single forward pass, giving every decision a fixed amount of computation. SanSi turns a language model pre-trained to loop into a typed decision model, reading the option probabilities after every loop and training each with a proper scoring rule, so one model serves every budget from one loop to eight. On 10,027 test decisions from 59 sources it reaches 72.0% accuracy, 13.5 points above a non-looped model of the same shape trained with the same recipe and 1.8 points below a single-pass model with three times its parameters, at 7.7 times the GPU time of one pass.

Typed decision models used to return option probabilities in a single forward pass, giving every decision a fixed amount of computation. SanSi turns a language model pre-trained to loop into a typed decision model, reading the option probabilities after every loop and training each with a proper scoring rule, so one model serves every budget from one loop to eight. On 10,027 test decisions from 59 sources it reaches 72.0% accuracy, 13.5 points above a non-looped model of the same shape trained with the same recipe and 1.8 points below a single-pass model with three times its parameters, at 7.7 times the GPU time of one pass.

The authors define five requirements for latent reasoning (useful, diverse, explainable, refinable, efficient) and present Flow-based Latent Reasoning (FLaRe), a recipe built on flow matching in a learned latent space: train a VAE on symbolic CoTs, corrupt inputs and latent codes, train the flow mostly near pure noise, let the answer reader see both noised codes and the model's own predicted endpoints, and self-train on verified rollouts; probes show FLaRe improves on Coconut, CODI and PCCoT on all five requirements, compares favorably on arithmetic benchmarks, and reaches 97% of explicit CoT accuracy at about a quarter of its latency.

The authors define five requirements for latent reasoning (useful, diverse, explainable, refinable, efficient) and present Flow-based Latent Reasoning (FLaRe), a recipe built on flow matching in a learned latent space: train a VAE on symbolic CoTs, corrupt inputs and latent codes, train the flow mostly near pure noise, let the answer reader see both noised codes and the model's own predicted endpoints, and self-train on verified rollouts; probes show FLaRe improves on Coconut, CODI and PCCoT on all five requirements, compares favorably on arithmetic benchmarks, and reaches 97% of explicit CoT accuracy at about a quarter of its latency.

The work introduces the Trajectory Adaptive Progress–Fluctuation Scheduler (TAPS), which uses exponential moving averages to estimate persistent progress and centered fluctuation in recurrent updates and adapts the per-step update scale online; it proves that under stated conditions TAPS reduces expected terminal loss and reaches a target quality in fewer loops, and empirically improves terminal accuracy with up to roughly 1.5x wall-clock speedup across Sudoku, Maze, language-model recurrence, and intermediate-layer recurrence.

The work introduces the Trajectory Adaptive Progress–Fluctuation Scheduler (TAPS), which uses exponential moving averages to estimate persistent progress and centered fluctuation in recurrent updates and adapts the per-step update scale online; it proves that under stated conditions TAPS reduces expected terminal loss and reaches a target quality in fewer loops, and empirically improves terminal accuracy with up to roughly 1.5x wall-clock speedup across Sudoku, Maze, language-model recurrence, and intermediate-layer recurrence.

The work shows that training latent recursive LLM systems with only final-answer cross-entropy leaves the thought representation subject to four failures, and introduces REST, which turns causality, minimality, separability and stability into differentiable losses added to cross-entropy, improving accuracy over the CE-only baseline by an average of 3.5 points in the multi-agent setting and 3.3 in the single-agent setting, up to 7.5 and 6.5 points at best, and raising the rate of convergence on a final answer from 73% to 95%, across 7 benchmarks in mathematics, science, medicine and code generation under matched training data and latent budget.

The work shows that training latent recursive LLM systems with only final-answer cross-entropy leaves the thought representation subject to four failures, and introduces REST, which turns causality, minimality, separability and stability into differentiable losses added to cross-entropy, improving accuracy over the CE-only baseline by an average of 3.5 points in the multi-agent setting and 3.3 in the single-agent setting, up to 7.5 and 6.5 points at best, and raising the rate of convergence on a final answer from 73% to 95%, across 7 benchmarks in mathematics, science, medicine and code generation under matched training data and latent budget.