FLaRe reasons in a latent space with flow matching, beating prior latent methods on five probes and reaching 97% of explicit CoT accuracy at a quarter of the latency

The paper defines five requirements for latent reasoning, that a thought be useful, diverse, explainable, refinable and efficient, and designs a probe for each. Prior work largely evaluates latent reasoning by accuracy or efficiency alone, without systematically testing whether the thought is actually used by the answer or whether resampling explores different trajectories. On the GSM8K test set with Llama-3.2-1B-Instruct backbones, compared against Coconut, CODI, PCCoT and explicit CoT; the twin test uses 824 pairs that change one number, the sampling test draws 16 thoughts per question, and the explainability test checks whether reference intermediate values appear in order.

The paper distills a training recipe, FLaRe, that makes latent flow matching work, covering the latent space, flow training, answer readout and self-training. A standard text-diffusion pipeline falls far behind explicit CoT on reasoning; controlled ablations identify the choices that matter, including symbolic CoT input with dual decoder routes, three corruptions, a time distribution shifted toward noise, an answer reader trained on the model's own detached endpoints, and a stage 2 that self-trains on verified rollouts. Ablations span 66 VAEs, several time distributions, five answer-pass settings, several training-data combinations and each component of stage 2; the default uses 8 latent slots, a Llama-3.2-1B encoder and a fine-tuned 3B decoder, with Llama-3.2-1B-Instruct as the flow model.

FLaRe improves on prior latent methods on all five requirements and compares favorably on arithmetic benchmarks in accuracy and efficiency. Existing methods follow an injected thought on 7% to 32% of twin pairs, gain little coverage from resampling, and are constrained by their training budgets; FLaRe stage 2 follows on 76% of clean twin pairs, gains 14.7 points of pass@16 on the decoded reading, and rises from 36.6 to 62.6 decoded accuracy as the budget grows from one to twenty steps. Compared on GSM8K, GSM8K-Hard, SVAMP and MultiArith at 0.5B, 1B and 3B scales; latency is measured one question at a time on a single RTX 3090, where the stage 2 direct reading reaches 57.4 in two steps, about half of CODI's latency and a quarter of explicit CoT's.

Stage 2 self-training needs only new questions and reference answers, without extra CoT annotations, a learned reward or a reinforcement learning objective. Unlike methods that rely on a CoT curriculum or distillation, this stage re-encodes the model's own thoughts, verified by a frozen decoder against the reference answer, as flow targets and backpropagates the answer loss through the full rollout. Ablations show that removing the answer loss returns to stage 1 levels, detaching the rollout endpoint loses 2.3 points of direct accuracy, a one-step rollout loses 2.8, and removing verification loses 2.0; the remaining variants stay within one point of full stage 2.

The recipe targets arithmetic-style reasoning tasks supervised by symbolic CoTs with automatically checkable answers; the default setting uses 8 latent slots, a Llama-3.2-1B encoder and a fine-tuned 3B decoder, with Llama-3.2-1B-Instruct as the flow model and GSM8K-Aug plus Diverse CoT as training data. For datasets with only natural language CoTs, the authors provide an offline LLM-based conversion into symbolic form. Variable-width latent thoughts bring no gain on GSM8K-Aug because 8 slots already reconstruct 98.7% of test CoTs exactly, and the authors expect them to help on data whose CoTs vary more in length. Stage 2 needs only new questions and reference answers, without extra CoT annotations, a learned reward or a reinforcement learning objective, making it suitable where an answer checker already exists.

The twin test and the explainability check rely on intermediate values of the reference CoT, so a thought that takes another correct route counts as a miss, making the explainability numbers a conservative lower bound. Stage 2 verification only compares the final answer of the decoded CoT with the reference, not its intermediate steps. SVAMP is the main exception in the benchmark comparison, which the authors attribute to nearly half of its questions adding distractor numbers that are rare in GSM8K-Aug. Variable-width latent thoughts show no gain on GSM8K-Aug, and their value remains to be tested on data with more varied CoT lengths. The symbolic conversion pipeline depends on an LLM converter and hand-specified symbolic language rules, and the downstream effect of conversion quality is not separately quantified.