Most of the no-think RL gain in hybrid reasoning models is thinking leakage: erasing an 8-D think subspace at 8B removes 75% of it

Reinforcement learning in no-think mode moves a hybrid reasoning model's outputs and internal state toward its own think mode at the same time as accuracy rises. The common recipe trains in no-think mode and reports the no-think accuracy gain as newly learned reasoning ability. For Qwen3-8B, accuracy rises from 0.17 to 0.43, responses grow from 2.7k to 7.2k tokens, reflection words rise from 1.3 to 13.0, the original model as judge rates the responses as more think-like in 94% of comparisons against its own no-think responses, and the middle-layer activation shift reaches 0.56 of the difference between the original model's two modes.

The gain is not a new capability but a reactivation of reasoning already present in the original model's think mode. Behavioral similarity alone cannot rule out that RL learned new problem-solving behavior that merely looks similar, so interventions on weights, activations and attention heads are needed. Scaling the RL update makes the model emit </think>, a token absent from 1,260 no-think responses, at nearly the same relative position as in think mode, and accuracy reaches 0.72 without exceeding the original think-mode 0.74; at 8B, replacing the 8-dimensional think subspace with the original model's value removes 75% of the gain and the reverse exchange transfers 50%, while a PCA control removes 22% and a random subspace 3%.

The shift passes through the attention heads that read the empty think block, and a source for it appears in the GRPO training signal. It was unclear which parts of the network carry the mode difference, and why a reward that only checks the final answer would push the model toward think mode. About 90% of heads place less than 1% of their attention on the empty think block and only five to seven heads reach 10%; knocking out the top 5% of heads by that attention nearly matches deleting the block, while random heads of the same count have almost no effect; single heads account for large shares of the activation shift, 44% for L18H19 at 8B and 54% for L19H13 at 1.7B, and these heads are among the strongest block attenders; within a group, responses favored by the advantage-weighted update are more think-like, and one gradient step from that signal moves the model toward its think state.

The results cover Qwen3-1.7B, 4B and 8B, trained with GRPO on DAPO-Math-17k and evaluated on 90 AIME 2022–2024 problems with up to 32,768 generated tokens per response. The weight-scaling factors were explored on the same 90 problems, so that probe concerns the direction of the learned update and its large factors lie outside the range seen in training. The activation patching and attention-head analyses center on 8B, and the head experiments read the first 2,048 tokens of fixed responses under teacher forcing.

The authors describe the gain as coming largely, not entirely, from thinking leakage, and the scaling factors were explored on the same AIME problems rather than held out. The training-signal analysis reports associations across groups, where the advantage is set by the number of correct responses, so more think-like responses also appear on harder questions. The one-step gradient probe measures the initial gradient direction rather than an actual training step. The authors note that the activation shift arises mainly from frozen weights acting on changed inputs, which by itself does not distinguish reuse of pretrained computation from new computation.