Scaffolding Minds swaps in a learnable scaffolding encoder and an adaptive Gaussian sampler for latent visual reasoning, gaining 9.5 points on FrozenLake and 5.6 points across nine visual reasoning benchmarks
The work proposes Scaffolding Minds, a two-stage framework in which Stage 1 replaces the frozen off-the-shelf vision encoder with a scaffolding encoder trained end-to-end through the downstream task loss to produce the latent target, and Stage 2 replaces deterministic latent regularization with a Gaussian sampler whose mean and variance are learned so that residual latent actions can be sampled for RL exploration, improving over the strongest latent-reasoning baseline by 9.5 points on average on FrozenLake (19 points at Level 32) and by 5.6 points on average across nine visual-centric reasoning benchmarks.