Gheorghe Asachi Technical University of Iași

A vision Transformer's depth-wise stack is compressed into a single recurrent Transformer block, where the FFN at each recurrent depth is a convex combination of a small shared expert bank programmed by a normalized depth coordinate, so shared weights still yield depth-dependent computation. Under supervised ImageNet-1k training, reViT-B/16 reaches DeiT III accuracy with about 70% fewer stored parameters; under distillation that matches only the DINOv2 teacher's final-layer features, an 8-expert model approaches the teacher on frozen classification, segmentation, and depth probes. One checkpoint also runs at multiple tested depths by resampling the depth coordinate.

A vision Transformer's depth-wise stack is compressed into a single recurrent Transformer block, where the FFN at each recurrent depth is a convex combination of a small shared expert bank programmed by a normalized depth coordinate, so shared weights still yield depth-dependent computation. Under supervised ImageNet-1k training, reViT-B/16 reaches DeiT III accuracy with about 70% fewer stored parameters; under distillation that matches only the DINOv2 teacher's final-layer features, an 8-expert model approaches the teacher on frozen classification, segmentation, and depth probes. One checkpoint also runs at multiple tested depths by resampling the depth coordinate.

The authors define five requirements for latent reasoning (useful, diverse, explainable, refinable, efficient) and present Flow-based Latent Reasoning (FLaRe), a recipe built on flow matching in a learned latent space: train a VAE on symbolic CoTs, corrupt inputs and latent codes, train the flow mostly near pure noise, let the answer reader see both noised codes and the model's own predicted endpoints, and self-train on verified rollouts; probes show FLaRe improves on Coconut, CODI and PCCoT on all five requirements, compares favorably on arithmetic benchmarks, and reaches 97% of explicit CoT accuracy at about a quarter of its latency.

The authors define five requirements for latent reasoning (useful, diverse, explainable, refinable, efficient) and present Flow-based Latent Reasoning (FLaRe), a recipe built on flow matching in a learned latent space: train a VAE on symbolic CoTs, corrupt inputs and latent codes, train the flow mostly near pure noise, let the answer reader see both noised codes and the model's own predicted endpoints, and self-train on verified rollouts; probes show FLaRe improves on Coconut, CODI and PCCoT on all five requirements, compares favorably on arithmetic benchmarks, and reaches 97% of explicit CoT accuracy at about a quarter of its latency.