# REST (REpresentation-Supervised Thoughts)

The work shows that training latent recursive LLM systems with only final-answer cross-entropy leaves the thought representation subject to four failures, and introduces REST, which turns causality, minimality, separability and stability into differentiable losses added to cross-entropy, improving accuracy over the CE-only baseline by an average of 3.5 points in the multi-agent setting and 3.3 in the single-agent setting, up to 7.5 and 6.5 points at best, and raising the rate of convergence on a final answer from 73% to 95%, across 7 benchmarks in mathematics, science, medicine and code generation under matched training data and latent budget.

The work shows that training latent recursive LLM systems with only final-answer cross-entropy leaves the thought representation subject to four failures, and introduces REST, which turns causality, minimality, separability and stability into differentiable losses added to cross-entropy, improving accuracy over the CE-only baseline by an average of 3.5 points in the multi-agent setting and 3.3 in the single-agent setting, up to 7.5 and 6.5 points at best, and raising the rate of convergence on a final answer from 73% to 95%, across 7 benchmarks in mathematics, science, medicine and code generation under matched training data and latent budget.