# Multi-hop Reasoning

PatternBloom changes the retrieval unit of agentic retrieval-augmented generation from what content to read to how to reason, using typed graph skeletons distilled from high-scoring Stage-I reinforcement-learning trajectories into an external Graph Pattern Memory, then coupling the policy to that frozen memory with a pattern-coverage reward. Across seven QA benchmarks, the 7B version ranks first on EM and F1 on all seven datasets, leads R1-Searcher on average out-of-distribution EM, and the same frozen pattern memory transfers to a Qwen3-8B backbone.

PatternBloom changes the retrieval unit of agentic retrieval-augmented generation from what content to read to how to reason, using typed graph skeletons distilled from high-scoring Stage-I reinforcement-learning trajectories into an external Graph Pattern Memory, then coupling the policy to that frozen memory with a pattern-coverage reward. Across seven QA benchmarks, the 7B version ranks first on EM and F1 on all seven datasets, leads R1-Searcher on average out-of-distribution EM, and the same frozen pattern memory transfers to a Qwen3-8B backbone.

The authors introduce universal transformers: fixed transformers whose internal parameters stay fixed while a suitable input embedding encodes a description of a target model, letting them simulate any transformer in a given class; they give explicit sparse constructions achieving universality when the embedding dimension is sufficiently large, show that randomly initialized transformers are universal almost surely, and empirically validate the theory on parenthesis balancing and multi-hop reasoning, suggesting much of a transformer's expressive power may reside in its input representation rather than its learned weights.

The authors introduce universal transformers: fixed transformers whose internal parameters stay fixed while a suitable input embedding encodes a description of a target model, letting them simulate any transformer in a given class; they give explicit sparse constructions achieving universality when the embedding dimension is sufficiently large, show that randomly initialized transformers are universal almost surely, and empirically validate the theory on parenthesis balancing and multi-hop reasoning, suggesting much of a transformer's expressive power may reside in its input representation rather than its learned weights.