Same weights, two passes: Metis-1.6 trains Logos on 1.02T tokens to seventh of 17 on 40 benchmarks and second in mathematics
Synopsis
Metis-1.6 releases two from-scratch language models, Praxis and Logos, each running the same hybrid Mamba-2 and attention mixture-of-experts stack twice per token and each trained on 1.02T tokens. Logos averages 52.93 across 40 post-trained benchmarks, seventh of 17 and 0.75 points behind Qwen3-4B, while the comparators that disclose a count report a median of 12T training tokens and every model above it used at least 28T. It ranks second of 30 in mathematics with 67.50% on AIME 2025, and is weak at code, instruction following and long context.
Interpretation
Metis-1.6 trades depth for repetition: the same hybrid stack runs twice for every token instead of stacking more layers, so it stores half as many layers but passes through twice as many sequence-mixing and expert stages. The earlier Metis-1.5 was a 19-layer latent mixture-of-experts decoder pretrained on 50B tokens; Metis-1.6 was pretrained on sixteen times that data. Logos passes 40 stages per token (20 layers, two passes) and Praxis 24 (12 layers, two passes), with parameter counts read directly from the released checkpoints' tensor shapes.
The efficiency comes from a latent mixture of experts: experts work in a shared 1,024-dimensional space, so Logos stores 10.23B parameters but touches only 2.36B per token. Every model in the comparison set that averages above Logos activates between 3.57B and 10.94B parameters per token. Logos averages 52.93 across 40 post-trained benchmarks, seventh of 17 and 0.75 points behind Qwen3-4B; Praxis averages 36.62, fifteenth of 18.
Mathematics is where Logos earns its keep, averaging 65.83 over eleven math benchmarks, second of 30 and behind only Gemma 4 12B. It holds the highest score in its set on HMMT February 2025 (62.92), HMMT February 2026 (63.64) and IMO-AnswerBench (57.00). One harness ran Metis and every comparison model with identical prompts and scorers, and contest mathematics is scored as the mean over eight samples per problem.
The report also records training mistakes: the configured optimizer never ran, and attention matrices trained with AdamW for all 804.76B pretraining tokens. A parameter router assigned Muon to attention matrices by matching names containing .mixer.impl., but the attention module was mounted at layers.N.mixer, so every projection fell through to the default group. Auditing the trainer and the 100B- and 600B-token Logos checkpoints found four deviations present for the whole of pretraining, and the loss curve never showed them.
Perspective
The evaluation covers two from-scratch models trained on 1.02T tokens each, compared under the Lernex harness, prompts and scorers against open post-trained models of similar size; the author states these scores are not directly comparable with numbers published by model developers. Logos, Praxis and their Base counterparts are published on HuggingFace without the n-gram memory, and each repository includes the benchmark data behind the comparison tables.
The decontamination manifest predates AIME 2025 and 2026, HMMT 2025 and 2026, Brumo 2025, IMO-AnswerBench and Apex 2025, exactly the benchmarks where Logos is strongest relative to its comparators, and the author lists a dedicated overlap check as the first addition. Attention trained with AdamW while everything else used plain Muon, so the results do not isolate the architecture, and the 10B-token ablations run one arm each, so their standard errors reflect evaluation noise rather than run-to-run variance. The advantage clamp was used in the code, knowledge and agentic specialists but not in the reasoning specialist where the sign inversion was observed.