# Kolibri

Aleph Alpha's Kolibri is a hybrid reasoning model whose system prompt carries one sentence selecting no reasoning, low, medium or high effort. Because its supervised fine-tuning traces carried no effort labels, the team annotated each trace post hoc by its reasoning length relative to traces from the same dataset at the same judged difficulty, with a 2B student model reproducing a larger model's difficulty grades. Median reasoning length orders low below medium below high on 23 of 24 benchmarks; low keeps 96% of the high score at 35% of the length, and the released model keeps 98% at 76% of the completion tokens.

Aleph Alpha's Kolibri is a hybrid reasoning model whose system prompt carries one sentence selecting no reasoning, low, medium or high effort. Because its supervised fine-tuning traces carried no effort labels, the team annotated each trace post hoc by its reasoning length relative to traces from the same dataset at the same judged difficulty, with a 2B student model reproducing a larger model's difficulty grades. Median reasoning length orders low below medium below high on 23 of 24 benchmarks; low keeps 96% of the high score at 35% of the length, and the released model keeps 98% at 76% of the completion tokens.

In sparse mixture-of-experts models, a router that favors a few experts starves the rest of training signal and makes devices wait under expert parallelism. Kolibri pre-training pairs Exact Quantile Balancing (EQB), which sets routing biases from exact global-batch quantiles via two-pass radix selection, with bounded Load-Error Injection (LEI), which adds each microbatch's load error directly to router gradients. On a 7.5B-parameter proxy trained for 100B tokens, EQB cut global maximal violation from 0.92 to 0.74, and adding bounded LEI cut local violation from 5.38 to 3.52 at a mean BPB cost from 0.6570 to 0.6720.

In sparse mixture-of-experts models, a router that favors a few experts starves the rest of training signal and makes devices wait under expert parallelism. Kolibri pre-training pairs Exact Quantile Balancing (EQB), which sets routing biases from exact global-batch quantiles via two-pass radix selection, with bounded Load-Error Injection (LEI), which adds each microbatch's load error directly to router gradients. On a 7.5B-parameter proxy trained for 100B tokens, EQB cut global maximal violation from 0.92 to 0.74, and adding bounded LEI cut local violation from 5.38 to 3.52 at a mean BPB cost from 0.6570 to 0.6720.