colibri runs hundred-billion-parameter MoE models on ordinary PCs: experts stay on disk, and 8 GB of RAM is enough to start
Synopsis
JustVugg released colibri, an open-source inference engine written in pure C with one file per model family and no GPU required. A mixture-of-experts model uses only a small part of itself for each token, so colibri keeps that part in RAM and reads the remaining experts from disk on demand. In the official example, the 744B-parameter int4 GLM-5.2 MoE model is ready in 32 seconds on CPU with 9.9 GB resident; the minimum is 8 GB of RAM, and the smallest model needs 22 GB of disk.
Interpretation
colibri runs open mixture-of-experts models in the hundreds of billions of parameters on an ordinary personal computer, with no graphics card, at least 8 GB of RAM, and 22 GB of disk for the smallest model. A mixture-of-experts model uses only a small part of itself for each token, so colibri keeps only that part resident in memory and leaves the remaining experts on disk, reading them when the model asks. The official example shows the 744B-parameter, int4-quantized GLM-5.2 MoE model ready in 32 seconds on CPU with 9.9 GB resident.
The engine is implemented in pure C, with one file per model family, and thirteen engines run today. Ten of them target language models, covering GLM-5.2/5.3, GLM-5.3-Flash, Inkling, Kimi K3, DeepSeek V4 Flash, DeepSeek V4.1 Flash, MiMo-V2.6 Flash and Pro, Qwen3.8-Flash-Next, Qwen3.6 and OLMoE. There is also one image engine, Qwen-Image-2.1, and two decision engines, Laya and GLiNER2.5-Decide, with a third decision model, Clef, on the Qwen3.6 engine.
Language models can read already-published GGUF files directly, run on CPU, and use any GPU through Vulkan or NVIDIA cards through CUDA. The setup script inspects RAM, disk, CPU and GPUs, recommends a model that fits, and builds the engine for the machine when a compiler is present, otherwise downloading a prebuilt one. Model downloads resume after interruption, and once started the engine exposes local OpenAI- and Anthropic-compatible endpoint addresses for other applications.
Perspective
The demonstrated scope is running the listed model families with a pure-C engine on CPU or an optional GPU, on a machine with at least 8 GB of RAM and 22 GB of disk for the smallest model, with local compatible endpoints exposed.
The official example gives only one set of figures, GLM-5.2 ready in 32 seconds on CPU with 9.9 GB resident, and does not state generation speed; integrated GPUs are judged faster only for Qwen3.6, Qwen3-Coder and Qwen3.8-Flash-Next, with no figures for other models on integrated graphics.