# FreeToken

FlashML released FreeToken, an edge-native Mixture-of-Experts serving engine aimed at running frontier open-weight MoE models above 290B parameters locally on personal and consumer hardware. It treats GPUs, CPUs, host memory and interconnects as one elastic inference platform, organizing compute and data movement through bandwidth-adaptive CPU–GPU co-execution, full-layer double-buffered prefill streaming and a global LRU expert cache. The engine also adds semantic anchor checkpoints to skip recomputation after agentic context edits and reallocates VRAM at runtime between expert caches and KV memory, covering models such as DeepSeek-V4-Flash, Qwen3.6-35B-A3B and GLM-5.2 across MXFP4, NVFP4, FP8 and BF16 formats.

FlashML released FreeToken, an edge-native Mixture-of-Experts serving engine aimed at running frontier open-weight MoE models above 290B parameters locally on personal and consumer hardware. It treats GPUs, CPUs, host memory and interconnects as one elastic inference platform, organizing compute and data movement through bandwidth-adaptive CPU–GPU co-execution, full-layer double-buffered prefill streaming and a global LRU expert cache. The engine also adds semantic anchor checkpoints to skip recomputation after agentic context edits and reallocates VRAM at runtime between expert caches and KV memory, covering models such as DeepSeek-V4-Flash, Qwen3.6-35B-A3B and GLM-5.2 across MXFP4, NVFP4, FP8 and BF16 formats.