# LLM Inference Engine

2 related articles

Subscribe to updates

Liquid AI Oct 22, 2026
Real-time inference stops ceding ground to gradient-boosted trees: Liquid's RIPL returns full generated responses in under 50 ms
Liquid AI released RIPL, a low-latency inference engine that puts Liquid Foundation Models directly in the path of a live request and returns complete generated responses end-to-end in under 50 ms while traffic keeps arriving at peak load. Most serving infrastructure targets general-purpose models, large batches and long-running agentic workflows where seconds per request are acceptable, while enterprise real-time workloads such as search ranking, recommendations and fraud decisions need answers in milliseconds and have relied on faster but less accurate methods like gradient-boosted trees or small embedding models. RIPL is aimed at changing that trade-off and is already running on Shopify production workloads including search enhancement.
Liquid AI released RIPL, a low-latency inference engine that puts Liquid Foundation Models directly in the path of a live request and returns complete generated responses end-to-end in under 50 ms while traffic keeps arriving at peak load. Most serving infrastructure targets general-purpose models, large batches and long-running agentic workflows where seconds per request are acceptable, while enterprise real-time workloads such as search ranking, recommendations and fraud decisions need answers in milliseconds and have relied on faster but less accurate methods like gradient-boosted trees or small embedding models. RIPL is aimed at changing that trade-off and is already running on Shopify production workloads including search enhancement.
LLM Inference Engine
lightseekorg/tokenspeed Oct 15, 2026
An inference engine splits control from execution: C++ state machine owns the KV cache, Python stays for researchers
TokenSpeed is an open-source LLM inference engine for agentic workloads, backed by the non-profit LightSeek Foundation, aiming at TensorRT-LLM-level performance with vLLM-level usability in production agentic serving. It separates the scheduling control plane from the execution plane: the control plane is a C++ finite-state machine that uses the type system to enforce safe reuse of request lifecycles and KV cache state at compile time, while the execution plane stays in Python for fast iteration. Kernels are treated as a modular subsystem decoupled from the core engine through a portable public API, a centralized registry, and a plugin mechanism, and the project states it includes one of the fastest MLA implementations on Blackwell for agentic workloads.
TokenSpeed is an open-source LLM inference engine for agentic workloads, backed by the non-profit LightSeek Foundation, aiming at TensorRT-LLM-level performance with vLLM-level usability in production agentic serving. It separates the scheduling control plane from the execution plane: the control plane is a C++ finite-state machine that uses the type system to enforce safe reuse of request lifecycles and KV cache state at compile time, while the execution plane stays in Python for fast iteration. Kernels are treated as a modular subsystem decoupled from the core engine through a portable public API, a centralized registry, and a plugin mechanism, and the project states it includes one of the fastest MLA implementations on Blackwell for agentic workloads.
TokenSpeed LLM Inference Engine LLM Inference

You’re all caught up

Real-time inference stops ceding ground to gradient-boosted trees: Liquid's RIPL returns full generated responses in under 50 ms

An inference engine splits control from execution: C++ state machine owns the KV cache, Python stays for researchers