Real-time inference stops ceding ground to gradient-boosted trees: Liquid's RIPL returns full generated responses in under 50 ms
Synopsis
Liquid AI released RIPL, a low-latency inference engine that puts Liquid Foundation Models directly in the path of a live request and returns complete generated responses end-to-end in under 50 ms while traffic keeps arriving at peak load. Most serving infrastructure targets general-purpose models, large batches and long-running agentic workflows where seconds per request are acceptable, while enterprise real-time workloads such as search ranking, recommendations and fraud decisions need answers in milliseconds and have relied on faster but less accurate methods like gradient-boosted trees or small embedding models. RIPL is aimed at changing that trade-off and is already running on Shopify production workloads including search enhancement.
Interpretation
Real-time enterprise workloads can now place a generative model directly in the request path instead of falling back to faster but less accurate methods such as gradient-boosted trees or small embedding models. Most serving infrastructure is built for general-purpose models, large batches and long-running agentic workflows where a few seconds per request is acceptable, whereas enterprise real-time workloads have far smaller latency budgets. Across two real-time workloads, LFM2.5-1.2B on RIPL has 14.4 to 28.4 times lower p95 latency than Qwen3.5-0.8B on vLLM, and LFM2.5-2.6B has 5.6 to 7.7 times lower p95 latency than Qwen3.5-2B.
The engine holds low latency while traffic arrives continuously at peak load, and it returns complete generated responses rather than partial output. RIPL is an inference engine built for real-time constraints, in contrast to general serving infrastructure aimed at large batches and long-running workflows. On Shopify production workloads, LFMs achieve sub-20 ms inference in search enhancement and use about 50% fewer parameters while delivering 2 to 10 times faster inference without compromising quality.
Perspective
The scope is Liquid AI's RIPL engine and its behavior on real-time enterprise workloads: complete generated responses end-to-end in under 50 ms, sustained while traffic arrives continuously at peak load. The comparison is against Qwen3.5 models running on vLLM, and the production case comes from Shopify latency-sensitive workloads such as search enhancement. RIPL is available for enterprise deployments of LFMs on NVIDIA GPUs including H100, H200, B200 and RTX PRO 6000.
The latency ratios come from p95 comparisons on two real-time workloads, and the workload composition and request shapes are not detailed. The sub-20 ms and roughly 50% fewer parameters results at Shopify are limited to production workloads such as search enhancement. RIPL currently targets enterprise deployments of LFMs on the listed NVIDIA GPUs.