# Kernel Fusion

6 related articles

Subscribe to updates

vLLM Blog Nov 12, 2026
vLLM serves Thinking Machines' Inkling 2 Small on day one: 594B total parameters, 17B active, about 630 tok/s for a single user
Thinking Machines' Inkling 2 Small gained vLLM support on its release day. It is a text-and-image-in, text-out mixture-of-experts model with 594B total and 17B active parameters, up to 1M tokens of context, and a bundled DSpark speculative-decoding drafter. vLLM fits it with kernel fusion, programmatic dependent launch, sequence parallelism and a Gluon attention kernel rewritten for the 6:1 GQA ratio, reaching about 630 tok/s for a single user at TP4 on GB200 with acceptance length 3.5, and also supports LoRA and reinforcement learning via SkyRL.
Thinking Machines' Inkling 2 Small gained vLLM support on its release day. It is a text-and-image-in, text-out mixture-of-experts model with 594B total and 17B active parameters, up to 1M tokens of context, and a bundled DSpark speculative-decoding drafter. vLLM fits it with kernel fusion, programmatic dependent launch, sequence parallelism and a Gluon attention kernel rewritten for the 6:1 GQA ratio, reaching about 630 tok/s for a single user at TP4 on GB200 with acceptance length 3.5, and also supports LoRA and reinforcement learning via SkyRL.
vLLM
Inkling 2 Small Kernel Fusion DSpark NVFP4
Engineering at Meta Nov 11, 2026
Meta moves GPU execution control into Triton: TLX lets production training and inference orchestrate warp groups directly
Meta released Triton Low-Level Extensions (TLX), which lets engineers explicitly orchestrate warp groups, asynchronous data movement and cluster execution while keeping Triton's programming model. Modern GPU features such as warp specialization, tensor memory and cluster execution were previously either hidden behind compiler abstractions or required large amounts of architecture-specific CUDA code. TLX is already in production across Meta's recommendation training, recommendation inference and large-scale LLM pretraining, where GEM training reached 20-25% end-to-end MFU on several thousand NVIDIA Blackwell GPUs.
Meta released Triton Low-Level Extensions (TLX), which lets engineers explicitly orchestrate warp groups, asynchronous data movement and cluster execution while keeping Triton's programming model. Modern GPU features such as warp specialization, tensor memory and cluster execution were previously either hidden behind compiler abstractions or required large amounts of architecture-specific CUDA code. TLX is already in production across Meta's recommendation training, recommendation inference and large-scale LLM pretraining, where GEM training reached 20-25% end-to-end MFU on several thousand NVIDIA Blackwell GPUs.
Triton Low-Level Extensions (TLX)Multiple Instructions, Multiple Warps (MIMW)Kernel Optimization Kernel Fusion
LMSYS Org Oct 15, 2026
An 816B open-weight multimodal model runs on day one: SGLang and Miles wire up inference and RL backends together
Meta released Muse Spark Open, an open-weight multimodal mixture-of-experts model, and SGLang and Miles added inference and reinforcement-learning training backends on launch day. Its text backbone has 816B parameters with about 42B active per token, accepting text, image, video and audio with a 1M-token context. On eight B300 GPUs with NVFP4 weights, an OpenHands multi-turn agentic coding replay reaches 80.3K total tokens/s per GPU at 61 tokens/s per user at concurrency 128.
Meta released Muse Spark Open, an open-weight multimodal mixture-of-experts model, and SGLang and Miles added inference and reinforcement-learning training backends on launch day. Its text backbone has 816B parameters with about 42B active per token, accepting text, image, video and audio with a 1M-token context. On eight B300 GPUs with NVFP4 weights, an OpenHands multi-turn agentic coding replay reaches 80.3K total tokens/s per GPU at 61 tokens/s per user at concurrency 128.
NVIDIA
Muse Spark Open SGLang Speculative Decoding NVFP4 Quantization Mixture-of-Experts
Baseten Oct 13, 2026
Voice agents stop waiting for the full sentence: Gradium hits 214 ms to first audio and 2–3x faster cold starts
Gradium TTS pushes time-to-first-audio below typical human speech response, reaching a median TTFA of 214 ms on the Coval benchmark as of September 2026 with an interquartile range tight enough for developers to set reliable latency budgets for the rest of the pipeline. It uses Kyutai's Delayed Streams Modeling, which treats text and audio as two parallel streams on a shared clock so audio can start before the upstream model finishes the sentence. On Baseten it ships as a dedicated deployment whose weights are pre-mirrored and cached, cutting cold starts by 2–3x.
Gradium TTS pushes time-to-first-audio below typical human speech response, reaching a median TTFA of 214 ms on the Coval benchmark as of September 2026 with an interquartile range tight enough for developers to set reliable latency budgets for the rest of the pipeline. It uses Kyutai's Delayed Streams Modeling, which treats text and audio as two parallel streams on a shared clock so audio can start before the upstream model finishes the sentence. On Baseten it ships as a dedicated deployment whose weights are pre-mirrored and cached, cutting cold starts by 2–3x.
Baseten
Gradium TTS Delayed Streams Modeling Text-to-Speech Kernel Fusion Voice agents
arXiv Oct 7, 2026
Cleave decouples algebraic search from operator scheduling to generate fused kernels up to 2.8x faster than the best baseline on LLM subgraphs
Cleave is an ML compiler built on symbolic decoupling: it discovers algebraic transformations by superoptimizing over graphs with symbolic shapes, then schedules each resulting graph on concrete shapes, fusing graphs with multiple reductions through iterative tiling and horizontal fusion; on common LLM subgraphs it generates kernels up to 2.8x faster than the best baseline (1.6x on average) and cuts compilation time by 5.9x on average versus Mirage.
Cleave is an ML compiler built on symbolic decoupling: it discovers algebraic transformations by superoptimizing over graphs with symbolic shapes, then schedules each resulting graph on concrete shapes, fusing graphs with multiple reductions through iterative tiling and horizontal fusion; on common LLM subgraphs it generates kernels up to 2.8x faster than the best baseline (1.6x on average) and cuts compilation time by 5.9x on average versus Mirage.
New York University Cornell University
David Pissarra（First author）
Cleave Symbolic Decoupling Operator Scheduling Kernel Fusion Split-K
arXiv Sep 29, 2026
SMAT decomposes merging into Scale, Mask and Perturb simulations, lifting the five-merger average by 1.07–2.16 points across four backbones with under 2% training overhead
The work introduces SMAT (Simple MAT), which from a single expert's perspective describes common model-merging methods through three operations—Scale (reweighting its own update), Mask (removing selected coordinates) and Perturb (adding other experts' updates)—and during training samples scaling coefficients, masks and additive noise to simulate merged parameters while jointly optimizing the expert loss and the expected loss at simulated merged parameters, made efficient by periodic scheduling, kernel fusion and parameter storage switching with one forward and one backward pass per step; across four backbones (Llama-3.2-1B-Instruct, Llama-3.1-8B-Instruct, CLIP ViT-B/32 and ViT-L/14), SMAT improves the mean score over five merging methods by 1.07–2.
The work introduces SMAT (Simple MAT), which from a single expert's perspective describes common model-merging methods through three operations—Scale (reweighting its own update), Mask (removing selected coordinates) and Perturb (adding other experts' updates)—and during training samples scaling coefficients, masks and additive noise to simulate merged parameters while jointly optimizing the expert loss and the expected loss at simulated merged parameters, made efficient by periodic scheduling, kernel fusion and parameter storage switching with one forward and one backward pass per step; across four backbones (Llama-3.2-1B-Instruct, Llama-3.1-8B-Instruct, CLIP ViT-B/32 and ViT-L/14), SMAT improves the mean score over five merging methods by 1.07–2.
Hong Kong Polytechnic University Chinese University of Hong Kong The Hong Kong University of Science and Technology (Guangzhou)
Yanggan Gu（First author）
SMAT Merge-Aware Training Model merging Kernel Fusion

You’re all caught up

vLLM serves Thinking Machines' Inkling 2 Small on day one: 594B total parameters, 17B active, about 630 tok/s for a single user

Meta moves GPU execution control into Triton: TLX lets production training and inference orchestrate warp groups directly

An 816B open-weight multimodal model runs on day one: SGLang and Miles wire up inference and RL backends together

Voice agents stop waiting for the full sentence: Gradium hits 214 ms to first audio and 2–3x faster cold starts

Cleave decouples algebraic search from operator scheduling to generate fused kernels up to 2.8x faster than the best baseline on LLM subgraphs

SMAT decomposes merging into Scale, Mask and Perturb simulations, lifting the five-merger average by 1.07–2.16 points across four backbones with under 2% training overhead