IBM Puts Chat, Decisions and Speech Behind One Endpoint: Three Adapters Share a KV Cache, VRAM Barely Moves
Synopsis
IBM released Granite Switch 4.1, which composes general decision-making, safety decision-making and speech transcription into a single hosted endpoint, with 8B and 30B versions occupying roughly the same GPU VRAM as the base model alone. The earlier Granite Switch project already allowed users to compose specialized adapters into custom models, while this release ships a Decision adapter, a Guardian safety adapter and Granite-Speech-5.0-Turbo-CTC precompiled into the model. Because the adapters are built as cache-efficient Activated LoRAs, the KV cache produced by a base-model generation can be reused for later adapter-based decisions, so switching between chat, decision and spoken requests does not rebuild the cache.
Interpretation
One hosted endpoint now serves natural-language chat, general and specialized decisions, and speech-to-text, so users no longer need a separate deployment per task type. The earlier Granite Switch project offered a modular development path in which users picked and composed specialized adapters from Granite Libraries to build custom models themselves. The 8B and 30B versions ship precompiled with the Decision adapter, the Guardian adapter and Granite-Speech-5.0-Turbo-CTC, and the three combined with the base model take up approximately the same GPU VRAM as the base model alone.
On safety-related decisions, the specialized adapter outperforms several general decision models on average. Closed decision models such as Jev focus on generality, while specialized adapters can do better on specialized classification tasks. The Guardian adapter averages higher than Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions and Clef-Flash on safety-related decision tasks.
Adapters share the base model's KV cache, making the cost of adding a specialist close to nothing. When each task type keeps its own separate model, adding capability usually means more GPUs. The Switch adapters are built as cache-efficient Activated LoRAs, so the KV cache from a base-model generation is reusable for subsequent adapter-based decisions, combined with vLLM contributions for low-rank adapter inference and a co-located ASR encoder.
Perspective
The demonstrated scope is the two precompiled 8B and 30B versions running together on a single hosted endpoint, plus a Speech-to-Decisions demo space where granite-switch-4.1-8b transcribes audio, generates summaries, screens for harmful content and tests 15 other user-defined flags. IBM says more details and data on Granite Switch will be released in the coming weeks, and plans an Adapter SDK for training adapters in Mellea along with tutorials.
IBM says more details and data will follow in the coming weeks, so the evaluation information currently checkable is limited. The Guardian adapter's advantage over general decision models is stated as an average, without a per-task distribution. The comparison between the 3B-parameter Granite Switch model making 60 to 100 decisions per second and Jev's reported 10 decisions per second is sourced to Jev's release blog.