Skip to main content

# Activated LoRA

2 related articles

Subscribe to updates

arXiv Oct 30, 2026
One 9B backbone reranks and answers: Glance lifts HotpotQA answer F1 by 0.045 over cross-encoder-selected context
Retrieval-augmented generation usually hands reranking and answering to two models, so candidate passages are processed twice. Glance adds a lightweight decision adapter and pointer head to the generator, prefills the candidates once, and reads one short probe per passage for a relevance score. After 39 minutes of MS MARCO fine-tuning it reaches nDCG@10 of 0.741 on SciFact and 0.345 on NFCorpus, against 0.721 and 0.326 for bge-reranker-v2-m3; on 300 HotpotQA questions the same base generator answers with F1 0.045 higher from its selected context than from cross-encoder-selected context.
Retrieval-augmented generation usually hands reranking and answering to two models, so candidate passages are processed twice. Glance adds a lightweight decision adapter and pointer head to the generator, prefills the candidates once, and reads one short probe per passage for a relevance score. After 39 minutes of MS MARCO fine-tuning it reaches nDCG@10 of 0.741 on SciFact and 0.345 on NFCorpus, against 0.721 and 0.326 for bge-reranker-v2-m3; on 300 HotpotQA questions the same base generator answers with F1 0.045 higher from its selected context than from cross-encoder-selected context.
Adilet Metinov（First author）
Glance Passage-specific relevance probes Retrieval-Augmented Generation Activated LoRA LoRA
Hugging Face - Blog Oct 16, 2026
IBM Puts Chat, Decisions and Speech Behind One Endpoint: Three Adapters Share a KV Cache, VRAM Barely Moves
IBM released Granite Switch 4.1, which composes general decision-making, safety decision-making and speech transcription into a single hosted endpoint, with 8B and 30B versions occupying roughly the same GPU VRAM as the base model alone. The earlier Granite Switch project already allowed users to compose specialized adapters into custom models, while this release ships a Decision adapter, a Guardian safety adapter and Granite-Speech-5.0-Turbo-CTC precompiled into the model. Because the adapters are built as cache-efficient Activated LoRAs, the KV cache produced by a base-model generation can be reused for later adapter-based decisions, so switching between chat, decision and spoken requests does not rebuild the cache.
IBM released Granite Switch 4.1, which composes general decision-making, safety decision-making and speech transcription into a single hosted endpoint, with 8B and 30B versions occupying roughly the same GPU VRAM as the base model alone. The earlier Granite Switch project already allowed users to compose specialized adapters into custom models, while this release ships a Decision adapter, a Guardian safety adapter and Granite-Speech-5.0-Turbo-CTC precompiled into the model. Because the adapters are built as cache-efficient Activated LoRAs, the KV cache produced by a base-model generation can be reused for later adapter-based decisions, so switching between chat, decision and spoken requests does not rebuild the cache.
Granite Switch 4.1 Activated LoRA speech-to-text

You’re all caught up