TopK-Guided combines bounded token-level sparsity with sensitivity-aware block budgets, improving perplexity and downstream accuracy on Llama-2 and Llama-3
TopK-Guided is a training-free activation sparsity method that combines bounded token-level sparsity adaptation with sensitivity-aware block-level budget allocation, consistently improving perplexity and downstream accuracy over TEAL and WINA across Llama-2 and Llama-3 models while preserving essentially the same sparsity-dependent projection compute as WINA, with the largest gains at high sparsity; ablations show the two components provide complementary improvements.