Beijing Normal-Hong Kong Baptist University

Non-prefix KV-cache reuse encodes retrieved chunks separately and concatenates their caches, losing 19% to 100% of context-dependent questions that full prefill answers, with same-kind distractor chunks adding to the loss. Comparing whether a small repair budget should go to distractors or to the answer-bearing chunk, the study finds that restoring all distractors costs nearly a full prefill, while recomputing the answer-bearing chunk beats token-level selectors at a 5% budget in all nine open-domain QA and API-documentation settings.

Non-prefix KV-cache reuse encodes retrieved chunks separately and concatenates their caches, losing 19% to 100% of context-dependent questions that full prefill answers, with same-kind distractor chunks adding to the loss. Comparing whether a small repair budget should go to distractors or to the answer-bearing chunk, the study finds that restoring all distractors costs nearly a full prefill, while recomputing the answer-bearing chunk beats token-level selectors at a 5% budget in all nine open-domain QA and API-documentation settings.

Target speaker extraction for multilingual overlapping speech no longer applies one fixed computation depth to every input. Fixed-depth frontends either under-process hard inputs or waste computation on inputs that have already converged. LATTE uses a difficulty estimator to route each mixture through two, four, or six shared extractor blocks, reaching 15.07 dB macro SI-SNRi at 0.30 GFLOPs on the eight-language PolyExtract benchmark, within 0.09 dB of a 0.53-GFLOP fixed-depth path while cutting computation by 43.4%; before frozen Qwen3-Omni it lowers average WER from TF-GridNet's 16.8% to 14.6%.

Target speaker extraction for multilingual overlapping speech no longer applies one fixed computation depth to every input. Fixed-depth frontends either under-process hard inputs or waste computation on inputs that have already converged. LATTE uses a difficulty estimator to route each mixture through two, four, or six shared extractor blocks, reaching 15.07 dB macro SI-SNRi at 0.30 GFLOPs on the eight-language PolyExtract benchmark, within 0.09 dB of a 0.53-GFLOP fixed-depth path while cutting computation by 43.4%; before frozen Qwen3-Omni it lowers average WER from TF-GridNet's 16.8% to 14.6%.

This work presents a unified theoretical analysis of sparse attention: it proves that the Jensen–Shannon divergence between full and sparse attention is determined solely by the discarded truncation mass with a closed-form expression, uses order statistics under i.i.d. sub-Gaussian attention-score assumptions to show that the high-probability lower bound on truncation mass increases with sequence length, and derives sparsity-tightened generalization bounds via Rademacher complexity and a Xu–Raginsky mutual-information bound with entropy and covering-number analysis, verifying the closed-form divergence and the growth of information loss on GPT-2 (124M) with WikiText-103.

This work presents a unified theoretical analysis of sparse attention: it proves that the Jensen–Shannon divergence between full and sparse attention is determined solely by the discarded truncation mass with a closed-form expression, uses order statistics under i.i.d. sub-Gaussian attention-score assumptions to show that the high-probability lower bound on truncation mass increases with sequence length, and derives sparsity-tightened generalization bounds via Rademacher complexity and a Xu–Raginsky mutual-information bound with entropy and covering-number analysis, verifying the closed-form divergence and the growth of information loss on GPT-2 (124M) with WikiText-103.