# Adaptive Computation

Medical and astronomical images are orders of magnitude larger than natural images, yet tumors and galaxies occupy a tiny fraction of the data, and uniform compression such as quantization or pruning can erode precision exactly there. Conformal Adaptation first screens candidates with a low-resolution model, calibrates the selection threshold with conformal risk control so the discarded cells carry at most a user-set expected fraction of true signal mass, then refines only the selected regions. Across brain-tumor segmentation, head-and-neck segmentation, calcium spike-rate inference and galaxy flux super-resolution, the pipeline cuts computation while keeping accuracy close, and VISTA3D reaches an 82.10x FLOPs speedup in CPU-only inference.

Medical and astronomical images are orders of magnitude larger than natural images, yet tumors and galaxies occupy a tiny fraction of the data, and uniform compression such as quantization or pruning can erode precision exactly there. Conformal Adaptation first screens candidates with a low-resolution model, calibrates the selection threshold with conformal risk control so the discarded cells carry at most a user-set expected fraction of true signal mass, then refines only the selected regions. Across brain-tumor segmentation, head-and-neck segmentation, calcium spike-rate inference and galaxy flux super-resolution, the pipeline cuts computation while keeping accuracy close, and VISTA3D reaches an 82.10x FLOPs speedup in CPU-only inference.

Target speaker extraction for multilingual overlapping speech no longer applies one fixed computation depth to every input. Fixed-depth frontends either under-process hard inputs or waste computation on inputs that have already converged. LATTE uses a difficulty estimator to route each mixture through two, four, or six shared extractor blocks, reaching 15.07 dB macro SI-SNRi at 0.30 GFLOPs on the eight-language PolyExtract benchmark, within 0.09 dB of a 0.53-GFLOP fixed-depth path while cutting computation by 43.4%; before frozen Qwen3-Omni it lowers average WER from TF-GridNet's 16.8% to 14.6%.

Target speaker extraction for multilingual overlapping speech no longer applies one fixed computation depth to every input. Fixed-depth frontends either under-process hard inputs or waste computation on inputs that have already converged. LATTE uses a difficulty estimator to route each mixture through two, four, or six shared extractor blocks, reaching 15.07 dB macro SI-SNRi at 0.30 GFLOPs on the eight-language PolyExtract benchmark, within 0.09 dB of a 0.53-GFLOP fixed-depth path while cutting computation by 43.4%; before frozen Qwen3-Omni it lowers average WER from TF-GridNet's 16.8% to 14.6%.

Low-rank compression previously assigned a fixed rank during inference, so every token cost the same compute. LRCC adds one lightweight router per Transformer block that selects among a few nested low-rank paths for each token, training only the routers while the low-rank factors stay frozen. Within the same average active-parameter budget, Llama-2-7B gains 7.6 percentage points in average downstream accuracy over static low-rank compression; at matched batch-size-1 decoding latency, Llama-3.2-1B improves both perplexity and downstream accuracy and Llama-2-7B stays competitive, without specialized CUDA kernels.

Low-rank compression previously assigned a fixed rank during inference, so every token cost the same compute. LRCC adds one lightweight router per Transformer block that selects among a few nested low-rank paths for each token, training only the routers while the low-rank factors stay frozen. Within the same average active-parameter budget, Llama-2-7B gains 7.6 percentage points in average downstream accuracy over static low-rank compression; at matched batch-size-1 decoding latency, Llama-3.2-1B improves both perplexity and downstream accuracy and Llama-2-7B stays competitive, without specialized CUDA kernels.

Using paired enabled/bypassed executions of the final attention layer to produce signed next-token loss differences in frozen SmolLM3-3B-Base and Qwen3.5-4B-Base, the study tests whether hidden-state extrapolation error, curvature and error change improve attention routing beyond uncertainty, one-step displacement, position and state projections; on 100 held-out PG-19 books at an identical causal 20% invocation quota, none of six prespecified comparisons shows a positive gain after Holm correction, and in Qwen3.5 a parameter-matched fixed-projection control lowers NLL by 0.00356 nats/token relative to the trajectory router.

Using paired enabled/bypassed executions of the final attention layer to produce signed next-token loss differences in frozen SmolLM3-3B-Base and Qwen3.5-4B-Base, the study tests whether hidden-state extrapolation error, curvature and error change improve attention routing beyond uncertainty, one-step displacement, position and state projections; on 100 held-out PG-19 books at an identical causal 20% invocation quota, none of six prespecified comparisons shows a positive gain after Holm correction, and in Qwen3.5 a parameter-matched fixed-projection control lowers NLL by 0.00356 nats/token relative to the trajectory router.

The authors present BranchIP (Branch Interatomic Potential), a single-model framework for learned adaptive tensor-product computation trained with a novel distillation loss; in experiments on two systems of physical interest, a heterogeneous catalysis system and a proton-conducting solid acid electrolyte, it accelerates equivariant machine learning interatomic potentials across model sizes by up to 2.4x while reducing memory usage by up to 2.6x, maintains physical fidelity, and uses the learned adaptive computation to reveal which interactions demand deeper computation and how computational depth relates to chemical complexity and dynamics.

The authors present BranchIP (Branch Interatomic Potential), a single-model framework for learned adaptive tensor-product computation trained with a novel distillation loss; in experiments on two systems of physical interest, a heterogeneous catalysis system and a proton-conducting solid acid electrolyte, it accelerates equivariant machine learning interatomic potentials across model sizes by up to 2.4x while reducing memory usage by up to 2.6x, maintains physical fidelity, and uses the learned adaptive computation to reveal which interactions demand deeper computation and how computational depth relates to chemical complexity and dynamics.