# Network Pruning

Safety alignment in open-weight models is highly sparse: pruning a small set of safety-critical neurons or ablating a single refusal direction can make a model produce harmful content. ABS-Align gradually prunes those identified safety-critical neurons during preference finetuning, forcing refusal behavior to be re-established through other parts of the network, and restores the pruned neurons afterward. Across Gemma-2-2B-IT, Llama-3-8B-Instruct, Llama-3.2-3B-Instruct and Qwen-3-4B, attack success generally falls against TwinBreak, NeuroStrike, refusal-direction ablation and ten SORRY-Bench black-box attacks, with reductions of about 90% for some attacks, while TruthfulQA improves and general capabilities change little.

Safety alignment in open-weight models is highly sparse: pruning a small set of safety-critical neurons or ablating a single refusal direction can make a model produce harmful content. ABS-Align gradually prunes those identified safety-critical neurons during preference finetuning, forcing refusal behavior to be re-established through other parts of the network, and restores the pruned neurons afterward. Across Gemma-2-2B-IT, Llama-3-8B-Instruct, Llama-3.2-3B-Instruct and Qwen-3-4B, attack success generally falls against TwinBreak, NeuroStrike, refusal-direction ablation and ten SORRY-Bench black-box attacks, with reductions of about 90% for some attacks, while TruthfulQA improves and general capabilities change little.

LLM pruning now builds its calibration activations from the dense model's own autoregressive decode steps while discarding prefill, aligning the pruning objective with generation-time activations; it beats fixed-text calibration on WritingBench and ClassEval and reaches up to 1.48x end-to-end decoding speedup on A100 GPUs.

LLM pruning now builds its calibration activations from the dense model's own autoregressive decode steps while discarding prefill, aligning the pruning objective with generation-time activations; it beats fixed-text calibration on WritingBench and ClassEval and reaches up to 1.48x end-to-end decoding speedup on A100 GPUs.

The work proposes SHarP, which treats tools, instructions, and supporting mechanisms in an agent harness as individually disableable modules, estimates each module's performance and efficiency saliency through single-module ablation, and prunes by an efficiency-oriented or performance-oriented rule; across five settings (OpenHands/GAIA, PaperQA2/LitQA2, LIFE-harness airline and retail, JIT agentfold/DeepPlanning-Shopping), most pruned configurations retain comparable performance, all four harnesses achieve at least a 10% token-cost reduction, and two improve task performance while cutting cost.

The work proposes SHarP, which treats tools, instructions, and supporting mechanisms in an agent harness as individually disableable modules, estimates each module's performance and efficiency saliency through single-module ablation, and prunes by an efficiency-oriented or performance-oriented rule; across five settings (OpenHands/GAIA, PaperQA2/LitQA2, LIFE-harness airline and retail, JIT agentfold/DeepPlanning-Shopping), most pruned configurations retain comparable performance, all four harnesses achieve at least a 10% token-cost reduction, and two improve task performance while cutting cost.