Zihao Qi

Large vision-language models process hundreds to thousands of visual tokens per image, and inference cost rises with them. Earlier training-free pruning methods ranked all visual tokens by one importance criterion, overlooking that tokens take on different roles as encoding proceeds. BtsPruner locates the role transition from changes in attention concentration, then picks globally referenced representative tokens from later layers and recursively retrieves local detail tokens from earlier layers, preserving both roles under a fixed budget; on LLaVA-1.5-7B it keeps only 32 of 576 visual tokens (5.6%) and retains 91.6% of the dense model's performance.

Large vision-language models process hundreds to thousands of visual tokens per image, and inference cost rises with them. Earlier training-free pruning methods ranked all visual tokens by one importance criterion, overlooking that tokens take on different roles as encoding proceeds. BtsPruner locates the role transition from changes in attention concentration, then picks globally referenced representative tokens from later layers and recursively retrieves local detail tokens from earlier layers, preserving both roles under a fixed budget; on LLaVA-1.5-7B it keeps only 32 of 576 visual tokens (5.6%) and retains 91.6% of the dense model's performance.