Instruction tuning on 0.5% of the data: QSGS cuts redundancy with a graph cut and matches full-data Mistral-7B
For targeted instruction-tuning data selection, QSGS organizes candidate examples by gradient similarity into a graph and applies a submodular graph cut that pulls the subset toward the target task while penalizing internal redundancy, then adds gradient-guided synthesis to fill directions the pool cannot cover. On MMLU, BBH and TydiQA across Llama-2, Llama-3 and Mistral, QSGS at a 0.5% budget (1,350 of 270,000 examples) matches or exceeds full-data performance on Mistral-7B and beats the strongest baseline on every benchmark.