Instruction tuning on 0.5% of the data: QSGS cuts redundancy with a graph cut and matches full-data Mistral-7B
Synopsis
For targeted instruction-tuning data selection, QSGS organizes candidate examples by gradient similarity into a graph and applies a submodular graph cut that pulls the subset toward the target task while penalizing internal redundancy, then adds gradient-guided synthesis to fill directions the pool cannot cover. On MMLU, BBH and TydiQA across Llama-2, Llama-3 and Mistral, QSGS at a 0.5% budget (1,350 of 270,000 examples) matches or exceeds full-data performance on Mistral-7B and beats the strongest baseline on every benchmark.
Interpretation
Instruction-tuning data selection moves from scoring examples one at a time to scoring a whole subset, so the chosen examples must both fit the target task and avoid repeating each other. Earlier methods such as LESS compare each candidate against the target set independently and keep the top-ranked examples, leaving overlap among candidates out of the score. At a 0.5% budget, QSGS averages 1.6% above LESS across the nine model-benchmark combinations, with gains in every one; on Llama-2-7B it leads by 2.2% on average.
The construction happens in gradient space: each training example is represented by low-rank gradients from several checkpoints of a warm-up model, and cosine similarity between examples becomes an edge weight. The objective adds a term for each example's cumulative influence on the target validation set to a penalty on pairwise similarity inside the subset, and a greedy procedure adds the example with the largest marginal gain at each step. In an ablation on Llama-2-7B and MMLU at the 0.5% budget, replacing top-score selection with the graph cut raises accuracy from 45.9 to 47.4, and adding gradient-guided synthesis gives the full method.
Perspective
The demonstrated scope is open-weight models of 7B to 8B parameters, the MMLU, BBH and TydiQA benchmarks, and data budgets from 0.5% to 5%; for Llama-3 the training set is extended to a combination of 17 datasets following prior practice. The authors propose extending evaluation to larger models, more domains, and multi-turn or tool-use tasks as a next direction.
The quality of the gradient similarity graph depends on the fidelity of the warm-up checkpoints and projected gradient features; very small warm-up sets produce noisy influence estimates, and a 0.5% warm-up ratio falls below even random selection. Synthetic examples are currently chosen only by gradient-residual alignment, with no semantic or correctness filter, and evaluation covers only 7B to 8B models and three benchmarks.