# Instruction Tuning

For targeted instruction-tuning data selection, QSGS organizes candidate examples by gradient similarity into a graph and applies a submodular graph cut that pulls the subset toward the target task while penalizing internal redundancy, then adds gradient-guided synthesis to fill directions the pool cannot cover. On MMLU, BBH and TydiQA across Llama-2, Llama-3 and Mistral, QSGS at a 0.5% budget (1,350 of 270,000 examples) matches or exceeds full-data performance on Mistral-7B and beats the strongest baseline on every benchmark.

For targeted instruction-tuning data selection, QSGS organizes candidate examples by gradient similarity into a graph and applies a submodular graph cut that pulls the subset toward the target task while penalizing internal redundancy, then adds gradient-guided synthesis to fill directions the pool cannot cover. On MMLU, BBH and TydiQA across Llama-2, Llama-3 and Mistral, QSGS at a 0.5% budget (1,350 of 270,000 examples) matches or exceeds full-data performance on Mistral-7B and beats the strongest baseline on every benchmark.

TCM language models have been trained on under 1GB of text and handle text alone, while TCM diagnosis itself draws on looking, listening, smelling and pulse-taking. ShizhenGPT pairs more than 100GB of text from 3,256 TCM books and web sources with over 200GB of multimodal data, including 1.2M annotated images, 200+ hours of audio and pulse, smell and ECG signals, through two-stage pretraining and instruction tuning. On five 2024–2025 TCM exams ShizhenGPT-32B averages 78.1 against 84.0 for DeepSeek-R1-671B, and on a 7,204-question TCM vision benchmark it scores 63.6 against 61.4 for Gemini-3-Pro.

TCM language models have been trained on under 1GB of text and handle text alone, while TCM diagnosis itself draws on looking, listening, smelling and pulse-taking. ShizhenGPT pairs more than 100GB of text from 3,256 TCM books and web sources with over 200GB of multimodal data, including 1.2M annotated images, 200+ hours of audio and pulse, smell and ECG signals, through two-stage pretraining and instruction tuning. On five 2024–2025 TCM exams ShizhenGPT-32B averages 78.1 against 84.0 for DeepSeek-R1-671B, and on a 7,204-question TCM vision benchmark it scores 63.6 against 61.4 for Gemini-3-Pro.

AutoAdapt is a modular instruction-tuning framework that first discovers latent domains with three methods (K-means, BERTopic, MNLI), then trains one LoRA adapter per domain independently and in parallel, and routes at inference with a parameter-free centroid mechanism; across 14 domain-specific benchmarks and GPT-4o pairwise judgements it reaches parity with a single LoRA adapter trained on all domains, while adding or updating a domain requires training only one adapter instead of full-model retraining.

AutoAdapt is a modular instruction-tuning framework that first discovers latent domains with three methods (K-means, BERTopic, MNLI), then trains one LoRA adapter per domain independently and in parallel, and routes at inference with a parameter-free centroid mechanism; across 14 domain-specific benchmarks and GPT-4o pairwise judgements it reaches parity with a single LoRA adapter trained on all domains, while adding or updating a domain requires training only one adapter instead of full-model retraining.

The work reports that directly incorporating a selection network into existing meta-learning for training-data selection (MTS) objectives causes unstable optimization and poor generalization due to weight suppression and persistent reliance on easy-to-learn features, and proposes Transferable Example Scoring and Selection (TESS), a scalable data-selection framework built on a Pointwise Value Matching (PVM) objective, which in experiments on LLM safety and targeted instruction tuning demonstrates strong transfer across datasets, from subsets to full corpora, and from smaller to larger models.

The work reports that directly incorporating a selection network into existing meta-learning for training-data selection (MTS) objectives causes unstable optimization and poor generalization due to weight suppression and persistent reliance on easy-to-learn features, and proposes Transferable Example Scoring and Selection (TESS), a scalable data-selection framework built on a Pointwise Value Matching (PVM) objective, which in experiments on LLM safety and targeted instruction tuning demonstrates strong transfer across datasets, from subsets to full corpora, and from smaller to larger models.

The work proposes MITE, which reformulates biomedical named entity recognition as a structure-to-structure generation task by rendering each training instance in multiple programming-language formats such as Python, C++, and Java to supply structurally diverse supervision without external biomedical knowledge or extra annotations, and at inference aggregates predictions from the different code formats through entity-level voting; on six widely used BioNER datasets MITE consistently outperforms representative BERT-based and LLM-based baselines and shows strong cross-dataset generalization, with ablation and parameter analyses further supporting the effectiveness and robustness of the proposed components.

The work proposes MITE, which reformulates biomedical named entity recognition as a structure-to-structure generation task by rendering each training instance in multiple programming-language formats such as Python, C++, and Java to supply structurally diverse supervision without external biomedical knowledge or extra annotations, and at inference aggregates predictions from the different code formats through entity-level voting; on six widely used BioNER datasets MITE consistently outperforms representative BERT-based and LLM-based baselines and shows strong cross-dataset generalization, with ablation and parameter analyses further supporting the effectiveness and robustness of the proposed components.