Tanzeel Abbas

Small language models for clinical decision support often answer at far greater length than the question requires after fine-tuning, and existing length control relies on preference data, reward models or proprietary teachers. PECA adds a differentiable conciseness term to the QLoRA supervised fine-tuning objective: once generation passes a token budget, it raises the end-of-sequence probability, with no preference data, no auxiliary model and a single commodity GPU. On the 1.7-billion-parameter Qwen3-1.7B-Instruct, PECA reduces mean response length from 256.0 to 69.7 tokens, a 72.8% cut, while raising MedMCQA multiple-choice diagnostic accuracy from 40.8% to 44.0%.

Small language models for clinical decision support often answer at far greater length than the question requires after fine-tuning, and existing length control relies on preference data, reward models or proprietary teachers. PECA adds a differentiable conciseness term to the QLoRA supervised fine-tuning objective: once generation passes a token budget, it raises the end-of-sequence probability, with no preference data, no auxiliary model and a single commodity GPU. On the 1.7-billion-parameter Qwen3-1.7B-Instruct, PECA reduces mean response length from 256.0 to 69.7 tokens, a 72.8% cut, while raising MedMCQA multiple-choice diagnostic accuracy from 40.8% to 44.0%.