Clinical small models learn when to stop: PECA cuts mean answers 72.8% and lifts MedMCQA accuracy to 44.0%

After domain fine-tuning, small language models in clinical settings answer far longer than the question warrants, and even sound reasoning becomes hard to act on at the bedside when the output runs long. Earlier parameter-efficient fine-tuning methods such as LoRA and QLoRA shrink the trainable parameter set but do not constrain output length, while length-control methods mostly sit at the preference-optimisation stage and need paired preference data, a learned reward model or distillation from a proprietary teacher. On the same 1.7B-parameter backbone, the same medical chain-of-thought data and the same QLoRA configuration, changing only the training objective and adding the conciseness term reduces mean response length from 256.0 to 69.7 tokens, a 72.8% reduction.

Conciseness is implemented through a differentiable end-of-sequence probability term, so the model learns during training to stop near the budget rather than being clipped at a fixed ceiling at inference. A penalty written directly over the number of generated tokens cannot work, because the token count is discrete and its gradient with respect to the parameters is zero almost everywhere, leaving generation length essentially unchanged. PECA raises the end-of-sequence probability at every response position beyond a 64-token budget, and the term is added to the cross-entropy loss inside the ordinary supervised training loop; across the 1,000 test items, every PECA setting shifts the response-length distribution left and concentrates it below 128 tokens.

Conciseness strength traces an interpretable accuracy-to-length frontier, and only the light setting improves length and accuracy together. Across the conciseness sweep, length changes little between settings while accuracy differs sharply: the light setting reaches 44.0% MedMCQA accuracy, the moderate setting falls to 31.9%, and the strongest setting falls to 4.6%, where most responses are only one or two tokens long. On the 1,000 test items, the light setting averages 69.7 tokens; on 1,000 MedMCQA validation items its accuracy is 44.0%, against 40.8% for vanilla QLoRA supervised fine-tuning; each condition trains in 153 to 163 minutes on a single NVIDIA A10G or L4 GPU with peak memory of 4.4 to 5.3 GB.

The demonstrated scope is medical question answering with Qwen3-1.7B-Instruct fine-tuned by QLoRA on the medical-o1-reasoning-SFT corpus, with response length evaluated on 1,000 test items and diagnostic accuracy on 1,000 MedMCQA validation items, and training on a single NVIDIA A10G or L4 GPU.

The results rest on a single random seed and a single model family, and the absolute MedMCQA figures reflect the capability ceiling of a 1.7B model on that benchmark rather than a fully engineered clinical decision-support system; accuracy drops sharply at high conciseness strength, where most responses at the strongest setting are only one or two tokens long.