On-Premise Detection of a Guideline-Driven Oral Anticoagulation Shift in German Doctors' Letters Using Local Large Language Models
Synopsis
Using an on-premise fine-tuned Llama-3.1-70b medication information extraction pipeline, the study automatically extracted medication information from 538 unannotated routine 2012 doctors' letters and compared them with 500 CARDIO:DE letters from 2020/21 (using gold-standard annotations), finding that the DOAC proportion rose from 16.9% to 59.9% while the VKA proportion fell from 37.7% to 9.9%, that the dominant active ingredient within DOACs shifted from rivaroxaban to apixaban, and that manual review showed remaining errors were mainly linked to generic medication mentions and missing medication-reason relations rather than incorrect extraction.
Interpretation
Under strict data-protection and transparency constraints, an on-premise fine-tuned Llama-3.1-70b medication information extraction pipeline can generalize to unseen older German doctors' letters without additional manual annotations and recover the expected oral anticoagulation treatment shift. Prior medication information extraction often relied on gold-standard annotations or cloud models; this work demonstrates cross-temporal generalization of an on-premise local LLM pipeline to unannotated historical letters. Based on a comparison of 538 routine 2012 letters and 500 CARDIO:DE letters from 2020/21, with manual review of the automatically annotated 2012 letters.
Oral anticoagulation treatment shows a guideline-driven shift: the DOAC proportion rose from 16.9% in 2012 to 59.9% in 2020/21, while the VKA proportion fell from 37.7% to 9.9%. Provides real-world evidence from German cardiology doctors' letters quantifying the magnitude of the VKA-to-DOAC shift. Comparison of two corpora from different time periods, with anticoagulant medications identified via extracted medication mentions and medication-reason relations and classified using guideline-based lexicons.
Within the DOAC class, the dominant active ingredient shifted from rivaroxaban in 2012 to apixaban in 2020/21. Beyond the class-level shift, further characterizes the temporal change in active-ingredient composition within DOACs. Analysis of active-ingredient composition within DOACs based on extraction results.
Manual review showed that remaining errors were mainly linked to generic medication mentions and missing medication-reason relations rather than incorrect medication information extraction itself. Localizes error sources to lexicon matching and missing relations rather than the core recognition capability of the extraction model. Manual review of the automatically annotated 2012 letters.
Perspective
This work targets settings where German clinical routine letters must be processed entirely inside hospital infrastructure under strict data-protection and transparency constraints, and is suited to clinical research teams seeking to use local LLMs for medication information extraction and medication trend analysis; its conclusions pertain to German cardiology doctors' letters and oral anticoagulant drug classes.
The 2012 corpus lacks gold-standard annotation, so its extraction quality depends on the coverage of manual review; remaining errors are linked to generic medication mentions and missing medication-reason relations, and how such issues manifest across different document styles or departments remains to be observed; the pipeline's generalization to other institutions, languages, or drug classes awaits further study.