Medicine & Health

466 items

PloS one Sep 21, 2026
The study scraped 15,300 HMPV-related comments from YouTube news channels in 2024-2025, retained 9,758 after preprocessing, generated weak sentiment labels with VADER, compared six transformer models (ELECTRA, RoBERTa, ALBERT, DistilBERT, XLNet, BERT), found XLNet best at 93.50% accuracy, and used SHAP's PartitionExplainer to produce word-level attributions on correctly classified samples, showing words such as "flu" and "fear" driving negative predictions, "save", "help" and "mad" associated with neutral, and "falling", "save", "god" and "america" associated with positive.
The study scraped 15,300 HMPV-related comments from YouTube news channels in 2024-2025, retained 9,758 after preprocessing, generated weak sentiment labels with VADER, compared six transformer models (ELECTRA, RoBERTa, ALBERT, DistilBERT, XLNet, BERT), found XLNet best at 93.50% accuracy, and used SHAP's PartitionExplainer to produce word-level attributions on correctly classified samples, showing words such as "flu" and "fear" driving negative predictions, "save", "help" and "mad" associated with neutral, and "falling", "save", "god" and "america" associated with positive.
Explainable AI for Sentiment Analysis of HMPV Using XLNet and SHAP
The study scraped 15,300 HMPV-related comments from YouTube news channels in 2024-2025, retained 9,758 after preprocessing, generated weak sentiment labels with VADER, compared six transformer models (ELECTRA, RoBERTa, ALBERT, DistilBERT, XLNet, BERT), found XLNet best at 93.50% accuracy, and used SHAP's PartitionExplainer to produce word-level attributions on correctly classified samples, showing words such as "flu" and "fear" driving negative predictions, "save", "help" and "mad" associated with neutral, and "falling", "save", "god" and "america" associated with positive.
The study scraped 15,300 HMPV-related comments from YouTube news channels in 2024-2025, retained 9,758 after preprocessing, generated weak sentiment labels with VADER, compared six transformer models (ELECTRA, RoBERTa, ALBERT, DistilBERT, XLNet, BERT), found XLNet best at 93.50% accuracy, and used SHAP's PartitionExplainer to produce word-level attributions on correctly classified samples, showing words such as "flu" and "fear" driving negative predictions, "save", "help" and "mad" associated with neutral, and "falling", "save", "god" and "america" associated with positive.
medRxiv Sep 21, 2026
This study introduces and evaluates a privacy-preserving knowledge distillation framework in which CTGAN-generated synthetic cohorts matching UK Biobank distributions are used to elicit multimorbidity scores from three teacher LLMs (GPT-4o, Gemini, DeepSeek) under zero-shot prompting, and compact student models (CoLLMs) are then trained to mimic those scores, enabling application to real UK Biobank data (N = 439,221) for multimorbidity scoring without exposing patient-level data to third-party APIs, with evaluation against the Charlson (CCI) and Elixhauser (ECI) indices via survival analysis, genome-wide association studies, and polygenic risk score associations.
This study introduces and evaluates a privacy-preserving knowledge distillation framework in which CTGAN-generated synthetic cohorts matching UK Biobank distributions are used to elicit multimorbidity scores from three teacher LLMs (GPT-4o, Gemini, DeepSeek) under zero-shot prompting, and compact student models (CoLLMs) are then trained to mimic those scores, enabling application to real UK Biobank data (N = 439,221) for multimorbidity scoring without exposing patient-level data to third-party APIs, with evaluation against the Charlson (CCI) and Elixhauser (ECI) indices via survival analysis, genome-wide association studies, and polygenic risk score associations.
Privacy-Aware Distillation of Large Language Models for Enhanced Multimorbidity Scoring
This study introduces and evaluates a privacy-preserving knowledge distillation framework in which CTGAN-generated synthetic cohorts matching UK Biobank distributions are used to elicit multimorbidity scores from three teacher LLMs (GPT-4o, Gemini, DeepSeek) under zero-shot prompting, and compact student models (CoLLMs) are then trained to mimic those scores, enabling application to real UK Biobank data (N = 439,221) for multimorbidity scoring without exposing patient-level data to third-party APIs, with evaluation against the Charlson (CCI) and Elixhauser (ECI) indices via survival analysis, genome-wide association studies, and polygenic risk score associations.
This study introduces and evaluates a privacy-preserving knowledge distillation framework in which CTGAN-generated synthetic cohorts matching UK Biobank distributions are used to elicit multimorbidity scores from three teacher LLMs (GPT-4o, Gemini, DeepSeek) under zero-shot prompting, and compact student models (CoLLMs) are then trained to mimic those scores, enabling application to real UK Biobank data (N = 439,221) for multimorbidity scoring without exposing patient-level data to third-party APIs, with evaluation against the Charlson (CCI) and Elixhauser (ECI) indices via survival analysis, genome-wide association studies, and polygenic risk score associations.
medRxiv Sep 21, 2026
The study introduces LLMPopSim, a generative population simulation framework that integrates U.S. Census and CDC data to construct synthetic individuals and uses an LLM to simulate individual health behaviors whose aggregate outcomes can be evaluated at the community level; developed with historical Hawaiʻi data and evaluated for temporal and geographic generalizability using held-out 2022 cohorts from Hawaiʻi and New York State with colorectal cancer screening and mammography as proof-of-concept behaviors, the four state-outcome evaluations showed mean absolute error of 3.5 to 15.0 percentage points and correlations between simulated and observed ZCTA-level prevalence of 0.26 to 0.69.
The study introduces LLMPopSim, a generative population simulation framework that integrates U.S. Census and CDC data to construct synthetic individuals and uses an LLM to simulate individual health behaviors whose aggregate outcomes can be evaluated at the community level; developed with historical Hawaiʻi data and evaluated for temporal and geographic generalizability using held-out 2022 cohorts from Hawaiʻi and New York State with colorectal cancer screening and mammography as proof-of-concept behaviors, the four state-outcome evaluations showed mean absolute error of 3.5 to 15.0 percentage points and correlations between simulated and observed ZCTA-level prevalence of 0.26 to 0.69.
Simulating Community Health Behavior with LLM Synthetic Populations: A Cross-State Evaluation of LLMPopSim
The study introduces LLMPopSim, a generative population simulation framework that integrates U.S. Census and CDC data to construct synthetic individuals and uses an LLM to simulate individual health behaviors whose aggregate outcomes can be evaluated at the community level; developed with historical Hawaiʻi data and evaluated for temporal and geographic generalizability using held-out 2022 cohorts from Hawaiʻi and New York State with colorectal cancer screening and mammography as proof-of-concept behaviors, the four state-outcome evaluations showed mean absolute error of 3.5 to 15.0 percentage points and correlations between simulated and observed ZCTA-level prevalence of 0.26 to 0.69.
The study introduces LLMPopSim, a generative population simulation framework that integrates U.S. Census and CDC data to construct synthetic individuals and uses an LLM to simulate individual health behaviors whose aggregate outcomes can be evaluated at the community level; developed with historical Hawaiʻi data and evaluated for temporal and geographic generalizability using held-out 2022 cohorts from Hawaiʻi and New York State with colorectal cancer screening and mammography as proof-of-concept behaviors, the four state-outcome evaluations showed mean absolute error of 3.5 to 15.0 percentage points and correlations between simulated and observed ZCTA-level prevalence of 0.26 to 0.69.
medRxiv Sep 21, 2026
This work presents MES, a multi-agent framework of six specialized agents (Supervisor, LiteratureMiner, RWD-Analyst, KG-Specialist, Statistician, SafeChecker) that integrates published literature and trial evidence, generates question-specific real-world evidence from real-world clinical data, and queries biomedical knowledge graphs while preserving source traceability; evaluated in two clinical use cases (an Alzheimer disease medication question with no matching literature and a septic shock beta-blocker question with inconsistent randomized evidence) and across 144 clinical queries spanning six evidence-based medicine categories, it produced structured reports that preserved source traceability, identified cross-source disagreement and communicated uncertainty, and its SafeChecker improv
This work presents MES, a multi-agent framework of six specialized agents (Supervisor, LiteratureMiner, RWD-Analyst, KG-Specialist, Statistician, SafeChecker) that integrates published literature and trial evidence, generates question-specific real-world evidence from real-world clinical data, and queries biomedical knowledge graphs while preserving source traceability; evaluated in two clinical use cases (an Alzheimer disease medication question with no matching literature and a septic shock beta-blocker question with inconsistent randomized evidence) and across 144 clinical queries spanning six evidence-based medicine categories, it produced structured reports that preserved source traceability, identified cross-source disagreement and communicated uncertainty, and its SafeChecker improv
MES: A Multi-Agent Evidence Synthesis System for Medical Decision-Making
This work presents MES, a multi-agent framework of six specialized agents (Supervisor, LiteratureMiner, RWD-Analyst, KG-Specialist, Statistician, SafeChecker) that integrates published literature and trial evidence, generates question-specific real-world evidence from real-world clinical data, and queries biomedical knowledge graphs while preserving source traceability; evaluated in two clinical use cases (an Alzheimer disease medication question with no matching literature and a septic shock beta-blocker question with inconsistent randomized evidence) and across 144 clinical queries spanning six evidence-based medicine categories, it produced structured reports that preserved source traceability, identified cross-source disagreement and communicated uncertainty, and its SafeChecker improv
This work presents MES, a multi-agent framework of six specialized agents (Supervisor, LiteratureMiner, RWD-Analyst, KG-Specialist, Statistician, SafeChecker) that integrates published literature and trial evidence, generates question-specific real-world evidence from real-world clinical data, and queries biomedical knowledge graphs while preserving source traceability; evaluated in two clinical use cases (an Alzheimer disease medication question with no matching literature and a septic shock beta-blocker question with inconsistent randomized evidence) and across 144 clinical queries spanning six evidence-based medicine categories, it produced structured reports that preserved source traceability, identified cross-source disagreement and communicated uncertainty, and its SafeChecker improv
medRxiv Sep 21, 2026
The study developed a HIPAA-compliant open-source pipeline in which off-the-shelf commercial large language models, without fine-tuning, abstracted variables from unnormalized, unlabeled, and unedited clinical notes, pathology reports, medication administration records, and demographics for 100 complex breast cancer patients (median chart over 3,100 pages, median 6.5 years of follow-up, median 7 lines of therapy), achieving high concordance with an expert oncologist for recurrence status (99%), germline BRCA1/2 pathogenic variants (100%), hormone receptor status (99%), HER2 status (96%), clinical stage (91%), PIK3CA mutation status (91%), and ESR1 mutation status (90%), approaching inter-oncologist variability for anti-cancer drug extraction, while exact therapy-line reconstruction remaine
The study developed a HIPAA-compliant open-source pipeline in which off-the-shelf commercial large language models, without fine-tuning, abstracted variables from unnormalized, unlabeled, and unedited clinical notes, pathology reports, medication administration records, and demographics for 100 complex breast cancer patients (median chart over 3,100 pages, median 6.5 years of follow-up, median 7 lines of therapy), achieving high concordance with an expert oncologist for recurrence status (99%), germline BRCA1/2 pathogenic variants (100%), hormone receptor status (99%), HER2 status (96%), clinical stage (91%), PIK3CA mutation status (91%), and ESR1 mutation status (90%), approaching inter-oncologist variability for anti-cancer drug extraction, while exact therapy-line reconstruction remaine
Fully Automated Abstraction of Longitudinal Breast Oncology Records with Off-The-Shelf Large Language Models
The study developed a HIPAA-compliant open-source pipeline in which off-the-shelf commercial large language models, without fine-tuning, abstracted variables from unnormalized, unlabeled, and unedited clinical notes, pathology reports, medication administration records, and demographics for 100 complex breast cancer patients (median chart over 3,100 pages, median 6.5 years of follow-up, median 7 lines of therapy), achieving high concordance with an expert oncologist for recurrence status (99%), germline BRCA1/2 pathogenic variants (100%), hormone receptor status (99%), HER2 status (96%), clinical stage (91%), PIK3CA mutation status (91%), and ESR1 mutation status (90%), approaching inter-oncologist variability for anti-cancer drug extraction, while exact therapy-line reconstruction remaine
The study developed a HIPAA-compliant open-source pipeline in which off-the-shelf commercial large language models, without fine-tuning, abstracted variables from unnormalized, unlabeled, and unedited clinical notes, pathology reports, medication administration records, and demographics for 100 complex breast cancer patients (median chart over 3,100 pages, median 6.5 years of follow-up, median 7 lines of therapy), achieving high concordance with an expert oncologist for recurrence status (99%), germline BRCA1/2 pathogenic variants (100%), hormone receptor status (99%), HER2 status (96%), clinical stage (91%), PIK3CA mutation status (91%), and ESR1 mutation status (90%), approaching inter-oncologist variability for anti-cancer drug extraction, while exact therapy-line reconstruction remaine
World Journal of Methodology Sep 20, 2026
This narrative review summarizes contemporary strategies for managing constipation, encompassing lifestyle and dietary modifications, pharmacological therapies, behavioral interventions, and surgical options for refractory cases, noting that traditional laxatives remain the mainstay while newer agents such as prosecretory drugs, serotonergic agonists, and bile acid modulators have expanded therapeutic possibilities, that non-pharmacological approaches including biofeedback and neuromodulation provide effective alternatives in selected patients, and that substantial methodological variability in clinical trials across diagnostic criteria, endpoints, and follow-up durations limits generalizability, with future research focused on individualized treatment for refractory constipation, surgical
This narrative review summarizes contemporary strategies for managing constipation, encompassing lifestyle and dietary modifications, pharmacological therapies, behavioral interventions, and surgical options for refractory cases, noting that traditional laxatives remain the mainstay while newer agents such as prosecretory drugs, serotonergic agonists, and bile acid modulators have expanded therapeutic possibilities, that non-pharmacological approaches including biofeedback and neuromodulation provide effective alternatives in selected patients, and that substantial methodological variability in clinical trials across diagnostic criteria, endpoints, and follow-up durations limits generalizability, with future research focused on individualized treatment for refractory constipation, surgical
Management of Constipation: A Narrative Review of Evolving Strategies and Methodological Challenges
This narrative review summarizes contemporary strategies for managing constipation, encompassing lifestyle and dietary modifications, pharmacological therapies, behavioral interventions, and surgical options for refractory cases, noting that traditional laxatives remain the mainstay while newer agents such as prosecretory drugs, serotonergic agonists, and bile acid modulators have expanded therapeutic possibilities, that non-pharmacological approaches including biofeedback and neuromodulation provide effective alternatives in selected patients, and that substantial methodological variability in clinical trials across diagnostic criteria, endpoints, and follow-up durations limits generalizability, with future research focused on individualized treatment for refractory constipation, surgical
This narrative review summarizes contemporary strategies for managing constipation, encompassing lifestyle and dietary modifications, pharmacological therapies, behavioral interventions, and surgical options for refractory cases, noting that traditional laxatives remain the mainstay while newer agents such as prosecretory drugs, serotonergic agonists, and bile acid modulators have expanded therapeutic possibilities, that non-pharmacological approaches including biofeedback and neuromodulation provide effective alternatives in selected patients, and that substantial methodological variability in clinical trials across diagnostic criteria, endpoints, and follow-up durations limits generalizability, with future research focused on individualized treatment for refractory constipation, surgical
World Journal of Methodology Sep 20, 2026
This study submitted 39 frequently asked patient questions about "acidity" (heartburn/dyspepsia/gastroesophageal reflux disease) to ChatGPT-5, Gemini-2.5, and Claude-4, had responses independently rated by three gastroenterologists for accuracy, comprehensiveness, empathy, and actionability and by 20 patients for empathy, comprehensiveness, actionability, compassion, and usefulness, and analyzed readability indices, finding significant inter-model differences across multiple physician-rated domains, with Gemini-2.5 and Claude-4 scoring higher than ChatGPT-5 for accuracy, comprehensiveness, and actionability, Claude-4 showing the highest empathy scores, uniformly high patient-rated comprehensibility across all models, patient ratings of Gemini-2.
This study submitted 39 frequently asked patient questions about "acidity" (heartburn/dyspepsia/gastroesophageal reflux disease) to ChatGPT-5, Gemini-2.5, and Claude-4, had responses independently rated by three gastroenterologists for accuracy, comprehensiveness, empathy, and actionability and by 20 patients for empathy, comprehensiveness, actionability, compassion, and usefulness, and analyzed readability indices, finding significant inter-model differences across multiple physician-rated domains, with Gemini-2.5 and Claude-4 scoring higher than ChatGPT-5 for accuracy, comprehensiveness, and actionability, Claude-4 showing the highest empathy scores, uniformly high patient-rated comprehensibility across all models, patient ratings of Gemini-2.
Do bots provide correct and adequate guidance regarding acidity: A blinded comparison rated by patients and physicians
This study submitted 39 frequently asked patient questions about "acidity" (heartburn/dyspepsia/gastroesophageal reflux disease) to ChatGPT-5, Gemini-2.5, and Claude-4, had responses independently rated by three gastroenterologists for accuracy, comprehensiveness, empathy, and actionability and by 20 patients for empathy, comprehensiveness, actionability, compassion, and usefulness, and analyzed readability indices, finding significant inter-model differences across multiple physician-rated domains, with Gemini-2.5 and Claude-4 scoring higher than ChatGPT-5 for accuracy, comprehensiveness, and actionability, Claude-4 showing the highest empathy scores, uniformly high patient-rated comprehensibility across all models, patient ratings of Gemini-2.
This study submitted 39 frequently asked patient questions about "acidity" (heartburn/dyspepsia/gastroesophageal reflux disease) to ChatGPT-5, Gemini-2.5, and Claude-4, had responses independently rated by three gastroenterologists for accuracy, comprehensiveness, empathy, and actionability and by 20 patients for empathy, comprehensiveness, actionability, compassion, and usefulness, and analyzed readability indices, finding significant inter-model differences across multiple physician-rated domains, with Gemini-2.5 and Claude-4 scoring higher than ChatGPT-5 for accuracy, comprehensiveness, and actionability, Claude-4 showing the highest empathy scores, uniformly high patient-rated comprehensibility across all models, patient ratings of Gemini-2.
World Journal of Methodology Sep 20, 2026
This review indicates that bone mineral density from dual-energy X-ray absorptiometry (DEXA) explains only part of fracture risk, while quantitative MRI techniques (T1ρ, T2 mapping, proton density fat fraction, diffusion-weighted imaging) and Vertebral Bone Quality (VBQ) scoring capture bone quality through collagen integrity, proteoglycan content, water distribution, and marrow adiposity; VBQ predicts vertebral fragility fractures independently of BMD with sensitivity exceeding 90% and discriminatory ability comparable to the fracture risk assessment tool and trabecular bone score, and integration with artificial intelligence can support opportunistic, radiation-free screening.
This review indicates that bone mineral density from dual-energy X-ray absorptiometry (DEXA) explains only part of fracture risk, while quantitative MRI techniques (T1ρ, T2 mapping, proton density fat fraction, diffusion-weighted imaging) and Vertebral Bone Quality (VBQ) scoring capture bone quality through collagen integrity, proteoglycan content, water distribution, and marrow adiposity; VBQ predicts vertebral fragility fractures independently of BMD with sensitivity exceeding 90% and discriminatory ability comparable to the fracture risk assessment tool and trabecular bone score, and integration with artificial intelligence can support opportunistic, radiation-free screening.
Quantitative MRI and Vertebral Bone Quality Scoring for Fragility Fracture Risk Prediction Beyond Density
This review indicates that bone mineral density from dual-energy X-ray absorptiometry (DEXA) explains only part of fracture risk, while quantitative MRI techniques (T1ρ, T2 mapping, proton density fat fraction, diffusion-weighted imaging) and Vertebral Bone Quality (VBQ) scoring capture bone quality through collagen integrity, proteoglycan content, water distribution, and marrow adiposity; VBQ predicts vertebral fragility fractures independently of BMD with sensitivity exceeding 90% and discriminatory ability comparable to the fracture risk assessment tool and trabecular bone score, and integration with artificial intelligence can support opportunistic, radiation-free screening.
This review indicates that bone mineral density from dual-energy X-ray absorptiometry (DEXA) explains only part of fracture risk, while quantitative MRI techniques (T1ρ, T2 mapping, proton density fat fraction, diffusion-weighted imaging) and Vertebral Bone Quality (VBQ) scoring capture bone quality through collagen integrity, proteoglycan content, water distribution, and marrow adiposity; VBQ predicts vertebral fragility fractures independently of BMD with sensitivity exceeding 90% and discriminatory ability comparable to the fracture risk assessment tool and trabecular bone score, and integration with artificial intelligence can support opportunistic, radiation-free screening.
World Journal of Methodology Sep 20, 2026
This review systematically evaluates the methodological quality, clinical validity, and translational evidence of cutting-edge technologies in the precision management of gastrointestinal tumor-associated osteoporosis (GTO), summarizing their current applications in AI-assisted early screening and risk prediction, nano-enabled targeted bone protection, and multiomics-based exploration of the tumor-bone-gut axis, and discussing barriers to clinical translation such as limited AI generalizability, nanomedicine safety and manufacturing challenges, difficulties in multidimensional data integration and standardization, imperfect multidisciplinary collaboration, and ethical concerns, concluding that these technologies are expected to move GTO management from empirical practice toward precision m
This review systematically evaluates the methodological quality, clinical validity, and translational evidence of cutting-edge technologies in the precision management of gastrointestinal tumor-associated osteoporosis (GTO), summarizing their current applications in AI-assisted early screening and risk prediction, nano-enabled targeted bone protection, and multiomics-based exploration of the tumor-bone-gut axis, and discussing barriers to clinical translation such as limited AI generalizability, nanomedicine safety and manufacturing challenges, difficulties in multidimensional data integration and standardization, imperfect multidisciplinary collaboration, and ethical concerns, concluding that these technologies are expected to move GTO management from empirical practice toward precision m
Precision Management of Gastrointestinal Tumor-Associated Osteoporosis Driven by Cutting-Edge Technologies: Current Status, Challenges, and Future Prospects
This review systematically evaluates the methodological quality, clinical validity, and translational evidence of cutting-edge technologies in the precision management of gastrointestinal tumor-associated osteoporosis (GTO), summarizing their current applications in AI-assisted early screening and risk prediction, nano-enabled targeted bone protection, and multiomics-based exploration of the tumor-bone-gut axis, and discussing barriers to clinical translation such as limited AI generalizability, nanomedicine safety and manufacturing challenges, difficulties in multidimensional data integration and standardization, imperfect multidisciplinary collaboration, and ethical concerns, concluding that these technologies are expected to move GTO management from empirical practice toward precision m
This review systematically evaluates the methodological quality, clinical validity, and translational evidence of cutting-edge technologies in the precision management of gastrointestinal tumor-associated osteoporosis (GTO), summarizing their current applications in AI-assisted early screening and risk prediction, nano-enabled targeted bone protection, and multiomics-based exploration of the tumor-bone-gut axis, and discussing barriers to clinical translation such as limited AI generalizability, nanomedicine safety and manufacturing challenges, difficulties in multidimensional data integration and standardization, imperfect multidisciplinary collaboration, and ethical concerns, concluding that these technologies are expected to move GTO management from empirical practice toward precision m
medRxiv Sep 20, 2026
This study built a proof-of-concept information extraction pipeline using three open-source LLMs (Athena-v3-AWQ, Gemma3-27B, and Llama-3.1-70B-Instruct) to extract 11 natural history study characteristics from PubMed abstracts labeled "2" in the CZI DRSM corpus (148 gold-standard and 3,547 full-corpus abstracts), finding that all three models exceeded 99% processing success, that Gemma was best overall on expert rating (68.0% of outputs rated "good") and full-corpus runtime (~16 minutes), that Llama scored higher on automated Token F1 (0.874 vs. 0.723), and that Athena performed worst largely because it copied source text verbatim rather than synthesizing it.
This study built a proof-of-concept information extraction pipeline using three open-source LLMs (Athena-v3-AWQ, Gemma3-27B, and Llama-3.1-70B-Instruct) to extract 11 natural history study characteristics from PubMed abstracts labeled "2" in the CZI DRSM corpus (148 gold-standard and 3,547 full-corpus abstracts), finding that all three models exceeded 99% processing success, that Gemma was best overall on expert rating (68.0% of outputs rated "good") and full-corpus runtime (~16 minutes), that Llama scored higher on automated Token F1 (0.874 vs. 0.723), and that Athena performed worst largely because it copied source text verbatim rather than synthesizing it.
An LLM-Enabled Pipeline for Natural History Study Information Extraction in Rare Disease Research
This study built a proof-of-concept information extraction pipeline using three open-source LLMs (Athena-v3-AWQ, Gemma3-27B, and Llama-3.1-70B-Instruct) to extract 11 natural history study characteristics from PubMed abstracts labeled "2" in the CZI DRSM corpus (148 gold-standard and 3,547 full-corpus abstracts), finding that all three models exceeded 99% processing success, that Gemma was best overall on expert rating (68.0% of outputs rated "good") and full-corpus runtime (~16 minutes), that Llama scored higher on automated Token F1 (0.874 vs. 0.723), and that Athena performed worst largely because it copied source text verbatim rather than synthesizing it.
This study built a proof-of-concept information extraction pipeline using three open-source LLMs (Athena-v3-AWQ, Gemma3-27B, and Llama-3.1-70B-Instruct) to extract 11 natural history study characteristics from PubMed abstracts labeled "2" in the CZI DRSM corpus (148 gold-standard and 3,547 full-corpus abstracts), finding that all three models exceeded 99% processing success, that Gemma was best overall on expert rating (68.0% of outputs rated "good") and full-corpus runtime (~16 minutes), that Llama scored higher on automated Token F1 (0.874 vs. 0.723), and that Athena performed worst largely because it copied source text verbatim rather than synthesizing it.

Page 24 · showing 10