A Guide to Building Efficient QA Systems for Medical Use: Helping Patients Gather Reliable Information

It introduces a way to scrape and de-identify patient-authored text from a public mental health forum (the schizophrenia subforum of The Mental Health Forum) to build a domain QA corpus that is privacy-conscious and lower in institutional bias. Compared with prior sources such as clinical charts, doctor-patient dialogues, or medical board exams, this path uses naturally occurring patient narratives; the authors frame it as reducing institutional and elicitation bias rather than eliminating all sampling bias. The corpus size is stated as 415,602 posts, with usernames removed, direct identifiers excluded when present, and only textual content retained; the authors also note that forum participation is self-selected, so bias reduction is relative.

It applies TGSM to automatically identify topics and aspects, organizing the corpus into 35 topic clusters and about 300 aspects, thereby narrowing the annotation space from the full forum archive to semantically organized topic paragraphs. Compared with standard LDA or embedding-only clustering, TGSM is explicitly integrated into downstream QA construction: topics structure the annotation space and generate question cues, and each topic is represented by explicit high-probability words or phrases, aiding annotator interpretability. The number of topics was chosen via held-out perplexity plus qualitative checks of topic coherence; 35 topics were reported as balancing interpretability and perplexity minimization, with fewer topics merging distinct concerns such as hallucinations, medication effects, and social functioning, and many more producing fragmented or redundant themes.

It constructs a schizophrenia QA dataset of 1,050 QA pairs, 30 per topic cluster, allowing multiple answers per question when supported by paragraph evidence. Compared with datasets such as COVID-QA that support only one answer per question, this dataset retains multiple valid expressions of patient experience; the annotation standard requires each question to rest on a coherent topic-aspect association and each answer to be explicitly supported by paragraph evidence. The construction is described as a topic-guided sampling and annotation procedure rather than compressing all posts into a few paragraphs; detailed statistics appear in Table 2, whose contents are not expanded in the text.

It implements a two-stage Retriever-Reader QA pipeline and fine-tunes BioBERT, RoBERTa, and DistilBERT on the schizophrenia QA dataset, with fine-tuned BioBERT performing best across datasets and metrics. Compared with using pretrained models alone, fine-tuning yields clear gains: on the schizophrenia QA dataset, precision rises 14.30% from base to fine-tuned BioBERT, exact match rises 34.4% from 0.427 to 0.617, and precision improves 32.59% over the lightest model, DistilBERT. Evaluation uses precision, recall, F1, and exact match on a held-out test set of 30% of the QA data, with training on 900 QA pairs (70%); fine-tuned BioBERT reaches precision 0.781 and F1 0.772 on MHQA and precision 0.816 on PubMedQA.

The framework targets settings where a QA dataset and a Retriever-Reader QA system are built from publicly accessible forum patient narratives for a specific disease (here schizophrenia); it suits researchers and developers seeking domain QA resources with lower privacy risk and lower annotation cost. The authors note it can extend to a broader spectrum of mental health conditions and patient demographics and can be used to create QA datasets for more diseases efficiently.

The authors explicitly state that systematically ensuring the reliability, accuracy, and clinical relevance of forum content remains an unresolved challenge, and that automated or semi-automated validation and filtering of clinically relevant content is a promising direction; replicating the framework across multiple mental health domains requires systematically identifying suitable forums, and the practical effect of automated question generation remains to be tested. This was a full-text reading, but the specific values and examples in Tables 2-5 are not expanded in the text, so checking dataset statistics, hyperparameters, and per-metric results still requires the original tables.