Skip to main content
Back to timeline
researchsquare.comSource publication:

Clinical history-taking stops following a fixed questionnaire: an explicit state picks each question, lifting BERTScore F1 from 0.68 to 0.84

Synopsis

Automated history-taking in telemedicine must collect a structured record over many turns while adapting to what the patient answers. MedAgent-RAG keeps an explicit clinical state of filled slots, a ranked differential diagnosis, questions already asked and red flags detected, selects every question from that state, and grounds its final assessment in clinical guidelines retrieved by a parent–child retrieval-augmented pipeline. Evaluated on the MediTOD corpus with a language-model patient simulator, it raises BERTScore F1 from 0.68 for a single-shot baseline built on the same language model to 0.84, at a cost of 15.8 turns on average.

AI-generated editorial illustration: MedAgent-RAG: an agentic retrieval-augmented system for adaptive clinical history-taking, with a network analysis of medical interviews

Interpretation

Automated history-taking no longer treats the next question as a language-generation problem but as a decision that depends on the current state of the interview, which the system maintains explicitly and updates every turn. Earlier language-model interviewers either walked through a fixed questionnaire, blind to what the patient had just said, or improvised, producing questions that were repetitive, off-target and unaware of what had already been established. On the MediTOD corpus with a language-model patient simulator, against a single-shot baseline of the same language model, BERTScore F1 rises from 0.68 to 0.84, ROUGE-L F1 from 0.20 to 0.35 and ROUGE-2 F1 from 0.15 to 0.25, while ROUGE-1 F1 falls from 0.35 to 0.25.

The state has five parts: the slots gathered so far, a ranked differential diagnosis, the questions already asked kept verbatim, the red flags detected, and the current questioning phase. The questioning policy reads only this state, so the next question depends on the consultation through the state rather than through the surface form of the conversation. The system is implemented as a LangGraph state graph; four questioning phases move only forward, driven by the number of answers and the completeness score, and a detected red flag short-circuits to the final phase, with a completeness threshold of 0.65 and a hard cap of twelve questions.

Once questioning ends, the collected state is rewritten deterministically into a clinical summary that becomes the retrieval query, and a reasoning agent produces the final assessment. Retrieval uses parent–child chunking: documents are split into small child chunks for precise matching, and each matched child is expanded to its parent passage before reaching the model, so precision in retrieval does not cost context in generation. Retrieval fuses dense sentence-embedding similarity with BM25 lexical ranking; in the reported runs every node uses the same backbone model, gpt-oss-120b, with all-mpnet-base-v2 embeddings and Qdrant as the vector store.

Perspective

The demonstrated setting is the 231 English staged respiratory consultations of the MediTOD corpus, paired with a language-model patient simulator grounded in each dialogue's annotations, with the final assessment compared against the physician's closing assessment in the corresponding transcript.

The evaluation compares one configuration with one baseline using automatic measures of text similarity, and each case is scored against a single physician response, which is enough to show that questioning and grounding change the output but not to claim clinical quality. The corpus is a single English-language collection of staged respiratory consultations, so the recovered modules reflect that case mix; the differential treated as the physician's reading is what the physician wrote down, not a verified outcome; and the networks describe practice rather than correctness.