On-premise medical AI agents for reliable clinical decision-making

It presents and evaluates a fully on-premise clinical agent that couples local operational control with a multi-perspective reliability framework to support selective autonomy. Relative to cloud-dependent approaches, it places deployment control inside the institution and treats reliability assessment as part of agent design rather than an add-on. Evaluated on two MIMIC-IV-derived benchmarks, reporting 90.04% accuracy on a seven-disease task and 83.8% on a four-disease task, and described as approaching a cloud baseline on the primary benchmark.

It quantifies three families of decision-time reliability measures for diagnosis and reasoning: internal-likelihood, language-based, and behavioral-stability measures. It expands uncertainty estimation from a single signal into a multi-perspective framework spanning both diagnosis and reasoning. The study quantifies and compares the three measure families and reports that diagnostic behavioral consistency provides the strongest discrimination of correctness.

Diagnostic behavioral consistency is the strongest signal for discriminating diagnostic correctness and remains informative under stress testing. It offers an actionable, behavior-stability-based indicator for decision-time reliability. Reported AUC = 0.860, and AUC = 0.875 under stress testing.

At a consistency threshold of 0.90, 49.4% of cases were retained at 98.9% diagnostic accuracy, identifying a lower-risk subset for autonomous handling while deferring the remainder for review. It turns reliability signals into an executable triage strategy, providing a concrete threshold basis for selective autonomy. Based on evaluation across two MIMIC-IV-derived benchmarks, reporting the retained proportion and corresponding accuracy.

The framework targets settings that need to deploy and govern clinical AI agents inside an institution, and is meant for users who want decision-time reliability signals to delineate autonomous handling from human review. Its evaluation rests on two MIMIC-IV-derived benchmarks, so results apply to that data source and the corresponding disease-task settings; at a consistency threshold of 0.90, retaining 49.4% of cases at 98.9% accuracy indicates the strategy addresses a subset identifiable as lower risk rather than all cases.

A careful reader may still watch how the multi-perspective reliability measures hold up on data distributions beyond the MIMIC-IV-derived benchmarks; how the 49.4% retention at a 0.90 consistency threshold varies across diseases and institutions; and how the remaining cases deferred for review are handled in real workflows. The current text is abstract-level information without figures or full methodological detail, so deeper understanding of measure implementation and the specific stress-testing setup would require consulting the original.