The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing

Abstract

Autonomous AI systems are transitioning from advisory roles to autonomous ones for medication prescriptions. Recent U.S. bill H.R. 238 and Utah's prescription-renewal pilot program both authorize AI to prescribe medications in an agentic capacity. While many regulatory guidelines suggest aggregate model performance metrics at the point of clearance, they do not require i) calibrated per-prediction confidence for action-gated thresholds, ii) differentiated communication between uncertainty arising from model ignorance (epistemic) from genuine clinical ambiguity (aleatoric), and iii) inferential transparency at the moment of decision enabling liability allocation. Here, we argue these three architectural features are minimum conditions for safe autonomous prescribing, and validate them with a survey of 136 U.S. prescribing clinicians. Our results suggest prescribing clinicians i) would not permit autonomous prescribing without a confidence-based escalation mechanism, ii) preferred a competing-options summary for aleatoric uncertainty but preferred abstention for epistemic uncertainty, and iii) were only willing to accept liability when inferential transparency enabled them to make a decision under acknowledged uncertainty. These findings indicate that our recommended architectural features would encourage higher rates of clinician adoption of autonomous AI prescribing, largely through collapsing much of what "autonomy" conventionally means.