Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2503.05793

MedSimAI: Simulation and Formative Feedback Generation to Enhance Deliberate Practice in Medical Education

Yann Hicke·Jadon Geathers·Kellen Vu·Justin Sewell·Claire Cardie·Jaideep Talwalkar·Dennis Shung·Anyanate Gwendolyne Jack·Susannah Cornes·Mackenzi Preston·Rene Kizilcec
View original source ↗Read source document ↗Export metadata ↓

Abstract

Medical education faces challenges in providing scalable, consistent clinical skills training. Simulation with standardized patients (SPs) develops communication and diagnostic skills but remains resource-intensive and variable in feedback quality. Existing AI-based tools show promise yet often lack comprehensive assessment frameworks, evidence of clinical impact, and integration of self-regulated learning (SRL) principles. Through a multi-phase co-design process with medical education experts, we developed MedSimAI, an AI-powered simulation platform that enables deliberate practice through interactive patient encounters with immediate, structured feedback. Leveraging large language models, MedSimAI generates realistic clinical interactions and provides automated assessments aligned with validated evaluation frameworks. In a multi-institutional deployment (410 students; 1,024 encounters across three medical schools), 59.5 percent engaged in repeated practice. At one site, mean Objective Structured Clinical Examination (OSCE) history-taking scores rose from 82.8 to 88.8 (p < 0.001, Cohen's d = 0.75), while a second site's pilot showed no significant change. Automated scoring achieved 87 percent accuracy in identifying proficiency thresholds on the Master Interview Rating Scale (MIRS). Mixed-effects analyses revealed institution and case effects. Thematic analysis of 840 learner reflections highlighted challenges in missed items, organization, review of systems, and empathy. These findings position MedSimAI as a scalable formative platform for history-taking and communication, motivating staged curriculum integration and realism enhancements for advanced learners.

Explore related subjects

cs.CYcs.AIcs.CL

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yann Hicke, Jadon Geathers, Kellen Vu, Justin Sewell, Claire Cardie, Jaideep Talwalkar, Dennis Shung, Anyanate Gwendolyne Jack, Susannah Cornes, Mackenzi Preston, Rene Kizilcec. 2026-01-22. MedSimAI: Simulation and Formative Feedback Generation to Enhance Deliberate Practice in Medical Education. https://doi.org/10.1145/3785022.3785092

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

An Investigation Into Secondary School Students' Debugging Behaviour in Python

Background and context: Debugging is a significant and often frustrating challenge for beginner programmers. Understanding students' debugging behaviours and strategies can help to identify common difficulties and inform approaches for alleviating these. Currently, there are limited studies of school students' debugging behaviour in a text-based programming language, a medium through which millions are learning to program. Objectives: In this paper, we investigate the debugging behaviour of 12-14-year-old students learning Python through a lesson-long classroom study. Method: We collected program snapshots from 73 students' attempts at a set of Python debugging exercises in an online code editor. Through qualitative content analysis of these snapshots, we developed a granular categorisation of the changes students made when debugging. Findings: A range of debugging behaviours were exhibited by students, many of which were ineffective. Students added errors through small-scale changes, reverted corrective changes, and repeatedly ran identical programs in quick succession. From the results, we identify four barriers to successful and reliable debugging for students learning a text-based programming language: fragile knowledge, a lack of systematicity and reflection, the syntax barrier, and dynamics of emotions and attitudes. Implications: This paper highlights some of the difficulties that secondary school students have when debugging in Python and the challenges of analysing program snapshots through manual inspection. We recommend that school teachers explicitly teach a systematic approach to debugging and discourage the use of ineffective debugging behaviours, and that programming environments should contain features that facilitate successful debugging.

cs.CY↗
arXiv2026

Printed but not benchmarkable: most building-decarbonisation disclosure cannot be matched to the pathways that stranding regulation assumes

Cities are beginning to enforce carbon limits on existing buildings. Science-based decarbonisation pathways set those limits one asset type and one jurisdiction at a time. Owners, however, report for the whole firm. We measure what that mismatch costs on two sets of public corporate reports: a census of 502 reports from the 119 listed built-environment firms with a collected report inside a 2,246-firm panel (2003-2023), and 519 real-estate reports from 101 firms (2007-2024). BeDA, a multimodal language-model tool whose reliability we test first, read them. Running the pathway frameworks' own entry tests over published disclosure: 16.5% of census reports (43.7% of real-estate reports) print an operational carbon intensity per square metre; 6.6% (25.0%) can be matched to a pathway for their property type in a covered jurisdiction; and only 5.0% (16.4%) disclose the floor area they divided by. Of the failures at the pathway test, 82-84% follow from reports lumping the portfolio together and 16-18% from a missing curve in the pathway library. The obstacle is the reporting unit, not missing data. The rate is roughly twice as high for European as for US listings (65-71% versus 35% in listed real estate). We also show that a US portfolio's carbon verdict cannot be worked out from disclosure at all. Within one climate zone, the pathway's carbon limit varies by up to 2.79-fold with the electricity subregion, which no report names; its energy limit does not move. Extraction is checked against the source PDFs (97.3% of extracted intensities appear verbatim) and repeats on a second extractor (kappa = 0.97). Recall of the non-disclosing class was 95.1% in a blinded hand audit of 122 reports. The fix follows from the measurement: split intensity by asset type and jurisdiction, and report floor area.

cs.CY↗
arXiv2026

Incipit: Axiom-Grounded Scaffolding for Human-AI Literary Creation

Large language models can produce fluent prose from short prompts, but a direct interaction gives writers little access to the assumptions that shape a long narrative. We present Incipit, an implemented research prototype that inserts an explicit planning layer between writer intent and generated prose. The layer is grounded in literary axioms, which are curated and reusable propositions about human experience and narrative craft. The prototype connects a knowledge base of 1455 axioms and 472 typed relationships to a five round direction dialogue, a retrieval and selection pipeline, and a three level blueprint covering creative premises, story beats, character arcs, and chapter outlines. Writers can inspect and edit the resulting structures before using them as context for scene generation. Additional modules support real event abstraction and five dimensional diagnostic feedback. We describe the design rationale, data flow, implementation boundaries, and a worked design example. Because no controlled user study or independently rated output study has yet been completed, we do not claim that the system improves literary quality. Instead, we outline a future preregistered comparison designed to distinguish the contribution of axiom grounding from that of hierarchical planning. The paper contributes a concrete architecture for making literary knowledge an inspectable coordination object in human AI writing.

cs.CY↗