Towards Anatomy Education with Generative AI-based Virtual Assistants in Immersive Virtual Reality Environments

Vuthea Chheang; Shayla Sharmin; Rommy Marquez-Hernandez; Megha Patel; Danush Rajasekaran; Gavin Caulfield; Behdokht Kiafar; Jicheng Li; Pinar Kullu; Roghayeh Leila Barmaki

arXiv · 2306.17278

Towards Anatomy Education with Generative AI-based Virtual Assistants in Immersive Virtual Reality Environments

Vuthea Chheang·Shayla Sharmin·Rommy Marquez-Hernandez·Megha Patel·Danush Rajasekaran·Gavin Caulfield·Behdokht Kiafar·Jicheng Li·Pinar Kullu·Roghayeh Leila Barmaki

View original source ↗Read source document ↗Export metadata ↓

Abstract

Virtual reality (VR) and interactive 3D visualization systems have enhanced educational experiences and environments, particularly in complicated subjects such as anatomy education. VR-based systems surpass the potential limitations of traditional training approaches in facilitating interactive engagement among students. However, research on embodied virtual assistants that leverage generative artificial intelligence (AI) and verbal communication in the anatomy education context is underrepresented. In this work, we introduce a VR environment with a generative AI-embodied virtual assistant to support participants in responding to varying cognitive complexity anatomy questions and enable verbal communication. We assessed the technical efficacy and usability of the proposed environment in a pilot user study with 16 participants. We conducted a within-subject design for virtual assistant configuration (avatar- and screen-based), with two levels of cognitive complexity (knowledge- and analysis-based). The results reveal a significant difference in the scores obtained from knowledge- and analysis-based questions in relation to avatar configuration. Moreover, results provide insights into usability, cognitive task load, and the sense of presence in the proposed virtual assistant configurations. Our environment and results of the pilot study offer potential benefits and future research directions beyond medical education, using generative AI and embodied virtual agents as customized virtual conversational assistants.

Explore related subjects

cs.HC

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Vuthea Chheang, Shayla Sharmin, Rommy Marquez-Hernandez, Megha Patel, Danush Rajasekaran, Gavin Caulfield, Behdokht Kiafar, Jicheng Li, Pinar Kullu, Roghayeh Leila Barmaki. 2024-01-24. Towards Anatomy Education with Generative AI-based Virtual Assistants in Immersive Virtual Reality Environments. https://arxiv.org/abs/2306.17278

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Describe Me Something You Do Not Remember - Challenges and Risks of Exposure Design Using Generative Artificial Intelligence for Therapy of Complex Post-traumatic Stress Disorder

Post-traumatic stress disorder (PTSD) is associated with sudden, uncontrollable, and intense flashbacks of traumatic memories. Trauma exposure psychotherapy has proven effective in reducing the severity of trauma-related symptoms. It involves controlled recall of traumatic memories to train coping mechanisms for flashbacks and enable autobiographical integration of distressing experiences. In particular, exposure to visualizations of these memories supports successful recall. Although this approach is effective for various trauma types, it remains available for only a few. This is due to the lack of cost-efficient solutions for creating individualized exposure visualizations. This issue is particularly relevant for the treatment of Complex PTSD (CPTSD), where traumatic memories are highly individual and generic visualizations do not meet therapeutic needs. Generative Artificial Intelligence (GAI) offers a flexible and cost-effective alternative. GAI enables the creation of individualized exposure visualizations during therapy and, for the first time, allows patients to actively participate in the visualization process. While GAI opens new therapeutic perspectives and may improve access to trauma therapy, especially for CPTSD, it also introduces significant challenges and risks. The extreme uncertainty and lack of control that define both CPTSD and GAI raise concerns about feasibility and safety. To support safe and effective three-way communication, it is essential to understand the roles of patient, system, and therapist in exposure visualization and how each can contribute to safety. This paper outlines perspectives, challenges, and risks associated with the use of GAI in trauma therapy, with a focus on CPTSD.

cs.HC↗

arXiv2026

KnowTeX: Visualizing Mathematical Dependencies

Dependency graphs that show how definitions, theorems, and proofs relate to each other are valuable for understanding the structure of mathematical texts. Existing tools such as Lean Blueprint and plasTeXdepgraph generate such graphs within formal proof ecosystems, but they require familiarity with proof assistants or specific compilation pipelines. We present KnowTeX, a standalone Python tool that extracts dependency graphs directly from LaTeX sources without requiring any external framework. KnowTeX supports two complementary modes: a manual mode where authors annotate their source with lightweight commands compatible with Lean Blueprint, and an infer mode that automatically discovers dependencies through a layered system of deterministic and heuristic rules. The tool handles multi-file projects, detects cycles, applies transitive reduction, and exports graphs in DOT, TikZ, and PNG formats with an interactive preview. We evaluate KnowTeX on several mathematical texts and discuss how it complements recent tools such as LeanArchitect, which operates from the Lean side, while KnowTeX works entirely on the LaTeX side without requiring any formalization.

cs.HC↗

arXiv2026

Helping Customers in Distress: An LLM-powered Agent that Converses, Probes, and Routes

Banks receive millions of reports of fraud, scams, and disputed transactions every year, making it challenging to accurately direct customers to the appropriate specialist teams for assistance. The existing manual process driven by humans is slow and stressful for both customers and staff. To address this, we develop a customer-facing AI powered triaging agent that leverages large language models (LLMs) to conduct multi-turn conversations, ask relevant questions, and classify cases for accurate, policy-guided routing, making it embedded in the customer journey. To evaluate and continuously improve the agent, synthetic digital twins of real customers were simulated, generating realistic, labelled dialogues based on historical data to test a wide range of real-world scenarios. This work details the triage agent's modelling approach, integration with policy, safety guardrails and reasoning frameworks, the use of the synthetic agent for scalable evaluation, and findings on the AI system's accuracy, robustness, and compliance. Results show that the agent successfully improves triaging of historical cases, achieving a 30.6% increase in classification accuracy, with high satisfaction levels reported by our subject-matter experts, highlighting how targeted probing can lead to more effective triage in banking operations at scale.

cs.HC↗