Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2608.20521

Large Scale AI Grading of Handwritten Physics Assessments: Score Agreement and Olympiad Team Selection Outcomes

Praveen Pathak·Siddharth Tiwary·Charudatt Kadolkar·Vijay Singh·David Rakestraw·Shirish Pathare·Anwesh Mazumdar
View original source ↗Read source document ↗Export metadata ↓

Abstract

Multimodal AI can read handwritten physics solutions, but high-stakes grading requires agreement with official scores and outcomes. This study evaluated GPT-5.5-based grading on 10364 scanned pages from 520 handwritten submissions by 416 unique candidates or students across three assessments: a national Physics Olympiad theory examination, the final Olympiad selection camp with theory and experiment components, and a university quantum-mechanics examination. Each submission was graded twice by AI using the official rubrics. The second round used revised page-by-page and evidence-location instructions developed after first-round disagreement analysis. During grading, AI did not see official human marks or AI--human comparisons. Total-score correlations with official marks were high (0.91--0.97). For the final Olympiad selection, AI recovered the same five-student team as official grading. The second round improved aggregate question-part agreement, especially where first-round disagreements were larger. The main difficulty remained exact partial-credit grading, especially in experimental work. Reliable AI grading therefore depends on detailed rubrics and should be used as a second reader or audit tool under examiner control.

Explore related subjects

physics.ed-phcs.AIcs.CY

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Praveen Pathak, Siddharth Tiwary, Charudatt Kadolkar, Vijay Singh, David Rakestraw, Shirish Pathare, Anwesh Mazumdar. 2026-08-20. Large Scale AI Grading of Handwritten Physics Assessments: Score Agreement and Olympiad Team Selection Outcomes. https://arxiv.org/abs/2608.20521

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Johann Bernoulli's analysis of elastic collisions (a teaching sequence to introduce the dynamic law thereby inspired)

In order to explain an elastic collision, Johann Bernoulli considered two bodies connected by a spring. Motion is defined as a succession of states of rest. Then, considering the spring to be a lever with a body at each extremity, the laws of equilibrium imply that `motion' is described by the time variation of the (common) quantity of motion of the bodies; the dynamic law is thus deduced. This inspires a teaching sequence to introduce the dynamic law (in one dimension) in introductory physics course; we call it ``bernoullian sequence''.

physics.ed-ph↗
arXiv2026

Design and Initial Evaluation of a Photovoltaics-focused Course-based Undergraduate Research Experience in Physics

Traditional physics laboratory courses often focus on experiments with well-known results, limiting students' engagement in authentic scientific practices. Course-based undergraduate research experiences (CUREs), where students engage in real research with unknown outcomes, have been shown to support positive student outcomes, such as increased self-efficacy, persistence, and engagement in scientific practices. However, discipline-specific studies of CUREs in physics remain limited. We describe the development, structure, and initial implementation of a photovoltaics-focused CURE in a second-year undergraduate physics laboratory course at the University of Colorado Boulder. To examine how students experienced the course, we analyzed end-of-semester reflection assignments using the five CURE components (i.e., scientific practices, discovery, relevance, collaboration, and iteration), as well as established dimensions of research authenticity, as analytic frameworks. Students described experiences associated with all five CURE components, with collaboration, relevance, and scientific practices appearing most prominently in their reflections. Students also associated authentic research with meaningful scientific contribution, engagement in authentic scientific practices, and navigating the uncertainty and setbacks inherent in research, although fewer explicitly identified themselves as researchers or scientists. A subset of students additionally connected the course to their immediate thinking about future academic and professional pathways. This work contributes both a discipline-specific model for implementing CUREs in experimental physics laboratory courses and provides insight into how students interpret and experience authentic research within this course context.

physics.ed-ph↗
arXiv2026

A Workshop Series for Effective Use of AI in Uncertain Times: Building a Physics Faculty Learning Community

Generative AI tools are being widely taken up by students in their physics courses and beyond, often before instructors and institutions can develop policies and effective approaches for the use of these tools. Building on a framework for change in the era of AI, we developed and implemented a faculty learning community to help a university physics department address these challenges collectively. Over six biweekly sessions, faculty worked through course policies, classroom conversations about AI, AI-integrated coursework tasks, and assessment. Each session shared a common structure: we presented local data and department-sourced materials, tested them in small groups, and discussed them together, emphasizing durable pedagogical approaches over specific tools and platforms, and leading with evidence of students' own AI use. The series produced a shared, evolving repository of resources for faculty to draw on. This workshop provides an adaptable, theoretically informed model for a faculty learning community that departments can build on.

physics.ed-ph↗