Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2609.35151

"What If I Had Participated?" A Machine Learning-Based Counterfactual Analysis of Non-Participants' Success Potential in STEM Competitions: The Case of the German Physics Olympiad

Paul Tschisgale·Knut Neumann
View original source ↗Read source document ↗Export metadata ↓

Abstract

STEM student competitions aim to identify capable, science-interested students and further support them in developing their STEM-related abilities and interests. When such students do not participate, they may appear to be missed not only by the competition but by this form of support altogether; unless they are reached through other competitions. The present study therefore investigated to what extent students who would likely have succeeded in the German Physics Olympiad are missed by it. The study sample comprised 282 Olympiad participants and 1,103 non-participants from academic-track secondary schools, all assessed on 31 indicators spanning sociodemographic background, cognitive abilities, self-related beliefs, motivational variables, personality traits, vocational interests, and prior participation across other STEM competitions. Three machine learning models (elastic net, random forest, gradient boosting) were trained on the participant sample to predict first-round success based on those indicators. The elastic net was found to perform best. Although predictive accuracy was modest, the model proved well-calibrated, supporting valid group-level inferences. Only a few variables predicted success: mathematics skills, physics- and engineering-related skills, and having skipped a grade were positive predictors, whereas conventional vocational interests were a negative predictor. When applied to the non-participants, the model identified 43 students (3.9%) as potentially successful. Crucially, 74% of these students had previously entered at least one selective STEM competition, leaving only 11 students (ca. 1% of all non-participants) entirely unreached by the selective STEM competition system. These findings indicate that the majority of potentially successful non-participants are not absent from the selective STEM competition system but are engaged by it through other domains.

Explore related subjects

physics.ed-ph

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Paul Tschisgale, Knut Neumann. 2026-09-28. "What If I Had Participated?" A Machine Learning-Based Counterfactual Analysis of Non-Participants' Success Potential in STEM Competitions: The Case of the German Physics Olympiad. https://arxiv.org/abs/2609.35151

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

AI-based scoring systematically underestimates conceptual understanding of linguistically weak students' explanations in physics

Students' explanations of scientific phenomena provide important evidence of their conceptual understanding. However, because conceptual understanding can only be inferred through language rather than observed directly, distinguishing conceptual understanding from linguistic quality represents a fundamental challenge for assessment. This challenge can also be expected to arise when artificial intelligence (AI) is used to score students' text-based explanations. We investigated whether AI-based scoring approaches assess students' conceptual understanding independently of the linguistic quality of their explanations about a phenomenon taken from a physics context. Conceptual understanding scores assigned by nine machine learning-based scoring approaches and two large language model-based scoring approaches were compared with expert-assigned scores for 116 explanations produced by secondary-school students in Germany. Multinomial logistic regression analyses examined whether expert-rated linguistic quality was associated with under- or overestimation of conceptual understanding. Across all eleven AI-based scoring approaches, lower linguistic quality was associated with a greater likelihood of underestimation, although the strength and statistical significance of this association varied between approaches. AI-based scoring approaches may thus systematically underestimate the conceptual understanding expressed in linguistically weaker explanations. This pattern resembles language bias previously documented in STEM teachers' assessment practices. The findings highlight the importance of examining not only agreement with expert ratings but also whether AI-based scoring approaches inadvertently rely on construct-irrelevant information. Addressing language bias is therefore essential for the validity and fairness of AI-supported assessment in physics education and STEM education more broadly.

physics.ed-ph↗
arXiv2026

Community agreements pedagogy in physics

The idea of engaging critically and creatively with physics theories is a stretch for many physicists and physics educators. Most physicists become enculturated into the belief that physics definitions, laws, and equations are read from the book of nature. However, this belief obscures how theories are collectively negotiated by people in specific socio-political and personal conditions, and how students' particular ideas and intuitions will inevitably shape their own practice of science. To help students build their skills for collective theory-building, we suggest a pedagogy for physics in which students actively negotiate the laws and definitions of theories.

physics.ed-ph↗
arXiv2026

From Two-Level Hamiltonians to Quantum Superposition and Measurement: A Traceable Classroom Module

We present a traceable instructional module that maps documented learner difficulties in quantum superposition and measurement to explicit learning goals, classroom activities, and assessment evidence. Using the general two-level Hamiltonian as the organizing physics framework, the module implements this mapping through a five-activity classroom sequence and grading rubric targeting four recurrent difficulties: interpreting superposition as physical splitting, confusing a quantum state with its representation or measurement context, mixing theoretical probabilities with finite-sample frequencies, and using inconsistent notation across states, amplitudes, probabilities, and measurement outcomes. The instructional design is organized through a reversible mapping from documented barriers to learning goals, activity responses, and assessment evidence, allowing each assessment item to be traced back to the specific difficulty it is intended to address. The module uses the standard general two-level Hermitian Hamiltonian, represented by a \(2\times2\) matrix, as the physical framework connecting its eigenstates, computational-basis amplitudes, Born probabilities, and sampled outcomes. The contribution is a challenge-targeted instructional design that organizes standard quantum mechanics into a traceable classroom sequence linking documented learner difficulties to learning goals, activities, assessment criteria, and implementation guidance. The mathematical structure uses standard Hamiltonian and measurement conventions, supporting direct incorporation into an undergraduate quantum-mechanics course.

physics.ed-ph↗