Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2604.22812

Cross-Course Generalizability of SRL-Aligned Predictive Models Using Digital Learning Traces

Jakob Schwerter·Loreen Sabel·Judith Bose·Matthew L. Bernacki·Di Xu·Marko Schmellenkamp·Thomas Zeume·Philipp Doebler
View original source ↗Read source document ↗Export metadata ↓

Abstract

STEM dropout rates remain high at universities, particularly in computer science programs with theory-intensive courses. Digital learning environments now capture rich behavioral data that could help identify struggling students early, yet the generalizability of data-driven prediction models across courses and institutions remains uncertain. Guided by self-regulated learning (SRL) theory, this study analyzed multimodal digital-trace data from three undergraduate theoretical computer science courses (N1 = 137, N2 = 104, N3 = 148) at two universities. Weekly SRL-aligned digital-trace indicators were modeled using Elastic Net, Random Forest, and XGBoost to evaluate predictive performance over time and across settings, and model calibration both within and across courses. Early prediction of at-risk students was feasible, with SRL-related behaviors such as time management, effort regulation, and sustained engagement emerging as key predictors. While Random Forest achieved the highest in-sample accuracy, Elastic Net generalized more robustly across contexts. Out-of-sample accuracy and calibration declined between institutions with different base rates, underscoring the contextual nature of predictive analytics in higher education. These findings suggest that digital learning traces enable early identification of at-risk students within courses, but generalizing predictive models beyond their original context requires caution, particularly if the at-risk rates differ between contexts.

Explore related subjects

cs.CYcs.LGstat.AP

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jakob Schwerter, Loreen Sabel, Judith Bose, Matthew L. Bernacki, Di Xu, Marko Schmellenkamp, Thomas Zeume, Philipp Doebler. 2026-04-14. Cross-Course Generalizability of SRL-Aligned Predictive Models Using Digital Learning Traces. https://arxiv.org/abs/2604.22812

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Political Persuasion and Endorsement in Large Language Models

Large Language Models (LLMs) are increasingly employed as proxies for human behavior in computational social science. However, their tendency to internalize biases from training data raises concerns about their reliability in politically sensitive domains, specifically regarding their endorsement of persuasive language. In this work, we examine whether LLMs endorse persuasion-infused messages and whether partisan persona prompting modulates such endorsement. We evaluate six LLMs from different geographic regions on content annotated with persuasion techniques drawn from real-world media sources, measuring the likelihood of endorsement using a five-point Likert scale. The models are prompted as either a neutral social media user or as a user with left- or right-leaning political views. Results show that, without political conditioning, LLMs generally do not endorse messages containing persuasion techniques, although model-level differences emerge, and that partisan persona prompting increases polarization of endorsement, particularly for persuasion-infused content. Endorsement further varies across persuasion techniques and topics. These findings reveal how political conditioning influences LLMs' endorsement behavior toward persuasion-infused content, raising concerns about their deployment as agents in politically sensitive environments and complicating their use as reliable simulators of human political cognition.

cs.CY↗
arXiv2026

When Technically Plausible Advice Is Unsafe: A Cross-Ecosystem Measurement of Online Support for Technology-Facilitated Abuse

Technology-facilitated abuse (TFA) creates an adversarial setting where sound cybersecurity advice can be unsafe: changing credentials or resetting devices may alert an abuser, destroy evidence, or increase escalation risk. Victims seek guidance from search engines, peer forums, and conversational AI, often evaluated for relevance and correctness rather than contextual safety. We measure whether these sources meet victims' needs. From a decade of r/Stalking narratives, we construct 2,797 victim-derived queries spanning 11 misuse categories. We analyze 27,162 Google webpages, 2,476 Reddit query--thread responses, and 250 responses from three general-purpose LLMs and two survivor-support chatbots. Our framework measures technical quality and damaging guidance, plus secondary-link integrity on webpages, toxicity on Reddit, and trauma-informed support in conversational systems. We find failures & risks that relevance, accuracy, or actionability alone do not capture. Web Search and conversational systems frequently return relevant information; Reddit responses are less consistently relevant and actionable. In our evaluated accuracy sample, 17.3% of webpages, 13.3% of Reddit threads, and 19.6% of conversational AI responses contained damaging guidance. Further, 65.5% of victim queries led to a webpage with a secondary URL flagged by multiple VirusTotal engines, over 20% received a toxic Reddit comment, and every conversational system produced guidance that overlooked escalation risk. Specialization did not guarantee better support: HopeChat underperformed general-purpose LLMs on several dimensions, while Ruth remained limited in trauma-informed support. These findings expose a gap between technical quality and contextual safety. Safe TFA assistance requires risk-aware recommendations, trustworthy sources, uncertainty communication, and human support, beyond technically plausible answers.

cs.CY↗
arXiv2026

Auditing Institutional Heterogeneity for Generative AI in Patient Education: A Large-Scale Study of 102 US Transplant Handbooks

Health systems are rapidly deploying generative-AI assistants that answer patient questions from institution-authored education materials, on the premise that grounding in local content yields consistent guidance. Do the underlying documents themselves agree? We use a structured-output large-language-model judge to audit 1{,}772{,}261 pairwise comparisons across 102 patient-education handbooks from 23 US solid-organ transplant centers, paired with 1{,}115 patient-derived questions (TransplantQA). Four findings bear directly on deployment: (1) same-center cross-organ agreement exceeds cross-center same-organ agreement by $0.024$ in the primary analysis (Holm-adjusted $p=0.011$), with sensitivity to document selection; (2) information gaps concern topics relevant to underrepresented subgroups, with reproductive health a \emph{double jeopardy}: 82\% absence and 86\% judge-rated high significance among divergent/contradictory pairs; (3) judge-derived themes form 991 clusters, with immunosuppression and pregnancy timing among the highest judge-rated priorities; (4) question and observed-coverage features predict high-divergence questions retrospectively (AUC $0.77$). We discuss implications for deploying patient-facing generative AI in transplant care.

cs.CY↗