AI-Guided Assignments in Practice: Students' Experiences in a Large Introductory Statistics Course

Abstract

Large language models (LLMs) raise new questions for instructors, among them how to integrate LLMs productively into coursework and what to evaluate in students' work when an LLM can produce much of it directly. LLM-guided homework assignments offer one answer. Instructors configure the LLM to guide students in ways that achieve specific pedagogical goals, such as recalling material, discovering new concepts, or critically evaluating deliberately flawed reasoning, and grade the full student--LLM conversation, and with it the student's reasoning, rather than only the final answer. In this paper, we report on the large-scale deployment of such assignments, implemented through LLteacher, an open-source tool that supports recall, discovery, and critical-thinking assignments, in a large introductory statistics course serving mostly pre-major undergraduates. Results on 134 students show meaningful gains in self-reported confidence, with over 80\% agreeing that the assignments improved their comprehension of course material and their ability to interpret statistical results; gains were largest among students who started with lower confidence, suggesting this approach may help narrow gaps in preparation. Interaction patterns varied widely, from long single messages to rapid iterative exchanges, and hallucinations, when present, were rarely disruptive to learning.