An Empirical Study on how Computing Students Interact with Large Language Models for Learning Purposes

Abstract

Large Language Models (LLM) have become popular tools in education. Therefore, understanding how students interact with LLMs for learning purposes is an important aspect of the responsible design and development of LLM based learning tools. However, existing research has focused on evaluating model performance, specific use cases, and students' attitudes toward LLM in education. There has been limited empirical work exploring students' interaction strategies with LLMs in computing education. This study draws on Self Regulated Learning (SRL) theory and conducts a survey of 225 undergraduate and graduate computing students to investigate how students use LLM tools during learning, the perceived learning impact, and the strategies they rely upon when using these tools. Students used LLMs across nine functional categories, most often to explain concepts, learn new material, find information, debug, and generate ideas. They also adopted some strategies such as supplying detailed context and iterative prompting. The most common difficulties encountered with LLM tools were incorrect information, generic responses, and difficulty trusting the model's reasoning. We synthesized these patterns through an SRL lens and offer a Probe Monitor Shape heuristic to map students' LLM interaction strategies onto the phases of self-regulated learning.