Actionable Insights from Observational Data: The Case of Advanced Classes in K-12 Education

Abstract

A fundamentally challenging question in K-12 education is about the effects of taking more advanced or challenging classes. It is particularly complex because students (and/or their parents) choose whether to enroll in these classes, making causal analysis challenging. In this paper, we begin to tackle this question by taking advantage of a novel dataset from a public school system in the US. This dataset records students' course enrollment decisions, prior academic histories, demographics, and subsequent outcomes around the time of a district-wide change that introduced optional open-enrollment advanced middle-school courses in subject areas. This is a rich observational dataset, but enrollment in advanced classes is driven by student characteristics and choices rather than random assignment. This creates a core identification challenge: the same factors that influence enrollment in advanced courses are also predictive of academic outcomes. As a result, simple comparisons between enrolled and non-enrolled students are confounded, and naive estimates may reflect underlying differences in student ability, motivation, or support rather than the impact of coursework itself. Our analysis shows that enrolling in advanced English courses has a net positive but modest effect on student achievement outcomes. However, these benefits are unevenly distributed: some students with relatively large predicted gains ("middle achievers" in prior years) are less likely to enroll than others. Some other groups (e.g. Black students and those with lower socio-economic status) also demonstrate significantly lower propensity to enroll. This gap between predicted benefit and observed enrollment illustrates how careful data analysis can extract actionable insights from large observational datasets, including identifying students who appear well-positioned to benefit but do not select into advanced options.