Discovering Misconceptions and Misunderstandings From Administrations of Research-Designed Multiple Choice Instruments

Abstract

Misconceptions are "alternate hypotheses" that are incorrect according to established theories of how the world works. Often held with confidence by students, they are relatively context-insensitive, can seem like common-sense views, and are noted for being resistant to remediation using traditional instruction. To find misconceptions in Newtonian mechanics, we analyze ~34,000 administrations of the pioneering Force Concept Inventory using a flexible multidimensional item-response model for multiple-choice data. In contrast to most earlier work, we allow answer choices within each question to have different directions in the multidimensional space of student knowledge, essential for concept inventories in which distractors often codify distinct misconceptions. We uncover 22 robust, partly-overlapping dimensions whose distractors share a coherent theme identifiable with a misconception or misunderstanding. Motivated by the realization that many mirror previously-accepted theories of mechanics, we broadly sort these by historical era: Ancient (learned by infants but codified by Greeks), Medieval (reactions and extensions of Aristotelian ideas), and Post-Newtonian (including known modern misconceptions as well as two which appear novel). We also present a simple approach for computing "misconception scores" for students and classes. Examining these scores before and after instruction reveals surprisingly varied patterns of remediation in our sample: some misconceptions persist largely unchanged by instruction, while others are better remediated in below- or above-average students. In general, we find that many misconceptions are poorly remediated for students of average or lower ability. We hope our work will serve as a guide for developing, evaluating, and improving interventions for these while providing physics instructors with a valuable tool for class-level formative assessment.