Integrating a Statistical Topic Model and a Diagnostic Classification Model for Analyzing Items in a Mixed Format Assessment - Choi, Hye-Jeong; Kim, Seohyun; Cohen, Allan S.; Templin, Jonathan; Copur-Gencturk, Yasemin (Frontiers in Psychology, 2021)
It is common for assessments in education to contain both selected response items and constructed response items in a mixed format. These two different formats may complement each other. A constructed response item requires a longer, more detailed response from the students. To date, however, little research has been conducted on students’ textual response to the constructed response items. The purpose of this study was to present an approach in which a diagnostic classification model was used with a statistical topic model. We demonstrate how to apply this approach for a formative test of English and Language Arts. The test was designed to measure students’ reading and writing skills. A diagnostic classification model was used to estimate students’ mastery status of reading skills and topic models were used to estimate students’ use of the latent topics in their writing. Further, we used this approach to investigate the relationships between students’ reading skills and students’ use of topics in writing. Topic modeling is relatively new to educational researchers. The current study showed it provided a useful set of methods for extracting this added information in the text of answers to constructed response items.