Enhancing writing analytics in science education research with machine learning and natural language processing—Formative assessment of science and non-science preservice teachers’ written reflections - Wulff, Peter; Westphal, Andrea; Mientus, Lukas; Nowak, Anna; Borowski, Andreas (Frontiers in Education, 2023)
Science educators use writing assignments to assess competencies and facilitate learning processes such as conceptual understanding or reflective thinking. Writing assignments are typically scored with holistic, summative coding rubrics. This, however, is not very responsive to the more fine-grained features of text composition and represented knowledge in texts, which might be more relevant for adaptive guidance and writing-to-learn interventions. In this study we examine potentials of machine learning (ML) in combination with natural language processing (NLP) to provide means for analytic, formative assessment of written reflections in science teacher education. ML and NLP are used to cluster physics and non-physics teachers’ written reports on a standardized teaching vignette with regards to higher-level reasoning and represented knowledge. Results indicate that ML and NLP can be used to filter higher-level reasoning elements in physics and non-physics preservice teachers’ written reflections. Furthermore, a clustering approach yields specific topics in the written reflections that indicate quality differences in physics and non-physics preservice teachers’ texts. Hence, we argue, analytic, formative assessment with ML and NLP is possible and can provide science education scholars valuable tools to develop robust and principled means for automated writing analytics.