AUTHOR=Wulff Peter , Westphal Andrea , Mientus Lukas , Nowak Anna , Borowski Andreas 

TITLE=Enhancing writing analytics in science education research with machine learning and natural language processing—Formative assessment of science and non-science preservice teachers’ written reflections

JOURNAL=Frontiers in Education

VOLUME=Volume 7 - 2022

YEAR=2023

URL=https://www.frontiersin.org/journals/education/articles/10.3389/feduc.2022.1061461

DOI=10.3389/feduc.2022.1061461

ISSN=2504-284X

ABSTRACT=Science educators use writing assignments to assess competencies and facilitate learning processes such as conceptual understanding or reflective thinking. Writing assignments are typically scored with holistic, summative coding rubrics. This, however, is not very responsive to the more fine-grained features of text composition and represented knowledge in texts, which might be more relevant for adaptive guidance and writing-to-learn interventions. In this study we examine potentials of machine learning (ML) in combination with natural language processing (NLP) to provide means for analytic, formative assessment of written reflections in science teacher education. ML and NLP are used to cluster physics and non-physics teachers’ written reports on a standardized teaching vignette with regards to higher-level reasoning and represented knowledge. Results indicate that ML and NLP can be used to filter higher-level reasoning elements in physics and non-physics preservice teachers’ written reflections. Furthermore, a clustering approach yields specific topics in the written reflections that indicate quality differences in physics and non-physics preservice teachers’ texts. Hence, we argue, analytic, formative assessment with ML and NLP is possible and can provide science education scholars valuable tools to develop robust and principled means for automated writing analytics.