Comparing Two Subjective Rating Scales Assessing Cognitive Load During Technology-Enhanced STEM Laboratory Courses - Thees, Michael; Kapp, Sebastian; Altmeyer, Kristin; Malone, Sarah; Brünken, Roland; Kuhn, Jochen (Frontiers in Education, 2021)
Cognitive Load Theory is considered universally applicable to all kinds of learning scenarios. However, instead of a universal method for measuring cognitive load that suits different learning contexts or target groups, there is a great variety of assessment approaches. Particularly common are subjective rating scales, which even allow for measuring the three assumed three types of load in a differentiated way. Although these scales have been proven effective for various learning tasks, they might not be an optimal fit for the learning demands of very complex environments like technology-enhanced STEM laboratory courses. The aim of this research was therefore to examine and compare existing rating scales in terms of validity for such a learning context and to identify options for adaptation, if necessary. For the present study, the two most common subjective rating scales that are known to differentiate between load types were slightly adapted to the context of learning through structured hands-on experimentation where elements like measurement data, experimental setups and experimental tasks affect knowledge acquisition. N=95 engineering students performed six experiments examining basic electric circuits where they had to explore fundamental relationships between physical quantities based on observed data. Immediately after experimentation, students answered both adapted scales. Various indicators of validity were analyzed, which considered the scales’ internal structure and their relation to variables like group allocation as participants were randomly assigned to two conditions with contrasting spatial arrangement of measurement data. For the given data set, the intended three-factorial structure could not be confirmed and most of the a priori defined subscales showed insufficient internal consistency. A multitrait-multimethod-analysis suggests convergent and discriminant evidence between the scales which could not be confirmed sufficiently. The contrasted presentation formats were expected to result in different ratings for extraneous load, which was solely detected by one adapted scale. As a further step, two new scales were assembled based on the overall item pool and the given data set. They revealed a three-factorial structure in accordance with the three types of load and seem to be promising new tools, although their subscales for extraneous load still suffer from low reliability scores.