Development and Examination of a Tool to Assess Score Report Quality - Roduta Roberts, Mary; Gotch, Chad M. (Frontiers in Education, 2019)
The need for quality in score reporting practices is represented in the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014). The purpose of this study was to introduce a ratings-based instrument to assess the quality of score reports and examine the reliability of scores obtained. Quality criteria were derived from best-practices published within the literature (Hambleton & Zenisky, 2013). The rating scale was used to assess a sample of 40 English-language individual student score reports for elementary-level accountability tests representing 42 states and 5 provinces in the United States and Canada. A two-facet generalizability study (i.e., sr x d x r) was completed with an overall reliability coefficient of G=0.78. Application of the rating scale may provide a means to support empirical study of relationships between score report quality and stakeholder outcomes including interpretation, use, and impact.