Assessment literacy (AL) has become one of the essential components of teachers to overcome educational assessment challenges in the twenty-first century globally. In general, an assessment-literate teacher (Popham, 2009) should be able to understand and differentiate both “declarative knowledge (‘know-what’) and procedural knowledge (‘know-how’)” (Koh, 2019, p. 21) of educational assessment, which eventually could “influence educational decisions” (Popham, 2011, p. 267). Despite the numerous paradigms, methodologies, and frameworks that constitute the connotation of AL, its practical applications for teachers are primarily separated into two genres: summative assessment (SA) and formative assessment (FA) (Lau, 2016). Koh et al. (2018) explained that the term “literacy” is “used to indicate an individual’s knowledge or competency in a specified subject area” (p. 265). Accordingly, summative/formative assessment literacy (S/FAL) is a sub-concept of AL that refers to a teacher’s understanding and practices of these assessment methods.

SA is a traditional evaluation method that uses rigorous standards, rubrics, or criteria to determine a final score or grade, typically at the end of a learning period, and it is often high-stakes, focusing on a limited number of cognitive outcomes (Lopez and Pasquini, 2017). Although SA is valuable for accountability purposes such as certification, academic ranking, and determining progression, relying on it excessively can lead to “teaching to the test” (Popham, 2001, p. 16) for teachers. As such, scholars such as Black and Wiliam (1998) called for harnessing FA, instead of SA, to hypostatize and promote teachers’ AL in the classroom, as FA has a powerful improvement orientation for student learning (Popham, 2009). As “assessment for learning” is a more comprehensive concept including “formative assessment” (e.g., Bennett, 2011; Klenowski, 2009), we treated them as equivalent terms in the literature and employed the more precise phrase “formative assessment (FA)” herein.

Within the characteristics of FA practice, it is about the process of assessment, with an emphasis on the trajectory of assessments rather than the culmination in a product, and it engages students as collaborators and co-navigators within the assessment landscape to improve learning ultimately (Bennett and Gitomer, 2009). In short, FA is a dynamic and ongoing process (Popham, 2009). Second, FA is predicated on evidence. The inception of FA, to a certain degree, stemmed from the recognition that teachers were unable to furnish adequate evidence through a singular examination (Stiggins and Chappuis, 2006). Black and Wiliam (2009) advocated that teachers ought to orchestrate a diverse array of assessment relating tasks in practice, such as well-designed classroom mutual dialogues, with the aim of eliciting evidence of student acquisition and achievement. The Assessment Reform Group (2002) pointed out the value of using evidence in education. Teachers and students can seek and interpret such evidence to determine learners’ current position in their educational journey. This information helps them identify the objectives they should pursue and plan the most effective strategies to achieve these goals.

CHC regions commonly refer to China, Japan, Korea, Vietnam, Malaysia, and Singapore, where societies historically influenced by Confucian values, ethics, and social norms (Paramore, 2015). According to Leong et al. (2018) review on FA research, K–12 teachers in the CHC societies are facing complex and significant challenges in implementing FA and developing FAL. Compared to their counterparts in Western classrooms, these struggles stemmed from several fundamental factors, most notably limited FAL (Ratnam-Lim and Tan, 2015). Additionally, in these contexts, high-stakes SA policies, particularly final examinations, remain predominant, e.g., China (Yan and Brown, 2021), Japan (Kuramoto and Koizumi, 2018), Singapore and Malaysia (Koh et al., 2018). Although the current study focused on the context of mainland China, the insights from other CHC regions can shed light on the core dilemmas of implementing FA and developing FAL in mainland China.

Among these factors, policy frameworks play a particularly decisive role in shaping teachers’ opportunities to develop FAL. In terms of legislation, high-stakes summative examinations continue to dominate China’s assessment landscape (Yan and Brown, 2021), though the government has promulgated reforms to reduce this reliance. First, government legislative documents tend to use the Chinese term “评价” (judgment), rather than “评估” (assessment/evaluation) (e.g., Ministry of Education, 2020). This terminology inherently violates the principle of teacher-student parity, a concept fundamental to understanding FA (Heritage and Wylie, 2018). Tan (2018) concluded that, at the policy level, Chinese educational reforms on assessment have had limited success. Ratnam-Lim and Tan (2015) pointed out this is partly because teachers, as policy actors, may respond in diverse ways depending on individual conceptions, skills, attitudes, understanding of the policy’s intent, and willingness to change. Additionally, Liu and Onwuegbuzie (2012) identified three major pressures on Chinese teachers, all tied to SA: high expectations to raise test scores, over-reliance from students and parents for score improvement, and the critical use of examination results in teacher evaluations in school. Consequently, teachers, students, and parents tend to overlook alternative assessment methods but put exclusive emphasis on SA results. Moreover, the dominance of high-stakes examination policies may restrict teachers’ opportunities to develop FAL, as their professional focus remains tied to test development and score improvement.

Beyond policy constraints, traditional socio-cultural influences from Confucianism represented another major barrier in the FA dilemma in China. Ratnam-Lim and Tan (2015) contended that the highly authoritative nature of the teacher-student relationship, rooted in the cultural mindset of respecting teachers, fundamentally conflicts with the equality principle of FA. Originally, it is the filial piety (obedience) from juniors (children/students) to seniors (parents/teachers), a key principle of traditional Confucian values, serves as a socio-cultural barrier (Guo and Cao, 2024). Within this mindset, teachers are often framed as knowledge transmitters through a teacher-centred approach, exemplified by the Virtuoso Model or as gardeners nurturing student growth with a sense of sanctity (Guo, 2005). Additionally, after the Reform and Opening-up in 1978, Socialism with Chinese Characteristics integrated the Confucian concept of He (和, harmony). In classroom practice, this often manifested as conformity, leading to passive student engagement and overlooking individual differences (Tan, 2018). As a result, Chinese teachers often have fewer opportunities to acknowledge and address classroom diversity and provide equitable engagement in FA (DeLuca and Lam, 2014) and undermine the positive attitudes required for FAL.

Lastly, in Chinese classroom practices, FA initially garnered attention primarily within the domain of English language teaching in the early 2000s and in the context of university (currently, it remains the predominant area of studies about FA in China) (e.g., Xu and Zhang, 2020). Consequently, FA practices and teachers’ FAL in other K–12 disciplines have remained under-researched.

Over the past three decades, numerous quantitative scales, questionnaires, inventories, and other instruments have been developed and validated for researching topics related to AL in Western classrooms (e.g., DeLuca et al., 2016; Plake et al., 1993). These instruments included limited items of FAL as one of the dimensions of AL. Further, DeLuca et al. (2016) argued that most AL instruments are outdated as they were based on early conceptions of AL that did not reflect significant changes in assessment principles, particularly those concerning FA.

Moreover, from a global perspective, few instruments were specifically designed to measure the different domains of FA from the perspectives of K–12 teachers. Among them, Yan and Pastore (2022a,b) developed and validated two FA scales: the Teacher Formative Assessment Literacy Scale (TFALS) and the Teacher Formative Assessment Practice Scale (TFAPS). Their findings from K–12 teachers in Hong Kong significantly shed light on FA practices in Asia. A limitation is that these two scales lack items designed specifically for the CHC context. This is because Hong Kong would represent a hybrid context shaped by both Confucian and British educational traditions. Although it is often viewed as an example of CHC and a high-stakes examination region, Hong Kong teachers’ FAL may differ from those in other traditional CHC contexts. This divergence is largely due to the widespread implementation of school-based assessment in Hong Kong, which was introduced to mitigate the dominance of high-stakes examinations (Xiao et al., 2023). Additionally, the items in these two scales had a Westernized wording, making it difficult to adapt them in a way that local teachers in mainland China could easily understand, such as the Western concept of “ethics” and “studnet right,” which may not have a direct equivalent or may be understood differently. Further, teachers’ communication with parents about student learning, another important scenario in Chinese schools regarded as an integral duty of teachers, has not been yet included in previous survey scales either.

Furthermore, in mainland China, there has been limited effort to develop questionnaires for FA. Wang (2017) designed the English Interpretation Competency Scale which included elements of FAL but was not designed to measure FAL comprehensively. Simultaneously, Guo and Xu (2021) developed a FA questionnaire specifically for university students’ English as a Foreign Language writing. These two instruments were solely aimed at investigating students’ English proficiency within the university context. Additionally, Zou et al. (2021) designed a Likert-style questionnaire using ICT tools/platforms for FA during the COVID-19 pandemic, which was heavily technology-oriented and lacked cultural elements. Although these instruments cannot be directly applied to survey Chinese teachers’ FAL, they provided valuable methodological references for designing the questionnaire in this study.

The participants for the study were K–12 teachers teaching across all subjects from 17 public schools in Lingang, Shanghai, China. Teachers from three schools (one primary, one secondary, and one high school) took part in the pilot study, while teachers from the remaining 14 schools (five primary, five secondary, and four high schools) participated in the main study. This study was approved by the research ethics committee of our affiliated university, and all participants provided informed consents before participating in the survey.

Drawing on the literature, the Formative Assessment Literacy Questionnaire (FALQ) was designed, developed, and validated based on Pastore and Andrade’s (2019) three-dimensional framework consisting of Conceptual, Practical, and Socio-emotional dimensions. The Conceptual dimension refers to the conceptions a teacher holds about FA, the Practical dimension emphasizes integrating the FA process with teaching practices to monitor, evaluate, and manage the teaching-learning process, and the Socio-emotional dimension involves managing the social and emotional aspects of FA. The items were developed and categorized according to keywords aligned with each dimension. The response format of the FALQ employed a five-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). The questionnaire consisted of 30 items, with 10 items allocated to each of the three dimensions (see Supplementary Appendix). In addition to the main body of the FALQ, two items were designed to collect participants’ demographic information about school type and teaching subject.

To ensure content validity, a panel of four experts, including two professors in psychometrics and two Chinese K–12 teachers (one experienced and one novice), reviewed the questionnaire. They evaluated the clarity, relevance, and wording of the items, as well as the participant instructions. Each item was rated on a four-point scale: 0 (needs removal/replacement), 1 (needs major revision), 2 (needs minor revision), or 3 (no revision needed) based on two criteria: (1) clarity and (2) essentiality. Additionally, both in-service teachers confirmed that the Chinese edition of the questionnaire is easily readable for regular in-service teachers.

For clarity, the average scores of items in the FALQ were 2.81 out of 3, while for essentiality, the averages were 2.93 out of 3, indicating strong overall quality. The majority of items achieved the highest rating (3 points), with no items deemed unsuitable (0 points). The items scoring 1 (requiring major revision) were prioritized for refinement, either through rewording or scenario adjustments, thereby enhancing the validity of the instrument. This procedure can strengthen the clarity, relevance, and validity of the FALQ before the pilot study.

The online link of the questionnaire on Qualtrics was distributed to approximately 160 teachers across three schools. Of these, 101 teachers completed the questionnaire. After removing missing or invalid responses, such as incomplete submissions or perfunctory responses where the same answer was given for all items, 96 valid responses remained for analysis. The participants (N = 96) included 31 primary school teachers, 33 secondary school teachers, and 32 high school teachers. Most school subjects were covered: Chinese Language (N = 26), English (N = 28), mathematics (N = 17), sciences (e.g., physics) (N = 12), social sciences (e.g., history) (N = 9), and arts and physical education (N = 4). The mean FALQ score was 118 (SD = 15.17) out of a maximal score of 150 (78.67% scoring rate).

According to Nunnally and Bernstein (1994), Cronbach’s alpha coefficient value of 0.70 was deemed to be acceptable for reliability in survey research. In this study, the Cronbach’s alpha for FALQ was ideal (= 0.90). Thus, the internal consistency reliability of the questionnaire data provided a solid foundation for the subsequent analyses.

Although the questionnaire structure and items were based on the Conceptual–Practical–Socio-emotional framework, it was necessary to conduct an Exploratory Factor Analysis (EFA) to distill the data and obtain a more accurate picture of the factors first, as the framework remained brief and insufficiently supported by empirical studies. This analysis is crucial for assessing the construct validity of the questionnaire (Fabrigar and Wegener, 2012). A small sample size of at least 50 and no more than 100 subjects is generally considered adequate for conducting EFA (Sapnas and Zeller, 2002). Additionally, while some statisticians have suggested various rules for the subject-to-item ratio, such as 3:1, Hogarty et al. (2005) pointed out that there is no specific minimum ratio required to achieve valid outcomes.

The data analysis software was Mplus 8.10 (Muthén et al., 2018). The value for Kaiser–Meyer–Olkin of Sampling Adequacy (KMO) was 0.905, and the Bartlett’s Test of Sphericity showed χ2 (96) = 3601.364, p = 0.000. A KMO index greater than 0.900 is considered highly suitable for factor analysis, while Bartlett’s Test of Sphericity should be significant (p < 0.05) (Williams et al., 2010). Therefore, the data of FALQ were suitable for EFA.

The EFA was conducted using maximum likelihood extraction with oblique rotation. The EFA yielded four latent factors (see Table 1). Factor retention was determined primarily through parallel analysis, which supported a four-factor solution, and was further corroborated by eigenvalues > 1.0. Item evaluation was based on factor loadings, communalities, and cross-loadings to ensure a clear and interpretable factor structure. Items were retained if they demonstrated a primary factor loading of at least 0.45 and were conceptually aligned with the corresponding factor. In addition, all items exhibited substantial communalities, ranging from 0.637 to 0.891, exceeding the commonly recommended threshold of 0.40 (Hair et al., 2018). These results indicated that a substantial proportion of variance in each item is explained by the extracted latent constructs.

Within the Conceptual dimension, the items clustered into two sub-dimensions based on their factor loadings and thematic content: (1) Teachers’ self-efficacy and (2) Knowledge for collaboration with students. Bandura (1997) explained that self-efficacy represents a psychological construct distinct from fundamental knowledge, while Black and Wiliam (2009) emphasized the critical role of teachers’ knowledge in fostering collaborative practices. This evidence supported distinguishing the two sub-dimensions within the broader Conceptual domain. Further, most of the items indicated significance in EFA, except Q18 Facilitating peer feedback (the only item showing a notable cross-loading), which was slightly controversial as it indicated similar values in the Practical dimension (= 0.479) and the Socio-emotional dimension (= 0.538). This item was about facilitating peer feedback, which was originally categorized in the Practical dimension. Upon scrutiny, Q18 Facilitating peer feedback was a practical question but can also be viewed as a question of sociocultural perspective, as peer feedback largely depends on factors such as the teacher’s role in facilitation and students’ attitudes (Ng and Yu, 2021). The Practical dimension included items focused on assessment activities with specific actions (Q18 used the key verb “facilitate”), whereas the Socio-emotional dimension comprised items reflecting abstract psychological concepts. Finally, to maintain consistency, it was better to place Q18 Facilitating peer feedback into the Practical dimension. Additionally, the dimension comprising items Q11–Q17, Q19 and Q20 aligned with the Practical dimension, and items Q21–Q30 consistently fell within the Socio-emotional dimension.

To verify the validity of psychometric research scale, the Rasch Model is often harnessed as one primary method in the related fields. In this study, we used the software Winsteps (Linacre, 2024) for the Rasch analysis of the FALQ at the overall scale level. Rasch analysis supported essential unidimensionality of the overall scale as indicated by Principal Component Analysis of Residuals, with the first contrast eigenvalue < 2.0. This suggests that the four dimensions identified through EFA are underpinned by a common latent construct. Additionally, local independence was confirmed, as all item residual correlations were < 0.30 (Linacre, 2024). Figure 1 presented the item-person maps, displaying person measures and item difficulties on a logit scale, with logit positions indicated on the sides like a ruler. All logit positions ranged from −2.5 to +3, within the acceptable range of −3 to +3 (Engelhard and Wang, 2021), indicating an appropriate spread of item difficulty and person ability along the latent continuum. This distribution suggests that the scale functions well across the targeted range of the construct. Figure 1 also showed that, while the distribution of persons’ FAL scores was wide, item difficulties were slightly skewed toward low to moderate levels, with limited coverage for higher FAL scores. This likely reflects both the characteristics of the pilot sample and the preliminary stage of the questionnaire, a phenomenon commonly observed in pilot testing (Bond et al., 2020). The rating scale functioning was validated by ordered category thresholds, showing a monotonic increase in difficulty across response options. Overall, the alignment between person ability and item difficulty (ranging primarily from −2.5 to +3 logits) indicates good targeting and minimal standard error for the target population.

The results of FALQ well fitted to the Rasch Model (Table 2): Infit MNSQ = 0.97; Outfit MNSQ = 0.89, which fall within the acceptable range of 0.5 to 1.5 (Bond et al., 2020). Each item’s measure value represented its level of difficulty. The total logits for FALQ (= 0.27) were positive and above zero, indicating that the overall difficulty was appropriate and capable of effectively assessing the participants’ actual proficiency (Andrich, 1988). However, some items indicated misfit: Q13 Feedback for different scenarios (= 0.48), Q17 Further suggestions (= 0.46), Q18 Facilitating peer feedback (= 0.44), Q19 Identifying strengths & weaknesses (= 0.41) and Q20 Comparing student own performances (= 0.31) were slightly lower than the acceptable range. As such, we excluded these items from the main study. In summary, the original questionnaire items generally fitted the Rasch model, although several items showed misfit. Given that it was a pilot study conducted in an uncharted area with a small sample size, the result of Rasch Analysis can still be considered acceptable (Bond et al., 2020).

In the main study, the revised FALQ was distributed via a Qualtrics online link to approximately 600 teachers across 14 public K–12 schools in Lingang. Of these, 411 teachers completed the questionnaires (65% participation rate), indicating a good return rate for online survey. However, after cleaning for erroneous, missing, or incomplete responses, 309 valid responses (N = 309) remained for analysis. The inclusion criteria required responses to be complete and submitted within an appropriate time frame of at least 10 min, reflecting a normal pace and helping to avoid perfunctory responses. Among them, 100 participants were from primary schools, 128 from secondary schools, and 81 from high schools. The participants taught a range of K–12 subjects, including Chinese Language (N = 76), English (N = 59), mathematics (N = 55), sciences (e.g., physics) (N = 40), social sciences (e.g., history) (N = 40), and other subjects (fine arts, physical education, computer science) (N = 39), representing the primary subjects taught in Chinese K–12 schools.

The mean overall score for FALQ was 102.55 out of 125 (SD = 13.69). By extension, the mean scores for each factor within the FALQ were as follows: Teachers’ self-efficacy: 15.03 out of 20 (SD = 3.096), Knowledge for collaboration with students: 25.25 out of 30 (SD = 3.617), Practical: 20.42 out of 25 (SD = 3.042) and Socio-emotional: 41.84 out of 50 (SD = 13.686).

In the main study, Cronbach’s alpha coefficient for the FALQ was ideal (α = 0.877), closely aligning with the pilot study result.

The value for Kaiser–Meyer–Olkin of Sampling Adequacy (KMO) was 0.856, and the Bartlett’s Test of Sphericity showed χ2 (309) = 1398.195, p = 0.000, indicating that FALQ was suitable for Confirmatory Factor Analysis (CFA). The initial findings of CFA for the four-factor model indicated that some fit statistics were satisfactory: CFI = 0.910 and TLI = 0.900 (≥ 0.9000); however, other statistics were less than ideal: RMSEA = 0.093 (exceeding the ideal goodness-of-fit threshold of ≤ 0.08) and SRMR = 0.064 (exceeding the ideal threshold of ≤ 0.05) (Hu and Bentler, 1999).

To address the suboptimal model fit, the factor structure was refined following a theory-driven and incremental approach. Modification indices were examined to identify areas of misfit; however, no correlated error terms were added, and no parameters were freed solely to improve fit. Across five theoretically plausible alternative models with restricted factor organizations (Chan and Luo, 2021), CFA results consistently suggested that the 10 Socio-emotional items reflected two interrelated but distinct sub-dimensions: Concerning student emotions (six items) and Respect for students (four items). This decision was informed not only by item factor loadings and content themes, but also by socio-cultural perspectives on teacher-student interactions that highlight the importance of emotional responsiveness and student dignity in China (Chan and Luo, 2021; Gan et al., 2019).

Then we conducted a CFA with this five-factor model: (1) Teachers’ self-efficacy, (2) Knowledge for collaboration with students, (3) Practical, (4) Concerning student emotions and (5) Respect for students. The outcomes demonstrated improved and acceptable model fit between the hypothesized model and the observed data: RMSEA = 0.078, CFI = 0.926, TLI = 0.916 and SRMR = 0.052. Although the SRMR value (= 0.052) slightly exceeded the common cutoff of 0.05, it remained within an acceptable range when considered alongside other fit indices (Kline, 2010). Additionally, each item indicated a high factor loading in the five-factor model (Figure 2). All factor loadings ranged from 0.598 to 0.933, indicating strong associations with their respective latent constructs (Gorsuch, 2015). Given the improvement in model fit and the enhanced conceptual clarity afforded by distinguishing Concerning student emotions and Respect for students, the five-factor model was deemed more appropriate than the original four-factor structure. Taken together, the CFA results provided strong evidence for the construct validity of the FALQ in the main study.

After revision, the mean scores for the new sub-dimensions were: Concerning student emotions: 25.10 out of 30 (SD = 9.4) and Respect for students: 16.77 out of 20 (SD = 6.3).