Teachers’ professional competence can be understood as a developmental construct encompassing cognitive and affective dispositions and skills necessary for effective teaching (Baumgartner, 2022; Kaiser and König, 2019; Kaiser et al., 2025). The ongoing debate in educational measurements about how teachers’ professional competence can be assessed has, in recent years, been shaped by integrative approaches that seek to combine the analytic measurement of aspects of professional competency with situation-specific skills and behavioral performance indicators (Baumgartner, 2018; Baumgartner, 2022; Blömeke et al., 2015; Krauss et al., 2020; Seidel et al., 2024).

A widely cited competence structure model in this context is that of Blömeke et al. (2015). This model, along with its adaptation and extension for the field of PE (Baumgartner, 2022), serves as the theoretical framework for the development of the instrument within the WiPE-Sport research project (Baumgartner et al., 2025). The framework comprises three interrelated competence facets: (1) aspects of competency (i.e., dispositions such as professional knowledge; cf. Baumert and Kunter, 2013), (2) the situation-specific skills of PID, and (3) performance. Together, these facets form a continuum of teachers’ professional competence. When performance is conceptualized as the target facet, the aspects of competency and situation-specific PID skills are understood to contribute directly to it, whereas the facet of performance, in turn, supports the positive development of students (Baumgartner, 2022). Within this framework, the three facets are additionally connected to distinct competence-areas (e.g., CM, feedback, etc.; Baumgartner, 2022).

Although the different competence facets can be empirically distinguished, they are also interrelated and mutually reinforcing (Bauersfeld et al., 2025a; Dückers et al., 2025; Gold and Holodynski, 2017; König, 2015; König et al., 2025). Aspects of professional competency, such as CM-related knowledge, shape teachers’ perception and interpretation of classroom situations and enable top-down information processing (Berthold et al., 2025a; Dückers et al., 2025; Seidel et al., 2024). This leads to more efficient information search and improved sense-making (Wolff et al., 2021). Expert teachers, for instance, rely on highly interconnected and flexible cognitive schemata and scripts for CM (Wolff et al., 2021), which supports situation-specific PID skills to support student learning (Gold and Holodynski, 2017; Stahnke and Blömeke, 2021; Wolff et al., 2016, 2017). Research on teacher expertise has consistently shown that novice and expert teachers differ substantially in their professional vision¹ (e.g., Gold and Holodynski, 2017; Reuker, 2017).

The development of teachers’ professional competence is a central goal of teacher education (Kaiser and König, 2019). With regard to situation-specific PID skills, it is assumed that this competence facet evolves over time through a combination of (a) increasing domain- and competence area-specific knowledge, (b) more learning opportunities through authentic teaching experiences, and (c) (guided) reflection on one’s own or others’ teaching practice (e.g., through video analysis) at the interface of theory and practice (Barenthien et al., 2023; Bauersfeld et al., 2025b; Eßling et al., 2023; Junker et al., 2025; Kaiser et al., 2025; Stahnke and Blömeke, 2021; Stahnke et al., 2016; Weyers et al., 2024; Wolff et al., 2016, 2017).

In summary, we consider situation-specific PID skills to be a crucial facet of teachers’ professional competence that should be explicitly targeted and developed within teacher education. The test instrument presented and validated in this study serves, among other purposes, to diagnose pre-service PE teachers’ developmental levels in situation-specific PID skills within the competence-area of CM.

The CM in PE refers to the sum of teacher actions aimed at creating and maintaining a productive learning environment that supports students’ cognitive, social–emotional, and motor development (Baumgartner et al., 2020; Brophy, 2006; Cothran and Kulinna, 2015; Putra and Yanto, 2025). These actions involve both proactive and reactive strategies to maximize on-task behavior, support instructional flow, and prevent or respond to disruptions (Emmer and Stough, 2001; Doyle, 2006; Oliver et al., 2011).

The CM is considered a crucial competence-area for teachers, as it can significantly improve students’ attention, motivation, engagement, and learning (Korpershoek et al., 2016, 2025; Kunter et al., 2007; Putra and Yanto, 2025). Mastering CM also supports teachers’ health, as it can reduce stress and burnout (Aloe et al., 2014; König and Rothland, 2016; McCarthy et al., 2022), promotes teacher well-being (Aldrup et al., 2018), and job satisfaction (Dicke et al., 2015).

Although CM is a generic aspect of teaching and teaching quality (Praetorius et al., 2018), it poses distinct challenges in PE due to the subject’s dynamic, noisy, and physically active environment (Baumgartner et al., 2020; Cothran and Kulinna, 2015; Herrmann and Gerlach, 2020; Ryan and van der Mars, 2022). Teaching in spaces such as gymnasiums, outdoor fields, or swimming pools requires specialized rules, routines, and strategies to manage transitions, coordinate equipment, and ensure student safety (Chappell, 2020; Lawrence, 2020). Active supervision is crucial, requiring PE teachers to move strategically, maintain visibility, and engage frequently with students to manage behavior and maintain rapport (Arbogast and Chandler, 2005; Schuldheisz and van der Mars, 2001; van der Mars et al., 1994). PE teachers must also address the needs of non-participating students and enforce safety protocols appropriate to the physical demands of specific sports (Chappell, 2020).

To systematically account for these specific subject-matter requirements, we adopt the framework by Baumgartner et al. (2020). In this model, CM in PE is conceptualized as a multidimensional construct that explicitly integrates general pedagogical strategies adapted to the specific demands of PE (monitoring, clarity of announcements, smooth transitions, overlapping, group mobilization, momentum, and dealing with disturbances) alongside two distinct PE-specific dimensions: ensuring safety and managing equipment.

In the competence structure model proposed by Blömeke et al. (2015), the situation-specific PID skills refer to three distinct cognitive processes required to address professional demands of teaching in classroom situations. Applied here within both a competence-area specific (CM) and subject-specific (PE) context, these skills captures (pre-service) PE teachers’ ability to identify and perceive CM-relevant situations in PE, interpret them based on their CM-related knowledge, experience, and beliefs, and make decisions on whether to continue the current action or implement alternative courses of action to support student learning (Blömeke et al., 2015, 2022; Kaiser et al., 2015).

In developing the test instrument, we follow this threefold cognitive structure and therefore use the term CM-related PID to denote competence-area specific situation-specific PID skills. Alternative frameworks describe similar constructs under different labels, such as professional vision (e.g., Gold and Holodynski, 2017; Reuker, 2017) or teacher noticing (e.g., König et al., 2022; Langer et al., 2023). Although these approaches often refer to the same foundational work (e.g., Goodwin, 1994; van Es and Sherin, 2002), they differ in scope and emphasis—for instance, in whether decision-making is included as a cognitive process, or whether the focus is placed on perception (noticing) and interpretation (knowledge-based reasoning; see Amador et al., 2021; König et al., 2022; Schicklinski and Reuker, 2023; Sherin et al., 2024; Stahnke et al., 2016).

Based on the systematic literature review by König et al. (2022), our understanding of CM-related PID can be classified within a cognitive-psychological perspective. Accordingly, in developing our test instrument, we specifically address the measurement of the cognitive processes of pre-service PE teachers’ situation-specific skills. This theoretical focus can, inter alia, be distinguished from a socio-cultural perspective, which emphasizes the social and situated nature of teacher noticing (Goodwin, 1994; Sherin and van Es, 2008) and from an expertise-related perspective focusing on differences between experts and novices (Reuker, 2017; Roose et al., 2018).

Research has shown that CM-related PID represents a learnable competence facet that can be developed during teacher education (Bauersfeld et al., 2025b; Gold et al., 2013; Hellermann et al., 2015; Junker et al., 2025; Kramer et al., 2017; Orschulik, 2021; Weber et al., 2020; Weber et al., 2018) and expert teachers tend to have better CM-related PID than novice teachers (Stahnke and Blömeke, 2021; Wolff et al., 2016, 2017).

Studies grounded in the competence-as-a-continuum framework (Blömeke et al., 2015) indicate that CM-related knowledge is significantly linked to CM-related PID (Gippert et al., 2022; Gold and Holodynski, 2017; König, 2015; Weyers et al., 2024). This relationship is complex and depends on the type of knowledge assessed (declarative vs. procedural), the specific cognitive process within PID, and the level of professional development (pre-service vs. in-service; Junker et al., 2021; Weber et al., 2023). Declarative knowledge, for instance, supports the perception of relevant classroom events, whereas integrated conceptual knowledge appears essential for interpretation and decision-making (Junker et al., 2021). Moreover, there is evidence that CM-related PID predicts various dimensions of teaching quality and student learning outcomes (Blömeke et al., 2022; König and Kramer, 2016; König et al., 2021). However, the empirical evidence in this regard remains limited and inconsistent, as some studies have found no or only partial significant relationship between CM-related PID and performance (Baumgartner et al., 2025; Gold et al., 2021; Junker et al., 2021).

Overall, it can be concluded that situation-specific PID skills represent a theoretically and conceptually multifaceted construct that is highly relevant for teachers’ professional development in general and for CM competence in particular.

There are several validated instruments to measure situation-specific skills in a standardized way, ranging from analytical (e.g., van Es and Sherin, 2002) or holistic testing methods (e.g., Roose et al., 2018) to data collection approaches on action (e.g., video-based testing instruments, Gold and Holodynski, 2017) and in action (e.g., eye-tracking studies, van Driel et al., 2022). Among these, video-based instruments have become the most widely used (Gold et al., 2024; Weyers et al., 2023). They typically employ video clips of teaching practice as stimuli for closed- or open-ended items or writing prompts.

A broad range of instruments have been developed to assess CM-related PID employing diverse methodological approaches (Bauersfeld et al., 2025a; Gippert et al., 2022; Gold and Holodynski, 2017; Jamil et al., 2015; König, 2015; Seidel and Stürmer, 2014; Weber et al., 2020; Weyers et al., 2024). The spectrum ranges from closed-test formats (e.g., Gold and Holodynski, 2017) to mixed-method approaches (e.g., Weber et al., 2020) and purely qualitative instruments (e.g., Gippert et al., 2022). Analytical approaches further differentiate situation-specific skills into two (Gold and Holodynski, 2017), three (Weber et al., 2020), or four (Gippert et al., 2022) cognitive processes, each empirically assessed as a separate subfacet. One instrument focuses exclusively on a single subfacet (decision-making; Weyers et al., 2024), supplementing an existing instrument (CME-PI; König, 2015). While some instruments focus solely on CM-related PID, others simultaneously capture additional teaching aspects such as instructional support (Bauersfeld et al., 2025a). In the field of PE, empirically validated instruments to measure teacher noticing are available (e.g., Langer et al., 2023, 2024), but no one specifically addresses CM.

Review studies highlight substantial differences in the development and design of video-based measures (Gold et al., 2024; Weyers et al., 2023). Overall, instruments using open-ended prompts, multimodal foci, or extended video sequences are theoretically better suited to capture complex cognitive processes such as perception (noticing) and decision making (Bauersfeld et al., 2025a; Dückers et al., 2022; Gold et al., 2024; Weber et al., 2020; Seidel and Stürmer, 2014). In closed test formats guided attention through rating items simplify the perception and selection of relevant classroom events and decision-making is typically measured only in terms of evaluating potential situational action alternatives (cf. Weber et al., 2020 for a critique). Consequently, such highly standardized approaches tend to be more strongly aligned with the cognitive process of interpretation or knowledge-based reasoning (cf. Gold et al., 2024; Weyers et al., 2023).² Nevertheless, highly structured, closed-task approaches offer advantages in construct alignment, scoring objectivity, and test efficiency (Bauersfeld et al., 2025a; Eßling et al., 2024; Seidel and Stürmer, 2014; Weyers et al., 2023).

From the data pool of 60 videotaped primary school PE lessons, a total of 10 unscripted video vignettes (duration 1:19–3:27 min.) were developed. The videos were recorded using both static and mobile camera perspectives (Jeisy, 2011). The teaching sequences were selected to ensure that all nine dimensions of good CM in PE (e.g., monitoring, clarity of instructions, and ensuring safety) were represented at least twice and that a broad range of subject-specific content areas (gymnastics, dance, circus, games) was included to enhance ecological validity (see Table 1; Baumgartner et al., 2023). The three cognitive processes of PID are each assessed using dichotomous items. Participants’ responses were coded as correct or incorrect by comparing them with an expert-defined master rating (see Section 3.2).

Content validation was established through a three-phase Delphi study (Häder, 2014) with eight CM-in-PE experts and a pretest with 54 pre-service PE teachers.³ Expert feedback informed item revisions and led to the establishment of a consensual master rating, while the pretest confirmed usability and item clarity but indicated low item difficulty and substantial time demands. Since all video vignettes proved to be effective in terms of content validity, none were excluded at this stage of test development. The video-based test instrument for the validation study comprised 10 video vignettes and 161 items, balanced across the three cognitive demands of PID and the nine CM dimensions (Baumgartner et al., 2023). Throughout the Delphi process, experts specifically evaluated whether vignettes and items adequately reflected PE-typical CM challenges (e.g., spatial dispersion, transitions, equipment and safety), thereby strengthening the instrument’s subject-specific content validity.

The present study aims to examine the internal psychometric properties, reliability, and dimensional structure of the test instrument, as well as its construct and criterion validity (Gold et al., 2024). Furthermore, the study investigates whether the instrument can be shortened by reducing the number of video vignettes, to facilitate smoother implementation within the WiPe-Sport intervention study (Baumgartner et al., 2025).

We expect the test instrument to demonstrate sufficient reliability when measuring CM-relevant PID, which is a prerequisite for its use in an intervention study (Weyers et al., 2024). In line with previous research (Bastian et al., 2024; Dückers et al., 2022; Gippert et al., 2022; Junker et al., 2021, 2025), we anticipate that the cognitive processes assessed by the CM-related PID test can be empirically differentiated into distinct subfacets.

To enhance test efficiency, we explore potential reductions in the number of vignettes or items while maintaining representation of all nine CM dimensions and ensuring comparable results in the measurement of individual skills.

Based on the theoretical framework of knowledge-based situation-specific PID skills and previous empirical findings (cf. Müller and Gold, 2023), we hypothesize a significant positive correlation between the two competence facets. Demonstrating such a correlation would provide evidence for the test’s construct validity as CM-relevant knowledge and CM-relevant PID are not the same construct but closely related (e.g., Junker et al., 2025; Weyers et al., 2024).

In Switzerland, teacher education follows a one-phase model, meaning that theoretical coursework (e.g., on CM) alternates with internships and practice-oriented courses that closely link theory and practice (see Section 2.1). Therefore, we expect third-year students to outperform first-year students on the CM-relevant PID test. Demonstrating such differences would provide evidence for the test’s criterion validity (Gold et al., 2024; Franz et al., 2019).

The pilot study included 877 pre-service PE teachers specializing in kindergarten and primary education (740 female, 130 male, 7 divers), with a mean age of 23.4 years (SD = 4.0, range = 18–54). Participants were recruited from four Swiss Universities of Teacher Education (UTEs); St. Gallen UTE [n = 473], Lucerne UTE [n = 357], Fribourg UTE [n = 41], Grisons UTE [n = 6]. They were in their first (n = 277), second (n = 283), or third (n = 313) year of study (four not reported). All participants were enrolled in bachelor-level teacher education programs qualifying them to teach PE at the kindergarten or primary level. Although the participants attended general seminars on pedagogy and PE as part of their standard curriculum, they did not receive any specific intervention or preparatory training regarding this video-based instrument prior to data collection. The test therefore serves as a diagnostic tool to assess the status quo of their skills within their regular education.

Data was collected online between March and April 2022 using a questionnaire that combined the CM-related PID and CM-related knowledge tests. A third section gathered personal information, including participants’ teaching experience in PE and their self-assessed CM-related competence. The survey was administered via the LimeSurvey platform (LimeSurvey GmbH, n.d.).

To balance participant burden and statistical power for IRT-based analyses, items were randomly assigned. For the PID test, participants were randomly assigned to complete four of the 10 video vignettes (58–71 items), resulting in 316–374 responses per vignette. To control for potential order effects, the sequence of the two competence facets (PID test before the knowledge test and vice versa) was randomized. On average, participants required 33 min (SD = 13) to complete the entire survey, spending approximately 17.1 min (SD = 8.0) on the PID test and 5.4 min (SD = 4.5) on the knowledge test.

A subset of the participants completed the assessment as part of a university course (St. Gallen UTE, Lucerne UTE), whereas others participated independently in their free time (Fribourg UTE, Grisons UTE). In total, 1,473 pre-service PE teachers registered for the study, of whom 1,076 submitted complete responses, resulting in a response rate of 73%. To ensure data quality, only cases in which all items were completed, and the total processing time fell within a predefined realistic range were included. These thresholds were based on empirical estimates of video duration and task completion time, corresponding to 15 min overall – at 2 min for the knowledge test and 8 min for the PID test. After data screening and cleaning procedures, 877 participants (59%) were retained for analysis.

All items were dichotomously scored (0 = incorrect, 1 = correct) by comparing participants’ responses with the master rating validated previously (Baumgartner et al., 2023). Items with extremely high or low solving rates and high expert ambiguity during pretest were re-evaluated prior to analysis.

Psychometric analyses followed an IRT-based approach to examine internal structure and reliability of the PID test. Model selection followed a confirmatory approach, comparing different a priori specified, theoretically and design-informed model structures. Validity examination was complemented by inferential statistics (RQ1; Moosbrugger and Kelava, 2020).

The most appropriate IRT model was identified by comparing: (a) models with different parameter specifications, (b) potential multidimensional structures and (c) local and differential item fit. (a) The three-parameter logistic (3PL) model was compared with the simpler, nested two- (2PL) and one-parameter (1PL) models. The 3PL was theoretically expected to fit best, given the dichotomous items based on diverse video vignettes. (b) Using the most parsimonious model, we then examined whether more complex multidimensional structures improved fit. The one-dimensional solution was compared with (1) a theoretically grounded three-dimensional model reflecting the subfacets of PID, (2) a nine-dimensional model representing the CM-related content dimensions, and (3) a ten-dimensional testlet model grouping items by video vignettes (cf. Gold and Holodynski, 2017). Model comparisons relied on likelihood ratio (LR) tests, Akaike (AIC) and Bayesian (BIC) information criteria, expected a posteriori (EAP) and weighted likelihood (WLE) reliability estimates. (c) Local item fit was evaluated using weighted likelihood estimates as well as infit and outfit statistics, ensuring values remained within acceptable ranges (0.8–1.2, or t-standardized values between −1.96 and 1.96; Bond et al., 2021). Items showing substantial misfit were examined individually. Because ecological validity was prioritized, items representing essential aspects of a vignette were retained even when discrimination values were low, acknowledging that the 2PL model weights them accordingly. Differential Item Functioning (DIF) analysis using the Mantel–Haenszel test was conducted to examine whether item responses were conditionally independent of demographic variables such as gender, mother tongue, university placement, and participation type (Diaz et al., 2021).

An Item-Person-Map (Wright Map) was employed to examine how well the distribution of item difficulty matched the range of participant ability levels, thereby assessing the overall adequacy of the test for the given sample. On this map, the average person’s ability is positioned at zero on the logit scale, with item difficulties displayed in relation to this reference point. When a person’s ability level coincides with an item’s position on the scale, the probability of answering that item correctly is approximately 50%. This graphical representation enables an intuitive understanding of how item difficulty aligns with participant ability (cf. Bond et al., 2021).

Finally, a shorter version of the instrument was created by reducing the number of video vignettes while maintaining representation of all 9 CM dimensions (each represented at least two times in the remaining vignettes). IRT models for both the full and the shortened versions were estimated and evaluated in terms of model fit, item parameters, person estimates, and reliability to ensure that the shorter version maintained adequate measurement precision (RQ2).

Construct validity (RQ3) was examined through correlations between person ability estimates from the knowledge and PID tests (long and short version). Criterion-related validity (RQ4) was then assessed by analyzing both test versions abilities to distinguish pre-service PE teachers across study years using ANOVA with post-hoc comparisons.

All analyses were conducted in R Project for Statistical Computing (RRID: SCR_001905) using RStudio (2023.06.01). Model comparisons among 1PL, 2PL, and 3PL specifications were conducted using the mirt package. Analyses of dimensionality and subsequent IRT analyses were performed using the TAM package (Version 4.1.4) with marginal maximum likelihood estimation (e.g., tam.mml.2pl). For multidimensional models, numerical integration used a Monte Carlo approach with 2,000 nodes and 100 iterations (convergence = 1e–4) to ensure stable parameter estimation.

The anonymized data and instrument materials are openly available at SWISS Ubase (Berthold et al., 2025b).

First, we conducted model selection to determine the most appropriate IRT model. The more complex 3PL model was initially estimated but rejected in favor of the simpler, nested 2PL model (Table 2). Although the LR test indicated a statistically significant improvement in model fit for the more complex 3PL model, information criteria (AIC, BIC) suggested that this improvement was offset by increased model complexity. However, the 1PL yielded insufficient reliability, likely reflecting its restrictive assumption of equal item discrimination. Therefore, the 2PL model was selected as the most parsimonious model.

Second, the test instrument was designed to assess pre-service PE teachers’ situation-specific PID skills as facet of their CM-related professional competency, which could lead to multidimensionality. Accordingly, several multidimensional solutions were tested: (1) a three-dimensional model reflecting the theoretical subfacets of perception, interpretation, and decision-making (PID), (2) a nine-dimensional model representing CM content domains; and (3) a ten-dimensional model accounted for potential testlet effects due to shared video stimuli.

Model comparisons indicated that the three-dimensional model provided a significantly better fit than the unidimensional solution [Δχ²(3) = 350.52, p < 0.001], whereas the nine- and ten-dimensional solutions did not improve fit (Table 3). The three-dimensional solution differentiated between P-I-D while accounting for their intercorrelations. While this model showed superior fit and the three subfacets were moderately correlated (r = 0.10–0.42), their subfacet reliabilities were below acceptable levels, particularly for decision-making (EAP = 0.68, 0.60, 0.50; WLE = 0.64, 0.55, 0.37).

Given these results, we used the unidimensional 2PL model for all subsequent analyses because it yielded substantially higher reliability and more stable person estimates than the three-dimensional scoring approach. Accordingly, all results reported below are based on the unidimensional model.

Local item fit was examined for the unidimensional model using mean-squared residual–based item-fit statistics (tam.fit). Outfit values ranged between 0.85 and 1.05 (t-standardized = −0.31 to 0.66) and Infit values between 0.99 and 1.01 (t-standardized = −0.19 to 0.19), indicating good local fit.

Differential Item Functioning (DIF) analysis using the Mantel–Haenszel test revealed no indications of DIF regarding gender, mother tongue, university placement, or participation type.

The Item-Person Map (Figure 5) displays person abilities (left) and item difficulties (right) on the same latent scale. While both appear normally distributed, the majority of items are located below the mean person ability. This suggests that many items were relatively easy for the sample, limiting the test’s ability to distinguish higher-ability respondents.

A shorter version of the instrument was created by reducing the number of video vignettes from 10 to 7, while maintaining representation of all 9 CM dimensions (each appearing at least twice). This shortened version showed acceptable reliability for a unidimensional solution (EAP = 0.674; WLE = 0.639) compared with the full version (EAP = 0.726; WLE = 0.709). Person estimates from the two versions were strongly correlated (r = 0.81; p < 0.001), indicating substantial overlap in the underlying ability estimates. Although reliability decreased slightly due to fewer items, the parameter structure and local fit remained stable, indicating that the short form maintained sufficient precision for research applications while improving efficiency.

Next, we examined construct validity through correlations between person ability estimates from the knowledge and PID tests. For the full PID version, a small but significant positive correlation was found with the knowledge test (r = 0.29, p < 0.001, 95% CI [0.23, 0.35]). For the short version, the correlation was smaller but still significant (r = 0.16, p < 0.001, 95% CI [0.10, 0.23]). As expected, both PID versions were strongly correlated with each other (r = 0.81, p < 0.001, 95% CI [0.79, 0.83]). These results indicate that, while related, the two instruments assess with knowledge and PID two distinct facets of CM competence—supporting the test’s construct validity.

Finally, criterion-related validity was assessed by comparing pre-service PE teachers’ test scores across study years. ANOVAs conducted separately for the short and full versions revealed significant effects of study year on test performance, indicating that the instrument distinguishes between different stages of teacher education. For the short version, the effect of study year was significant (f[2, 859] = 11.94, p < 0.001). Tukey post hoc comparisons showed that second-year students scored higher than first-year students (ΔM = 0.33, 95% CI [0.04, 0.63], p = 0.022), and third-year students scored higher than first-year students (ΔM = 0.60, 95% CI [0.31, 0.88], p < 0.001). The difference between second- and third-year students was not significant (p = 0.076).

For the full version, the effect of study year was likewise significant (f[2, 870] = 7.57, p < 0.001). Tukey post hoc comparisons indicated that third-year students outperformed first-year students (ΔM = 0.39, 95% CI [0.15, 0.62], p < 0.001), whereas differences between first- and second-year (p = 0.12) and between second- and third-year students (p = 0.14) were not significant.

Together, these results demonstrate that both the full and short versions of the test effectively distinguish between pre-service PE teachers at different stages of their education, providing evidence for criterion-related validity.

The analyses (RQ1) showed that a unidimensional two-parameter logistic (2PL) model offered the most parsimonious and psychometrically stable solution for the full instrument. While the three-dimensional model (corresponding to P-I-D) achieved superior fit compared to a strictly unidimensional model, the reliabilities of the subfacets (especially decision-making) were low.

Importantly, retaining a unidimensional solution should not be interpreted as evidence against the theoretical differentiation of perception, interpretation, and decision-making. Rather, the unidimensional score summarizes the shared variance across P-I-D–related item sets and serves as a pragmatic scoring model for the present instrument. This is also plausible from a cognitive-psychological perspective, as P-I-D draw on partially overlapping knowledge-based resources and are sequentially interdependent in real-time classroom processing (Weyers et al., 2024).

From a measurement perspective, recent methodological work demonstrates that the empirical separability of PID subprocesses is substantially shaped by assessment format (Gold et al., 2024). In the present study, the use of a closed-response format likely contributed to the predominantly unidimensional structure observed in the analyses. Although each video vignette was designed to address perception, interpretation, and decision-making through dedicated questions, the dichotomous response options primarily assessed participants’ ability to recall and evaluate CM-relevant classroom events based on their existing knowledge (Eßling et al., 2024). While accurate recall presupposes perceptual awareness (Seidel and Stürmer, 2014), the item format may have shifted the cognitive focus toward knowledge-based reasoning – that is, the situation-specific ability to interpret and judge CM events through a knowledge-driven lens rather than capturing the full spectrum of P-I-D processes (cf. Gold et al., 2024). This aligns with findings that closed, rating-based formats tend to emphasize interpretive and evaluative processes rather than spontaneous perception or in situ decision-making (Gold et al., 2024; Weber et al., 2020). Consequently, the instrument may have captured CM-related situation-specific reasoning as an integrated ability more strongly than the distinct cognitive subfacets postulated in the PID framework.

Finally, the Item-Person Map confirms a mismatch between item difficulty and participant ability, as most items fall below the mean person ability. This imbalance indicates that the test may insufficiently capture variation among higher-ability participants, likely due to a ceiling effect (Berthold et al., 2025a). Consequently, the instrument’s sensitivity at the upper end of the scale and its validity for assessing advanced levels of CM-related PID appears limited. Interestingly, this high level of demonstrated competence contrasts with the subjective experience often reported in the literature, where pre-service teachers express difficulties with CM (Dicke et al., 2015; Stokking et al., 2003) and criticize insufficient training in teacher education (Stevenson et al., 2020). However, our findings align with other empirical studies that – contrary to self-reports – document relatively high levels of CM-related knowledge (Gippert et al., 2022; Junker et al., 2021; Schlag and Glock, 2019), CM-related PID (Gold et al., 2021; Junker et al., 2021), and even performance among pre-service teachers (Gold et al., 2021). Thus, the observed ceiling effect may not only reflect test ease but also indicate that the target group possesses a more solid foundation in CM skills than their subjective perception suggests.

Beyond these psychometric considerations, the results should be interpreted in light of the subject-specific demands of CM in PE. In PE, CM is closely intertwined with movement dynamics, spatial dispersion, frequent transitions, and equipment logistics, with safety as a central boundary condition. By embedding these PE-typical demands in unscripted PE video vignettes and aligning items with a PE-specific CM framework (including ensuring safety and managing equipment), the present instrument provides a context-sensitive operationalization of CM-related situation-specific PID. This strengthens its applicability for PE teacher education and the WiPe-Sport research project (Baumgartner et al., 2025).

Regarding test economy (RQ2), the shortened version of the instrument (seven video vignettes) demonstrated satisfactory model fit and reliability, with person ability estimates strongly correlating with those from the full version (r = 0.81). These findings suggest that the shorter form can serve as a practical alternative in time-sensitive research settings, such as intervention studies within the WiPe-Sport project (Baumgartner et al., 2025).

Nevertheless, some caution is warranted, as slight losses in precision, discriminative power, and subject-specific (PE) ecological validity may occur. Although all 9 CM dimensions remain represented at least twice in the short version, its subject-specific (PE) scope in the video sequences is restricted to the content areas games and gymnastics. In contrast, the long version also includes dance and circus, providing a more comprehensive representation of PE teaching diversity (see Table 1).

As hypothesized (RQ3), correlations between CM-related knowledge and CM-related PID scores were significant yet moderate (r = 0.29 for full version; r = 0.16 for short version). This supports construct validity: while related, knowledge and situation-specific PID skills are not redundant but represent distinct facets of the competence construct (Baumgartner, 2022; Blömeke et al., 2015). The moderate correlation aligns with prior findings (cf. Müller and Gold, 2023), emphasizing that the transition from theoretical understanding to in-situ application requires cognitive integration rather than a simple transfer of knowledge (Berthold et al., 2025a).

The test successfully differentiated pre-service PE teachers across study years, providing evidence for its criterion validity (RQ4). Both the full and short versions showed significant improvements from the first to the second and from the first to the third year, although differences between the second and third year were not consistently significant.

We interpret this finding in two ways. First, it supports previous research indicating that situation-specific PID represents a learnable competence facet of teachers’ professional competence that develops within teacher education programs combining practical learning opportunities and theoretical coursework (Barenthien et al., 2023; Bauersfeld et al., 2025b; Eßling et al., 2023; Junker et al., 2025; Kaiser et al., 2025; Stahnke and Blömeke, 2021; Stahnke et al., 2016; Weyers et al., 2024; Wolff et al., 2016, 2017). However, it is important to acknowledge that our data stem from cross-sectional cohort comparisons and thus cannot capture individual developmental trajectories. Furthermore, data were collected at several Swiss UTEs, which, although all follow a one-phase teacher education model, may differ in the design and implementation of their programs. Further analyses would be needed to account for these institutional variations and to compare our findings with studies conducted in other countries, where the development of situation-specific PID skills has been investigated within a two-phase teacher education system (e.g., Germany, Bauersfeld et al., 2025b).

Second, the results are encouraging from a practical perspective. The instrument’s ability to distinguish between pre-service PE teachers at different stages of their development of professional competence highlights its diagnostic potential for assessing developmental levels of CM-related PID within teacher education. Such differentiation provides valuable feedback for program evaluation and for the targeted design of learning opportunities that foster CM-related PID.

Several limitations of this study must be acknowledged. First, the CM-related PID instrument was validated with pre-service kindergarten and primary school PE teachers enrolled at four UTEs in Switzerland. Given the differences between PE teacher education programs (cf. MacPhail et al., 2019) and the subject-specific nature of the instrument (PE), the findings cannot be generalized to other school levels, subjects, or teacher education systems. Moreover, the applicability of the instrument to in-service PE teachers remains untested and should be examined in future research to determine its relevance across different career stages.

Further methodological limitations concern the limited reliability when distinguishing between the cognitive subprocesses of PID. Consequently, CM-related PID was modeled as a unidimensional construct, limiting fine-grained interpretation. Future developments might therefore consider more open-ended formats that better capture the nuanced differences between the subprocesses (cf. Weyers et al., 2023). Also, the relatively low item difficulty (high solution rates) may cause ceiling effects in more advanced cohorts, reducing sensitivity to growth.

Additionally, focusing on a single competence-area (CM) does not fully reflect the multidimensional reality of teaching (Praetorius and Charalambous, 2018). Effective teaching in PE simultaneously involves multiple, interrelated generic competence-areas such as instructional support, student engagement, and content representation (Bauersfeld et al., 2025a; Dückers et al., 2022, 2025) and PE-specific competence-areas such as cognitive-motor activation (Kruse et al., 2024). The presented CM-related instrument thus assesses only one limited aspect of pre-service PE teachers’ situation-specific PID skills. Similarly, the use of a master rating as a reference standard must be interpreted cautiously – within the complexity of real classroom interactions, the notion of “correct” or “incorrect” teaching behavior is inherently debatable (Gold et al., 2024). The results should therefore be viewed as an alignment with a normatively defined yet contestable benchmark of good teaching rather than an absolute measure of professional competence.

Another limitation concerns the instrument’s ecological validity. The video-based test captures teachers’ situation specific PID skills on action – that is, reflective judgments about observed situations – rather than in action, where decisions are made in real time within the dynamic flow of classroom interaction (cf. Witt et al., 2024). Consequently, the test reflects only part of the situational complexity inherent in authentic teaching situations.

Finally, recent evidence suggests that individual cognitive characteristics, such as selective attention, may influence performance in video-based PID assessments (Dückers et al., 2025). While situation-specific PID skills remains a primarily knowledge-based process, such findings indicate that general cognitive abilities might moderate the extent to which individuals can translate knowledge into situational PID. Accounting for these interindividual differences could improve the precision and fairness of future test designs.

Taken together, these limitations point to several directions for future research. Subsequent studies should (a) test the instrument across broader educational contexts and teacher populations, (b) refine item formats to increase cognitive differentiation and difficulty, and (c) explore ways to assess multiple competence-areas simultaneously. Developing integrated test instruments that encompass several generic and PE-specific dimensions of teaching – such as CM, instructional quality, student support and cognitive-motor activation – would offer a more holistic and ecologically valid assessment of pre-service PE teachers’ professional competence.