Critical thinking (CT) is widely recognized as a core 21st-century skill and a key predictor of success in education and society (Voogt and Roblin, 2012; Rusmin et al., 2024; Rothinam et al., 2025). This is largely because CT is associated with key competencies for effective functioning in contemporary society, such as informed decision-making (Stupple et al., 2017; Ren et al., 2020), employability, and civic engagement (Facione and Facione, 2001; Minnameier and Hermkes, 2020; Simonovic et al., 2022). Moreover, CT is increasingly viewed as a central outcome of higher education, underscoring the importance of fostering this ability among university students, whose academic achievement has also been shown to be linked to CT (Rivas et al., 2023).

Although many definitions of CT have been proposed (e.g., Ennis, 2011; Halpern, 2014; Moore, 2011), a particularly comprehensive one was formulated by the American Philosophical Association Delphi panel of 46 experts, defining critical thinking as “purposeful, self-regulatory judgment which results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgment is based “(Facione, 1990, p. 3). Although CT may manifest differently across contexts and disciplines, a shared feature across conceptualizations is the ability to analyze and evaluate arguments and the evidence supporting them – skills widely recognized as central to critical reasoning (Dwyer et al., 2014; Mueller et al., 2020; Liu et al., 2014; Stanovich et al., 2016). Despite variations in definitions, this common emphasis supports viewing CT as a coherent construct centered on analytic and evaluative reasoning.

Given its importance, the accurate assessment of CT is essential for both research and educational practice. Although ministries of education worldwide emphasize the development of CT, only a limited number of assessment tools have been validated for use in different populations (Hassan and Madhum, 2007). Several standardized instruments exist, including the Watson–Glaser Critical Thinking Appraisal (WGCTA; Watson and Glaser, 1980), the Cornell Critical Thinking Test (Ennis and Millman, 1985), the California Critical Thinking Skills Test (Facione, 1990), and the Halpern Critical Thinking Assessment (Halpern, 2010). Despite their widespread use, concerns remain regarding their psychometric quality. Research has revealed inconsistent evidence of validity and reliability, including low reliability coefficients (Abrami et al., 2008; Verburgh et al., 2013), unstable factor structures (Abrami et al., 2008; Bernard et al., 2008; Leach et al., 2020; Verburgh et al., 2013), problematic response formats (Bernard et al., 2008; Ku, 2009), ambiguous instructions (Fawkes et al., 2003; Possin, 2014), limited research applications (Liu et al., 2014) and cross-cultural equivalence (Butler et al., 2012). A persistent challenge in the critical thinking literature concerns how to validly assess such a multifaceted construct, further complicated by the limited empirical evidence supporting the correspondence between test items and their underlying theoretical dimensions (Leach et al., 2020). These issues highlight the need for thorough psychometric evaluation of CT assessments in diverse populations. The present study therefore focuses on the Watson–Glaser Critical Thinking Appraisal (WGCTA)—the oldest, most widely used, and most extensively studied CT test (Bernard et al., 2008). Although the WGCTA has been applied across a wide range of educational and professional contexts and is valued for assessing core components of CT (Wayas et al., 2024; Afzal et al., 2024; Šeboková et al., 2025), its continued use depends on robust evidence of its psychometric adequacy, making its evaluation both theoretically and practically important.

The WGCTA defines critical thinking as “the ability to identify and analyze problems, seek and evaluate relevant information, and reach an appropriate conclusion” (Watson and Glaser, 2018). It assesses five interrelated skills in a verbal context: (1) Inference – judging the likelihood that conclusions follow from given information; (2) Recognition of assumptions – recognizing implied assumptions or presuppositions behind the provided statements; (3) Deduction – evaluating whether conclusions logically follow from premises; (4) Interpretation – assessing whether conclusions are justified by evidence; and (5) Evaluation of arguments – determining the strength and relevance of arguments (Watson and Glaser, 2018). Since its introduction, the WGCTA has undergone several revisions. The original 1964 version included 100 items (Forms Ym and Zm). In 1980, Forms A and B (WGCTA-II, 80 items) were published, followed by a UK adaptation of Form B (Form C; Rust, 2002). A 40-item Short Form S and later parallel Forms D and E were also developed. The most recent revision, WGCTA-III (Watson and Glaser, 2018), introduced updated, business-oriented items and is now available in an online format.

Although the WGCTA has been extensively studied internationally, its psychometric properties have not yet been systematically examined in Slovakia. The Slovak adaptation of WGCTA-II (Form C) exists (Watson and Glaser, 2000), but no published data are available regarding its reliability, validity, or factor structure in Slovak university students. Moreover, the administration of the WGCTA-II requires approximately 40 min, which may limit its feasibility for large-scale research applications. Examining the Slovak version of the WGCTA-II in this context provides valuable empirical evidence on its psychometric soundness and practical applicability, thereby contributing to a broader understanding of the cross-cultural validity of critical thinking assessment tools, specifically in central-eastern European context.

Over the years, numerous studies have examined the psychometric properties of various WGCTA versions, highlighting concerns about their validity and reliability, particularly regarding the factor structure of the test. Some studies suggest a unidimensional solution rather than separate scores for the five subscales (Bernard et al., 2008; Hassan and Madhum, 2007). Bernard et al. (2008) examined 60 sets of subscale means and 13 sets of intercorrelations reported in published studies with diverse learner groups, identifying a consistent one-factor solution and concluding that the WGCTA measures a general critical thinking skill rather than five distinct dimensions. Similar findings emerged from a study with Lebanese university students, where exploratory factor analysis of the WGCTA-S also indicated a unidimensional structure (Hassan and Madhum, 2007). Taken together, these findings challenge the theoretical assumption of five separable dimensions and raise questions about the construct validity of the WGCTA subscales.

Another concern involves the wide range of reliability coefficients, reported between 0.23 and 0.73 (Bernard et al., 2008) or 0.17 to 0.74 (Loo and Thorpe, 1999). This variability suggests that the reliability of the WGCTA is not a stable property of the instrument but may be strongly context dependent. Previous research suggests that reliability may be influenced by contextual differences across learners and settings (Bernard et al., 2008), the multiple-choice format that increases the likelihood of guessing (Wagner and Harvey, 2006), and challenges related to cross-cultural applicability and face validity. Additional limitations of the WGCTA include non-comparable test forms, unclear evidence of differential validity across respondent groups, and limited evidence of incremental validity (Liu et al., 2014). Together, these issues limit the interpretability and comparability of WGCTA scores across studies and raise concerns about its suitability for cross-cultural research without thorough local validation.

Despite these issues, some studies provide evidence of convergent and criterion validity. For instance, WGCTA-S scores moderately correlated with analysis and problem-solving (r = 0.52) and judgment and decision-making (r = 0.52) (Ejiogu et al., 2006), as well as with undergraduate course grades in psychology and education (r = 0.20–0.62) (Gadzella et al., 2006). Overall, the available evidence indicates that while the WGCTA demonstrates meaningful convergent and criterion validity, its limitations are primarily related to the psychometric functioning of specific items and subscale structures, pointing to the need for systematic refinement rather than rejection, which subsequently motivated revisions of the test.

In response to these considerations, the Red Model was introduced in WGCTA-II, reorganizing items into a simplified three-factor structure: Recognize Assumptions, Evaluate Arguments, and Draw Conclusions (combining Inference, Deduction, and Interpretation). Confirmatory factor analysis supported this structure, with internal consistency ranging from 0.81 to 0.89 (NCS Pearson, Inc., 2009). Convergent and criterion validity were demonstrated through correlations with WAIS-IV (r = 0.52), Raven’s APM (r = 0.53), and the Advanced Numerical Reasoning Appraisal (r = 0.68). The RED model formed the basis for WGCTA-III, which features a new, globally applicable item bank. However, the RED model and its three-factor structure have not yet been independently and extensively evaluated. This gap in empirical evidence constrains both the measurement and conceptual understanding of critical thinking within educational settings. As Ku (2009) noted in her review of the psychometric properties of CT assessments, test developers and affiliated researchers tend to report more favorable psychometric qualities than independent researchers. This underscores the need for more impartial, externally conducted validations.

Moreover, no comprehensive item-level psychometric investigation has been conducted. IRT has several advantages over the classical test theory (CTT). In CTT, reliability is considered a property of the entire test, typically measured using coefficients such as Cronbach’s alpha. This approach necessitates re-evaluation if the test undergoes modifications, such as shortening or adaptation. In contrast, Item Response Theory (IRT) evaluates reliability at the item level, providing a more precise analysis of how each question contributes to assessing critical thinking, thereby enhancing the accuracy and flexibility of the measurement (Bond, 2015). IRT provides an accurate and consistent measurement of CT by accounting for both item difficulty and individual ability. Unlike traditional methods, it separates ability from item characteristics, ensuring more valid and reliable estimates while maintaining fairness through probabilistic modeling, where higher-ability students are more likely to answer difficult questions correctly (Bond, 2015). Critical thinking comprises multiple subscales, necessitating measurement tools that can assess each ability separately. Empirical research indicates that using the item response theory approach enhances the precision of cognitive and complex thinking skill assessments by providing a more detailed analysis of an instrument’s reliability and validity (Suwita et al., 2024). Therefore, this methodological framework seems suitable for present study.

Given the unresolved issues with the WGCTA – unclear factor structure, inconsistent reliability, cultural applicability concerns, and the absence of item-level psychometric investigations—further examination is needed. In addition, the test’s considerable length and limited large-scale research applications in higher education contexts point to the need for a shorter yet psychometrically sound version. Building on the limitations, the present study had three main aims. First, we examined the psychometric properties of the Slovak version of the WGCTA – II (Form C) in a sample of university students. Specifically, we conducted reliability analyses and confirmatory factor analyses to clarify the underlying structure of the test’s five subscales. This step was necessary to determine whether the original multidimensional structure of the WGCTA would be empirically supported in our cultural context and to identify potential sources of psychometric weakness. Second, based on the results of the factor analyses and the item-level diagnostics, we performed an item response theory (IRT) analysis using the Rasch model to evaluate a shortened version of the WGCTA that would retain only the most informative and psychometrically valid items. The goal was to develop a more practical and efficient measure of critical thinking suitable for research and educational settings, without compromising reliability or validity. Third, we examined the reliability and the convergent validity of the shortened version of WGCTA on two samples of university students.

The present study examined the psychometric properties of the Slovak version of the WGCTA-II (form C) in a sample of university students and evaluated a shortened version that maintains psychometric rigor while offering greater practical feasibility. Specifically, the study assessed whether the original structure of the WGCTA is supported in a new cultural and linguistic context and sought to identify potential sources of measurement instability at both the subscale and item levels. By integrating empirical analyses and theoretical considerations, we aimed to provide a reliable and efficient tool for both research and educational applications.

Across multiple levels of analysis, the results consistently indicated limited support for the intended multidimensional structure, including poor model fit of several subscales, weak and unstable item–factor relationships, and low internal consistency at the dimension level. From a measurement perspective, these findings also precluded meaningful testing of higher-order or hierarchical CFA models, as the prerequisite unidimensionality at the subscale level was not met. Although the total WGCTA score demonstrated acceptable reliability, this result should be interpreted with caution, as it appears to be driven primarily by test length rather than by coherent measurement of distinct components of critical thinking. In contrast, the convergence of low subscale reliability, inadequate factor loadings, and inconsistent dimensional fit raises substantial concerns regarding the construct validity of the original subscale configuration. The findings point to a broader pattern of psychometric weaknesses in the full Slovak version of the WGCTA-II, and together with its considerable time demands and complex administration, suggest that it is not optimal for research purposes. This conclusion aligns with previous findings (e.g., Bernard et al., 2008; Ku, 2009; Liu et al., 2014; Verburgh et al., 2013), and reflects the broader challenges associated with validating the factorial structure of instruments designed to measure complex constructs such as critical thinking, particularly when individual items are intentionally constructed to engage multiple, interrelated cognitive processes rather than discrete abilities (Leach et al., 2020).

Current trends in critical thinking research point toward the development of shorter, psychometrically robust, and culturally adapted instruments (Payan-Carreira et al., 2022; Rivas et al., 2023; Fabio et al., 2025; Butler et al., 2012). Test shortening allows for more efficient assessment while maintaining validity and reducing the administrative burden on both participants and researchers. In line with this approach, we focused on developing a shortened version of the WGCTA that combines conceptually key dimensions and empirically validated items while preserving psychometric quality.

In selecting the most appropriate indicators of critical thinking, we applied a combination of theoretical and empirical criteria (DeVellis, 2017; Kline, 2016). Rather than attempting to preserve the original five-subscale structure at all costs, we adopted a parsimonious approach that prioritized dimensions demonstrating both empirical stability and conceptual centrality to critical thinking. The empirical criteria included factor loadings, model fit, and reliability indices, while the theoretical criteria were derived from the RED model of critical thinking and considered the content validity. From the construct perspective, we retained two dimensions—Interpretation and Evaluation of Arguments—which represent the core processes of critical thinking. The Interpretation dimension reflects the ability to assess the reliability and relevance of evidence and to avoid common cognitive biases such as reasoning fallacies, jumping to conclusions, confusing correlation with causation, or the indefinite pronoun fallacy (Stanovich and West, 2008; Kahneman, 2011). The Evaluation of Arguments dimension complements Interpretation by focusing on the assessment of the logical strength of claims and the identification of manipulative or insufficiently supported arguments. These dimensions showed comparatively higher internal consistency, more interpretable factor loadings, and more balanced item difficulty profiles than the remaining subscales.

The selection of these two dimensions is also consistent with broader theoretical conceptualizations of critical thinking (e.g., Facione, 1990; Halpern, 2014). The core skills identified by the widely recognized Delphi panel – analysis, evaluation, and inference – align with our dimensions, as they involve recognizing argument structure, assessing credibility, and evaluating evidence. Similarly, Reflective judgment theory (King and Kitchener, 2004), the Dual Process Model (Stanovich et al., 2016) and Kuhn’s framework of argumentative reasoning (Kuhn, 1991) emphasize analytic and evaluative reasoning as central components of critical thinking. Taken together, this body of evidence indicates that interpretation and evaluation of arguments constitute the core of critical thinking across theoretical models and represent fundamental skills for both academic and civic competence, internationally (Van Gelder, 2012; Liu et al., 2014; Mueller et al., 2020; American Psychological Association, 2013, 2016; Yulian, 2021) as well as within the Slovak context (Research Team, 2018; Kotlebová and Hankerová, 2024). In today’s information-rich environment, these two processes are critical for accurate reasoning and judgment, whereas other components, though supportive, do not form the core of critical thinking.

The only RED Model dimension not included in the final version was Recognizing Assumptions. Although this dimension – the ability to distinguish facts from opinions and identify implicit beliefs – represents a relevant but not central component of critical thinking, its psychometric performance was unsatisfactory. The items showed low internal consistency and limited contribution to the overall construct, failing to adequately differentiate between individuals with higher and lower levels of critical thinking. This pattern is consistent with theoretical perspectives on paradigmatic assumptions, which suggest that deeply held assumptions are often implicit and difficult to access through self-report or decontextualized questionnaire formats, thereby limiting the sensitivity of such items to individual differences (Brookfield, 2015).

Moreover, item selection also considered content and contextual appropriateness, in addition to statistical indicators. Items requiring work with evidence, identification of logical fallacies, and evaluation of argumentative strength were prioritized, while less suitable items were excluded due to culturally specific contexts, ambiguous wording, or excessive linguistic complexity. A balanced representation of items with correct and incorrect answers was maintained to support content balance and measurement fairness. These results highlight the importance of culturally sensitive adaptation, as even originally validated items may function differently across linguistic and cultural contexts (Butler et al., 2012; Fabio et al., 2025). In addition, findings underscore that psychometric quality cannot be separated from cultural and linguistic appropriateness, highlighting the necessity of careful adaptation when applying standardized critical thinking measures across contexts.

The results of the IRT analysis indicated that selecting items that were both psychometrically and content-wise strong produced a unidimensional model with a reliable and valid structure, confirmed in an independent replication sample. Conceptually, interpretation and evaluation of arguments represent distinct but closely related processes of critical thinking. Empirically, however, their strong interrelation within the psychometrically refined item set resulted in a unidimensional measurement model in the shortened WGCTA. Importantly, the unidimensionality should not be interpreted as evidence that critical thinking itself is a single, undifferentiated ability, but rather as a property of the selected measurement model. This distinction is crucial, as unidimensionality at the measurement level does not preclude multidimensionality at the conceptual level and is consistent with theoretical and empirical perspectives that conceptualize critical thinking as a system of interrelated core processes (Bernard et al., 2008; Dwyer et al., 2014; Grimm and Richter, 2024; Fabio et al., 2025; Rivas et al., 2023). Given the limited fit and instability of multi-factor solutions observed in the present data, the unidimensional model provided the most robust and interpretable representation.

The relationship between the shortened version and the Cognitive Reflection Test (CRT) provided additional evidence of criterion validity. A moderate relationship suggests that the two tests capture related, yet not identical, processes. While the CRT focuses on the inhibition of intuitive responses (System 1), the WGCTA reflects engagement of System 2 as well as broader analytical and argumentative skills. The ability to stop and start logical processing of information can be considered a prerequisite for CT. In this context, cognitive reflection may reflect a core cognitive mechanism underlying critical thinking, whereas the WGCTA captures the broader set of reasoning and evaluative skills, indicating a complementary, rather than redundant, relationship between the two measures. By examining this relationship across both the original and shortened WGCTA versions and replicating the findings in an independent sample, the present study further demonstrates the robustness, generalizability, and preservation of the theoretical and empirical properties of the shortened form (Furr, 2021).

Despite these limitations, the validated and reliable 9-item version of the WGCTA provides a practical and culturally adapted tool for the rapid assessment of critical thinking in higher education. Its shortened format reduces administration time and cognitive load, making it suitable for large-scale research, course evaluation, and studies examining instructional interventions aimed at developing critical thinking. For educational institutions, it offers an accessible means of monitoring the development of a key 21st-century competency, while maintaining sufficient psychometric quality for research and applied assessment contexts.