Student engagement has been variously referred to as ‘learner’ or ‘school’ engagement (Finn and Zimmer, 2012). According to Mosher and MacGowan’s (1985, p. 14) classic definition, engagement is ‘the attitude leading to, and the behaviour of, participation in’ the education institution’s programmes. More recently, Skinner et al. (2009, p. 494) define this construct as ‘the quality of a student’s connection or involvement with the endeavour of schooling and hence with the people, activities, goals, values and place that compose it.’

There has been long on theoretical debates but short on empirical evidence concerning the dimensionality of engagement. Some researchers argue for the unidimensional structure (Johnson et al., 2001; Skinner et al., 2009; Meng and Jin, 2017; Tsao et al., 2021), two (Green et al., 2007; Brutt-Griffler and Jang, 2019), three (Wang and Degol, 2014) or even four dimensions (Fredericks et al., 2016; Philp and Duchesne, 2016; Wang et al., 2016). For example, Green et al. (2007) conceptualised engagement as a bidimensional construct with one affective (i.e., positive attitudes towards school) and one behavioural (i.e., absenteeism, homework completion and class participation) component. On top of a three-dimensional structure (affective, behavioural and cognitive), Wang et al. (2016) proposed an additional social dimension, which purportedly could reflect the quality of social interactions with peers and adults while learning. Readers are reminded that the empirical studies to be reviewed below, more often than not, operationalise engagement in different ways (Reschly and Christenson, 2012), which reflects the conceptual challenges surrounding engagement.

In the past 30 years, a myriad of studies have documented the positive associations between student engagement and academic achievement (Finn and Voelkl, 1993; Connell et al., 1994; Philp and Duchesne, 2016; Wang et al., 2016).² To our best knowledge, Skinner et al. (1990) is probably the earliest widely cited quantitative study on student engagement and school achievement. It confirmed the link between students’ academic achievement in two subjects and their engagement among a sample of 200 students (aged 9–12) at one suburban elementary school. In this study, the students’ engagement was measured through a 10-item scale scored by their teachers. One major finding was that the strength of the association (r = 0.41, p < 0.001) between student engagement and reading score reached the very strong benchmark (0.30) in the effect size interpretation system adopted in the present study,³ which was similar to that between engagement and maths score (r = 0.40, p < 0.001).

Voelkl’s (1997) methodologically strong study examined the relationship between student engagement (with two distinct theoretical dimensions: affective and behavioural) and academic achievement among 1,335 eighth graders. Affective engagement correlated weakly with the fourth-grade achievement (r = 0.07, p < 0.05) and was linked to the seventh-grade achievement at a typical level (r = 0.10, p < 0.01). In contrast, behavioural engagement very strongly correlated, respectively, with the fourth-grade achievement (r = 0.42, p < 0.01) and seventh-grade achievement (r = 0.49, p < 0.01).

A most recent quantitative study by Martin et al. (2020) demonstrated the correlations of students’ engagement, respectively, with age, socio-economic status (SES) and language background among 2,803 teenagers (average age: 14.14) at six Australian high schools. For science subject-specific engagement, these authors identified two types: positive and negative. Their path analysis revealed that (1) age (β = −0.16) and SES (β = −0.06) statistically significantly correlated with positive engagement and (2) non-English speaking background statistically significantly correlated with negative engagement (β = 0.03). The latter result is of particular interest to us because it pointed to the possible link between L2 proficiency and engagement. Unfortunately, while Martin et al. (2020) reported βs for the focal (independent) variables, the effect size index β is inconducive to comparisons across different studies; in contrast, ΔR² is much more useful in cross-study comparisons.

In the field of applied linguistics, only a limited number of empirical studies have investigated the complex construct of engagement, which can be divided into two subcategories. As Philp and Duchesne (2016, p. 385) rightfully note, engagement is often ‘not theorised or operationalised specifically for language learning’, which is similar to the situation outside of the field of applied linguistics (cf. Reschly and Christenson, 2012).

One subcategory of the extant research on engagement comprises studies focussing on more macro levels, such as the school subject level (viz. teaching L2 as a holistic school subject, see DeWaelsche, 2015) and the school level (see Archambault et al., 2009). For example, Brutt-Griffler and Jang (2019) discussed the relationship between L2 (English) proficiency and engagement among 53 Grade 6 students in New York, most of whom immigrated from Spanish-dominant societies (e.g., Puerto Rico) into the US. L2 proficiency was evaluated with a student’s self-rated proficiency. They claimed that L2 proficiency ‘was not correlated with’ behavioural engagement (p. 17), although the correlation coefficient was 0.26. This biased conclusion was largely attributed to the over-reliance on the value of p, as can be reflected in many instances of Brutt-Griffler and Jang’s (2019) paper where only statistically significant results were deemed important and discussed. However, effect size is much more important than the value of p (Wei et al., 2019; Wei and Gao, 2022).

According to the benchmark system cited above, the effect size r 0.26 was close to the very large benchmark (0.30) and hence should not be deeded as unimportant. The lack of statistical significance in Brutt-Griffler and Jang’s (2019) results was probably due to the relatively small sample size (n = 30) in their correlation analysis. Despite this limitation, Brutt-Griffler and Jang’s (2019) findings provided correlation coefficients (viz. effect sizes) for future studies to compare and contrast.

The other subcategory consists of studies focussing on more micro levels (e.g., at the mathematics classroom level, see Mesa and Chang, 2010; at the L2 classroom level, see Yu et al., 2019; at the knowledge construction level,⁴ see Berthoud and Gajo, 2020; at the instructional task-design level, see Lambert and Zhang, 2019; Sert and Amri, 2021). For instance, Aubrey et al. (2020) reported on a longitudinal study exploring the factors contributing to learners’ (lack of) engagement in an EFL classroom at one Japanese university; based on qualitative self-reported reflection data from 37 second-year sociology, they found that a lack of engagement could be attributed to causes, such as ‘lack of vocabulary’ and being unable to say what the student wanted to ‘say quickly in English’ (p. 8). In other words, L2 proficiency level could affect engagement.

Bai et al. (2020) investigated the relations between behavioural engagement and three motivational variables (academic self-efficacy, task importance and interest) among 1,954 secondary school English learners of the same grade level (i.e., the 9th grade) in Singapore. The self-reported behavioural engagement level was evaluated via a five-point Likert scale comprising four items. Their multivariate regression analyses showed that: (1) each of these three motivational variables statistically significantly (p < 0.001) predicted students’ behavioural engagement; and (2) these three motivational variables together accounted for 17.9% in the behavioural engagement variance. However, no attempts were made to ascertain the unique contribution of each predictor in the regression model.

All in all, the above-reviewed studies suffer from at least two of following three limitations: (1) the sample representativeness (e.g., Meng and Jin, 2017; Bai et al., 2020), (2) limited use of effect size (e.g., Brutt-Griffler and Jang, 2019; Yu et al., 2019) and (3) the over-reliance⁵ on the statistical significance level, namely, the value of p (e.g., Brutt-Griffler and Jang, 2019; Bai et al., 2020). Accordingly, the present study attempts to overcome the above limitations by covering different grade levels, making full use of effect size and avoiding over-reliance on ps.

In connection with academic fields other than applied linguistics (e.g., personality psychology), Reschly and Christenson (2012) note that engagement still suffers from a jingle (i.e., different terms being used to refer to identical notions or constructs) and jangle (i.e., the same terminology being used to describe distinct notions and constructs) in the way it is defined and operationalised. In applied linguistics research, this state of affairs is no different (Hiver et al., 2021). In their most recent synthesis entitled ‘Measuring L2 Engagement: A Review of Issues and Applications,’ Zhou et al. (2020) do not propose a definition of ‘L2 engagement’, let alone its measurement. Another limitation is that a few recent studies (e.g., Aubrey, 2017; Brutt-Griffler and Jang, 2019) from major journals in the field unfortunately escaped Zhou et al.’s (2020) attention. Notwithstanding these limitations, they highlight ‘the variety of operational definitions’ (p. 79) concerning engagement used across studies; and they also rightfully observe that ‘one researcher’s conceptualisation of cognitive engagement is used as another’s measurement of behavioural engagement’, which is indicative of the conceptual challenges surrounding engagement.

Against this backdrop, L2 engagement is defined as the type of engagement specifically relating to the domain of a student’s L2 learning. In the present study, a student’s L2 engagement is operationalised as his/her score gained on a L2 engagement scale adapted from the UWES-S. To the best of our knowledge, the present study represents the first attempt to quantitatively measure ‘L2 engagement’ in a large sample, aiming to shed light on the empirical dimensionality of this construct and explore potential factors affecting it.

RQ1: What is the factorial structure of the L2 engagement scale?

RQ2: To what extent do the selected sociobiographical variables (e.g., L2 proficiency, parental attention and parental coaching frequency) predict L2 engagement?

RQ1 was addressed using both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), in addition to reliability analysis. Through the random sampling function of SPSS, the data set was divided into two halves, which were, respectively, subject to EFA and CFA to ascertain the construct validity of the L2 engagement scale. To gauge the scale’s reliability, Cronbach’s alpha (a measure of internal consistency) was used in that it is the most frequently used reliability index (Derrick, 2016).

The selected sociobiographical variables in RQ2 totalled 11 (see Table 1 for a complete list). RQ2 was addressed with hierarchical regression because it helps ascertain the unique contribution from each predictor variable (Leech et al., 2014; Kong and Wei, 2019) to the variance in our focal variable, L2 engagement. As the L2 engagement variance explained by each predictor may change depending on the order in which the predictor is entered into a regression model, the chosen order is crucial (Luo and Wei, 2021). Researchers may determine the order for predictor entry based on theoretical grounds, if well-established theories exist; but in the absence of such theoretical resources, researchers are advised to attempt all possible sequences and provide a range of effect sizes (rather than one single effect size) for each predictor, which represents a more refined approach to gauge the unique contribution of a predictor to the dependent variable in hierarchical regression (Wei et al., 2020). The latter practice was adopted below because of the exploratory nature of the present study.

CFA was performed with Amos 23.0, whereas the other statistical procedures were run with SPSS 27.0. For sake of convenience, the conventional statistical significance cutoff level (α = 0.05, non-directional) was adopted. Exact values of p were reported, except that very small values of p were reported as p < 0.0005.

The instrument comprised two major parts: the L2 engagement scale and the sociobiographical information section. This scale was adapted from the Chinese version of UWES-S (Fang et al., 2008). The UWES scale has been widely used to evaluate engagement in different cultural contexts, including China (Meng and Jin, 2017), India (Rastogi et al., 2017), Japan (Tsubakita et al., 2017), Turkey (Çapri et al., 2017) and the USA (Mills et al., 2012). It has also been adapted to some languages other than English, such as Chinese (Fong and Ng, 2012), Dutch (Schaufeli and Bakker, 2003) and Spanish (Schaufeli and Bakker, 2003).

In order to specifically focus on L2 learning (English learning in our case) rather than general learning, we replaced reference to ‘learning’ with ‘English learning’ on the Chinese version of UWES-S. The scale measured L2 engagement with 17 items on a seven-point Likert scale, with a score of 1 indicating ‘never’ and a score of 7 ‘always’. A participant’s score on this scale was an average of the scores from these 17 items; the higher the score, the higher degree of L2 engagement of this participant.

The sociobiographical information section covers 11 variables (e.g., L2 proficiency), most of which were included in previous studies (e.g., Johnson et al., 2001; Aubrey et al., 2020); for instance, L2 proficiency was evaluated with a student’s self-rated proficiency in English, which is consistent with previous research (e.g., Brutt-Griffler and Jang, 2019). It is worth noting that we included four under-investigated⁶ variables. Two concern parental input: degree of parental attention to their children’s L2 learning and frequency of parental coaching for L2 learning; our measures of parental input were different from those adopted in the few extant studies, including Butler (2015) who measured the so-called parents’ ‘direct behaviour.’⁷ Each of them was measured by one five-point Likert scale item (a higher score indicating a higher degree/frequency).

The other two under-investigated variables are a respondent’s study time devoted to L2 learning and his/her status of being the only child of the family or not. The former was gauged by one five-point Likert scale item asking the participant’s study time spent on L2 learning every day, with a score of 1 indicating ‘less than one hour’, 2 ‘one to two hours’, 3 ‘three to four hours’, 4 ‘five to six hours’ and 5 ‘more than six hours’; the latter was measured with one binary variable: being the only child of the family or not. The inclusion of these four under-investigated variables helps broaden the research scope for the research line investigating student engagement with L2 learning.

Among the valid sample of 21,157 Chinese students, 3,350 (68.4%) were studying at Grade Seven, 2,918 (13.8%) Grade Eight, 1905 (9.0%) Grade Nine, 6,409 (30.3%) Grade Ten, 5,268 (24.9%) Grade 11 and 1,307 (6.2%) Grade 12. Slightly over half of the sample (55.1%, n = 11,647) were females and the remaining 44.9% (n = 9,510) males. About a quarter of the students (25.9%) were the only child in their family, and the others (74.1%) were not. At the time of the survey, a quarter of the students (26.3%) lived in villages, another quarter (24.6%) in townships and approximately half (49.1%) in cities.

The survey instrument was an anonymous, open-access questionnaire on Wenjuanxing,⁸ a free China-based survey provider similar to SurveyMonkey.com. The draft questionnaire had undergone several rounds of revision and was piloted among a small group of secondary school students before the questionnaire was finalised. The final questionnaire was published on Wenjuanxing between March and May in 2020. This final version of the questionnaire was distributed to students at a total of over 200 secondary schools in 18 out of the 31 ‘provinces’, which covered all of the six administrative regions of mainland China: the Central (e.g., Jiangxi), East (e.g., Zhejiang), North (e.g., Hebei), Northeast (e.g., Jilin), Northwest (e.g., Tibet), South (e.g., Guangdong) and Southwest (e.g., Chongqing). Specifically speaking, at each of the sampled schools, the contact person (viz. the head teacher of an intact class or the English subject teacher) explained the purpose of this survey and promised anonymity to the students at the end of a class; the students gained access to the online questionnaire by scanning the bar code provided by the contact person.

Before performing EFA to explore the factorial structure of L2 engagement, we checked the relevant assumptions. The factorability of the data set was confirmed through the KMO test (0.977) and Bartlett’s test of sphericity [χ²(136) = 170813.572, p < 0.0005]. Furthermore, the sample-size-to-variables ratio (608.8) was very high. These assumptions checking results showed that our data set was appropriate for factor analysis.

We selected maximum likelihood as the factor extraction method and the direct oblimin rotation in that we assumed the extracted factors to be correlated, which is typical ‘for naturalistic data and certainly for any data involving humans’ (Field, 2013, p. 644). Aiming to extract the most appropriate number of factors, we employed both the Kaiser criterion of using eigenvalues over 1 and the visual inspection of a scree plot. We adopted Field’s (2013) suggested cutoff point of 0.40 for factor loadings. Only one factor, which accounted for 66.024% of the L2 engagement variance, was extracted (see Appendix 1). To ascertain the dimensionality of the L2 engagement scale, CFA was employed to compared four competing models: Model 1 defined one primary factor only and tested whether L2 engagement can be regarded as a unidimensional construct without three facets; Models 2 and 3 (see Appendix 2), respectively, defined three correlated and uncorrelated primary factors which correspond to the three theoretical factors defined by Schaufeli and Bakker (2003); Model 4 (see Appendix 3) defined a higher-order model with three first-order facets and one second-order underlying factor (L2 engagement).

Consequently, the fit indices for these models (Table 2) revealed that only Models 3 and 4 had adequate fit to the data. When simply based on the fit indices, it can be seen that Models 3 and 4 yielded virtually the same degree of model fit. However, considering very strong intercorrelations (above 0.9, see Model 3 in Appendix 2 for details), we suggest that L2 engagement should be treated as a latent variable indicated by the three factors, or as a multi-facet construct. Put differently, the empirical construct of L2 engagement has been shown to be unidimensional with three facets with our large-sample data set.

Reliability analysis indicated that this unidimensional scale had very high internal consistency (Cronbach’s α = 0.974). In a word, our answer to RQ1 was that the L2 engagement scale has sufficiently high reliability and validity in the Chinese EFL context.

Before running hierarchical regression, we conducted two rounds of data checking. The first round was a preliminary analysis; it aimed to explore which of the initial 11 independent variables could be used as predictors in regression analysis because entering too many independent variables into a regression model violates the principle of parsimony (see Luo and Wei, 2021). Consistent with the good practice of pre-selecting independent variables based on effect size rather than p (e.g., Kong and Wei, 2019; Wei et al., 2020), only potential predictors with an effect size value exceeding the typical benchmark (e.g., r = 0.1) were used in later regression.

The preliminary analysis, in which a series of bivariate analyses were conducted, confirmed that the first eight variables in Table 3 were suitable for inclusion into later regression. Specifically, one independent-samples t-test confirmed a statistically significant difference of medium magnitude between participants who was the only child of his/her family and their counterparts (r = 0.127); the former (M = 4.09, SD = 1.16, N = 5,314) scored higher on L2 engagement than the latter (M = 3.88, SD = 1.07, N = 15,302); that is to say, it appears that if a participant was the only child of his/her family, (s)he tended to have a higher L2 engagement score. Regarding the other seven variables, each of them statistically significantly correlated with L2 engagement, with their Spearman correlation coefficients ranging between 0.117 (monthly family income) and 0.563 (L2 proficiency); parental attention had the third highest effect size (0.307), which already exceeded the ‘very strong’ benchmark (0.30), suggesting that the more parental attention a respondent could receive for his/her L2 learning, the higher his/her L2 engagement.

The second round of data checking aimed to ensure that the assumptions (e.g., normality and homoscedasticity) for regression were met, in order to echo recent calls for the good practice of assumption checking (Hu and Plonsky, 2019). For example, when checking for potential outliers, based on several rounds of casewise diagnostic analyses, about 750 cases, which had a standardised residual greater than 3 or smaller than−3, were deleted from the initial sample (N = 21,370). The revised sample size for later analysis was 20,616.

After the above two rounds of checking, we performed a series of hierarchical regression analyses. In each hierarchical regression, we entered each of the eight predictors, one by one, into each of the eight models (or ‘blocks’ as called in SPSS). As mentioned above in the analytical strategy, when the predictors are entered one by one into regression, the entry order is crucial. A total of eight predictors will generate 40,320 (8×7×6×5×4×3×2×1) possible sequences and hence up to 40,320 different scenarios; that is to say, for each predictor, there could be 40,320 different effect size values. Hence, providing a range of effect size values for each predictor is much more informative and comprehensive, although many studies employing hierarchical regression simply report one single effect size value. Two sets of important findings emerged from our hierarchical regression analyses.

The first set included (1) the eight predictors (see Model 8 in Table 4) statistically significantly (p < 0.0005) predicted L2 engagement and explained a total of 41.3% in the L2 engagement variance (adjusted R² = 0.413) and (2) each of the other seven models statistically significantly (p < 0.0005) L2 engagement.

The second set of important findings comprised the ranges of the effect size 𝛥R² for each of the predictors: L2 proficiency (21.7–34.2%), study time (2.4–11.9%), parental attention (1.4–10.0%), parental coaching frequency (0.7–6.7%), mother’s educational qualification (0–2.8%), father’s educational qualification (0–2.6%), income (0.1–1.3%) and being only child or not (0–0.7%). It is worth noting that for the first two variables, their maximums in the effect size range went above the very large benchmark (9%), and their minimums exceeded the large benchmark (2%), suggesting that both L2 proficiency and study time were very important predictors for L2 engagement. Regarding parental attention, its effect size maximum also exceeded 9%, although its minimum was lower than 2%, meaning that parental attention could also be a very important predictor, similar to study time. In contrast, parental coaching frequency, mother’s educational qualification, father’s educational qualification, respectively, had a maximum effect size higher than the large benchmark, which indicated that they could be important predictors for L2 engagement. Regarding the last three predictors, their effect size minimums could drop to 0.1% or zero, which showed that they might exert negligible or nil effect on L2 engagement and hence were relatively unimportant.

Table 4 summarises the key information of one example from the 40,000+ hierarchical regression scenarios predicting L2 engagement. In this scenario, parental attention was entered into the first block, parental coaching frequency the second, study time the third, being one child or not the fourth, monthly family income the fifth, mother’s and father’s educational qualification the sixth and seventh, respectively, and finally L2 proficiency; each block statistically significantly (p < 0.0005) added to the prediction of L2 engagement. The 𝛥R² column in Table 4 revealed the most important findings: (1) L2 proficiency and parental attention, respectively, explained 21.7% and 10.0% of the variance in L2 engagement, which exceeded the very large effect size benchmark (9%), (2) study time accounted for 6.5% of the L2 engagement variance, which was higher than the large effect size benchmark (2%), (3) frequency of parental coaching contributed 1.9% in the L2 engagement variance, which nearly met the large effect size benchmark and (4) the unique contribution to the L2 engagement variance, respectively, from family’s monthly income (0.3%), mother’s education (0.2%), father’s education (0.1%) and being only child or not (0.1%) fell below the small effect size benchmark (0.5%) and hence seemed negligible. All in all, in this hierarchical regression scenario, L2 proficiency, parental attention and study time turned out to be/emerged as particularly important in terms of predicting L2 engagement, and frequency of parental coaching was also important. In addition, the positive β values (see Appendix 4) indicated the corresponding positive links; for example, the positive β (0.132 in Model 8, Appendix 4) for parental attention indicated that the more attention a participant received from his/her parents, the higher L2 engagement level he/she had; similarly, the higher L2 proficiency that a student had, the more engaged (s)he was with regard to English learning; the same could also be said about the other important predictors (e.g., study time).