In the theoretical frameworks outlined above, the aspects listed below are assumed to be conducive to student outcomes (such as student achievement, student effort, and student motivation) in distance education during the school lockdown caused by COVID-19.

Regarding student achievement (assessed by self-measures and external ratings), existing studies have identified the following positively correlated individual predictors: student age (Huber and Helm C. 2020b), students’ subject-specific (e.g., grade point average) and interdisciplinary skills (Züchner and Jäkel, 2021), students’ self-reported ability to use digital media before school closure (Züchner and Jäkel, 2021), and students’ self-organization skills (Huber and Helm C. 2020b). Further, students’ engagement (as reported by their parents, Steinmayr et al., 2021) and students’ self-reported invested learning time (Huber and Helm C. 2020b) positively predict learning success during COVID-19 pandemic-related distance education. Lastly, positive emotions (Huber and Helm C. 2020b) and leisure activities that promote learning, such as reading (Champeaux et al., 2020), are positively associated with student achievement during lockdown. Significantly positively correlated contextual predictors include features of the quality of distance education, such as teacher feedback on learning tasks (Huber and Helm C. 2020b; Steinmayr et al., 2021; Züchner and Jäkel, 2021) and student/parent-teacher communication (Steinmayr et al., 2021) as well as parental satisfaction with school support (Nusser et al., 2021). Negatively correlated individual predictors of learning success include the learning time invested by students at home (Züchner and Jäkel, 2021), the time spent at home in front of the screen, the time spent for extracurricular activities (Champeaux et al., 2020), and negative emotions experienced by students during the pandemic (Huber and Helm C. 2020b). The assessment of student solutions by teachers (Steinmayr et al., 2021) and regular family support for learning (Züchner and Jäkel, 2021) represent negatively correlated contextual predictors of self- and externally rated student achievement. The negative influence of invested learning time and family support is surprising. These unexpected findings are good examples of the challenging interpretation of cross-sectional results. Züchner and Jäkel (2021) argued that these findings might be interpreted in terms of reversed causality:

“Rather, the results of this sample indicate that those who spend more time on tasks and need regular parental support more often have difficulties coping with tasks—possibly family support and the amount of learning time is increased when task coping is less successful.” (Züchner and Jäkel, 2021).

In the study by Huber and Helm C. (2020b), the indicators of the quality of the teacher-student relationship and the quality of the family’s handling of the crisis also showed surprisingly weak negative effects on self-reported learning success. However, the corresponding bivariate correlations were positive. Hence, these findings should be interpreted with caution. Steinmayr and Christiansen (2020) also argued that the observed unexpected negative influence of parents’ assessed language competence of their children on perceived learning success in lockdown should be interpreted with caution, as no theory is able to plausibly explain this negative influence, and as bivariate correlations turn out positive.

When predicting students’ learning time during lockdown, the following variables have been identified as positively correlated individual predictors: age, gender, performance, diligence, emotions. Regarding students’ ages, the findings are heterogeneous. While Grewenig et al. (2020) pointed out that the reduction in learning time due to school closure was significantly less pronounced for younger students, Grätz and Lipps (2021) reported that this reduction was greater for secondary school students than for students older than 18 years. Grewenig et al. (2020) also reported that the reduction in learning time was significantly higher for boys than for girls. Students’ prior performance also matters: low achievers had a significantly greater reduction in learning time than high achievers. In addition to performance, students’ independence (Huber and Helm C. 2020b) and diligence (Grewenig et al., 2020) predict the amount of learning time invested during COVID-19 pandemic. Lastly, at the student level, positive and negative emotions (Huber and Helm C. 2020b) are important positive predictors of students’ effort. However, the positive influence of negative emotions should not be overinterpreted, as this effect is not observable in bivariate analyses and could therefore represent a methodological artifact (see Steinmayr et al., 2021). Positively correlated contextual predictors include school type (more learning time is invested by students from academic track, Grewenig et al., 2020), teaching quality characteristics such as the intensity of teacher support and task control by the teacher (Dietrich et al., 2020; Huber and Helm C. 2020b), learning support from classmates or friends (Dietrich et al., 2020), as well as home/family resources such as regular learning support, handling of the crisis, and technical equipment at home (Huber and Helm C. 2020b; Züchner and Jäkel, 2021). So far, no negatively correlated individual predictors have been identified in the previous studies available to us. By contrast, a few negatively correlated contextual predictors of learning time during COVID-19 pandemic-related school closures were observed. In particular, a home learning environment that is judged to be less conducive to learning (Dietrich et al., 2020) and errands for parents that keep students from learning (Huber and Helm C. 2020b) have a negative effect on the amount of time invested in learning in distance education. ² It should be pointed out that there are also studies (e.g., Nusser et al., 2021) that could not identify any significant predictors of students’ learning time during school closure. This indicates that predicting student learning time in distance education is likely to be difficult.

“Overall, the regression results show that neither gender, type of school attended, nor reading skills measured in the previous year (start of school year 2018/2019; grade 7) have an impact on reported learning time during school closures. The educational background of the parents also has no influence on the mean reported learning time. This means that these characteristics hardly explain the differences in the average learning times of the students per week during the school closures. This is also confirmed by the low variance explanation of 6 percent (R2 = 0.06).” (Nusser et al., 2021, p. 40, p. 40).

Regarding the prediction of students’ intrinsic learning motivation during lockdown, the following positively correlated individual predictors proved significant: In line with self-determination theory, students’ individual experiences of autonomy and competence are positive predictors of learning motivation in distance education (Holzer et al., 2021). Furthermore, a number of students’ learning strategies (study goals and plans, meta-cognition, time management, Pelikan et al., 2021), as well as their learning engagement and diligence (Steinmayr et al., 2021), are positively correlated to students’ intrinsic motivation to learn. Lastly, students’ hours spent at home on extracurricular activities and learning are significant positive predictors of students’ emotional status (Champeaux et al., 2020). Positively correlated contextual predictors were identified in the study by Steinmayr et al. (2021). Teaching quality characteristics, such as student/parent-teacher communication, the frequency of learning tasks sent, and teacher feedback, represent significant predictors of students’ intrinsic motivation. Negatively correlated individual predictors of students’ learning motivation are students’ age (Huber and Helm C. 2020b; Zaccoletti et al., 2020), students’ negative emotionality (Steinmayr et al., 2021), students’ individual social inclusion and self-regulated learning (Holzer et al., 2021), and students’ procrastination (Pelikan et al., 2021). Negatively correlated contextual predictors are the grading of learning tasks by teachers (Steinmayr et al., 2021).

Having presented the state of research regarding predictors of student outcomes in COVID-19 pandemic-related distance education, we want to point out methodological issues that make it difficult to compare and contrast the findings of the different studies. First, these studies used different effect sizes (e.g., unstandardized vs. standardized coefficients). Second, some studies (e.g., Huber and Helm C. 2020b; Steinmayr et al., 2021; Züchner and Jäkel, 2021) reported possible suppression effects, which can lead to unexpected findings. ³ Third, the studies used different informants; that is, they were either based on data from student surveys or from parent surveys. Given the different perspectives associated with different validities and biases, the question arises as to the extent to which the findings of these studies, which are based on different informant groups, are comparable or similar.

The findings in the present paper are based on data from the second measurement occasion of the school-barometer survey (www.schul-barometer.net), which was conducted from 11 June 2020 to 22 July 2020 in Germany, Austria, and the German-speaking regions of Switzerland, including all stages of compulsory education (i.e., primary, lower secondary, and upper secondary school). We developed online questionnaires for students, parents, and teachers. We carefully recorded the same constructs with the same items in all questionnaire versions so that we could perform multiple informant analyses of predictors of student outcomes from all three perspectives.

As the response rate was low for the Swiss sample, as well as for the primary school type, we decided to exclude these samples and focus on secondary school data from Germany and Austria. In order to maintain a sufficient sample size for all three survey groups, we decided against conducting the analyses based on data from one country. However, pooling the data from Germany and Austria runs the risk of further increasing the heterogeneity in the data. Although the education systems and also the education policy responses to the COVID-19 pandemic are very similar in the two countries, they are not identical. Nevertheless, in distance learning aspects of different educational systems are assumed to be of less importance when predicting students’ learning outcomes because students do learn outside the school system in their homes and families which are assumed not to differ systematically between the two countries. Nevertheless, to exclude the influence of the country on the findings, we included country as a control variable in the subsequent analyses.

Table 1 provides information on the composition of the three assessed informant groups (students, parents and teachers). The ratio of the school type is quite similar across the parent and teacher sample (54% and 51%). Only in the student sample low secondary students are less strongly represented (38%).

For reasons of anonymity and ease of data collection, we refrained from collecting matched data. That is, we did not design the data collection process in a way that would have allowed links to be made between students, parents, and teachers; for instance, by means of a self-generated code. Thus, the analyzed samples are “separate samples of each group.”

Table 2 provides an overview of the constructs assessed in the school-barometer survey and analyzed in this paper. All of the items were self-developed but validated in previous research (Huber S. G. and Helm C., 2020). The response options for all the items ranged from 1 (strongly disagree) to 5 (strongly agree). Only student effort was assessed with an item that ranged from 1 to 40 h a week.

Information on the descriptive statistics (mean, standard deviation) and the reliability (Cronbach’s alpha) of the constructs are provided in Table 2 for the three different informant groups. In examining the mean values of the constructs, lack of technical equipment, lack of parental support, and student’s workplace at home showed comparatively low/high approval ratings in the student and/or parent sample, which may indicate bottom/ceiling effects. That is, many students and/or parents (but not as many teachers) reported that the lack of technical devices, the lack of parental learning support, and the quality of students’ workplace at home were not detrimental for distance education. These three predictors showed quite low standard deviations. Due to the low variance in these constructs, they are less likely predictive of student outcomes and less likely to be predicted by other variables themselves.

A closer look at the group differences in the descriptive statistics offered the following insights: The effect size (Cohen’s d) of the group mean differences (of at least one of the three possible group comparisons) is relevant (i.e., d > 0.5) for all constructs, except for student effort. The reported average number of hours students invest does not significantly differ between the three informant groups. However, regarding all other constructs several different pictures emerged. In general, student ratings have the highest average approval rate, except regarding the two instructional features, learning support and structure, which were rated more positively by the teachers. While teachers tended to rate instructional features comparatively positively, parents tended to rate these features comparatively less positively. By contrast, students and parents rated home learning resources (parental support, workplace at home) comparatively positively, while teachers rated these resources comparatively less positively. Lastly, students rated their learning progress and learning motivation, as well as their self-organization skills, significantly better than parents and teachers did. The descriptive results also showed that most of the constructs assessed by teacher ratings have comparatively low standard deviations. Teachers seemed to agree more strongly regarding the evaluation of the constructs assessed.

Regarding the internal consistency of the scales, the low alpha values (below 0.6) of the constructs teachers’ support of cognitive activation, lack of parental support, and student’s workplace at home across all three informant groups were striking. The low values were primarily explained by the low number of items (two for each construct) that were available in the data for the operationalization of the respective construct. Across groups, the reliability values were lowest for the instructional features in the teacher sample. Beyond that, however, no further obvious deviations were discernible in the reliability values.

In Table 3 we present indices for evaluating the model fit of a) the latent correlations of all study variables (shown in Tables 4–6; Supplementary Tables SA7–SA9) and b) the latent regression analysis for each of the three outcome variables and the three samples (shown in Table 7). The indices point to an acceptable model fit for all models estimated. That is, with exception of T LI the indices are above (CFI) or below (RMSEA, SRMR) the cut-off values suggested in the literature. According to the simulation study by Shi et al. (2019), CFI and T LI values can be underestimated when large models are estimated based on small samples. Hence, we argue that our estimated models are sufficiently well able to reproduce the data (i.e., the variance-covariance matrix) and thus are reasonably consistent with the data.

See Supplementary Tables SA7–SA9 for the full correlation matrices.

Latent correlation analyses show that many of the study variables were significantly related with each other. This raises the question of whether and to what extent the predictors of student outcomes are confounded, and thus lead to spurious relationships among the predictors and the outcome variables. Therefore, there is a risk of reporting spurious correlations; that is, two variables are associated but not causally related due to either coincidence or the presence of a certain third, unseen variable. To prevent this risk of misinterpretation, we performed multivariate regression analyses that controlled for other relevant variables by including them as explanatory variables. Table 7 shows the results of nine different regression models. For each of the three informant groups, the three student outcomes are predicted by all other study variables. However, some of the very high regression coefficients—particularly when student effort was regressed on the dimensions of instructional quality (teachers’ support of individual learning: β = −1.515, teachers’ support of structure in online lessons: β = 0.922 and teachers’ maximization of learning time: β = 0.812)—and some of their huge standard errors (S.E. = 1.076–2.453) point to issues of multicollinearity and are thus not trustworthy. Moreover, the high latent correlations observed in Supplementary Tables SA7–SA9 (e.g., between teachers’ support of individual learning and teachers’ maximization of learning time, r = 0.778) also point to multicollinearity problems. These issues call for other analytic approaches that are less sensitive to multicollinearity, such as relative weight analyses.

Table 7 provides information on the latent regressions predicting central student outcome variables in the three different samples.

Tables 8–10 provide information on the results of the relative weight analyses (RWA) for the three different informant samples. Each table shows the absolute and relative R² values. Absolute R² values refer to each predictor’s absolute contribution to the total explained variance in the outcome variable (left part of the tables). The tables also show the relative size of the absolute share when set in relation to the total explained variance (right part of the tables). As highlighted above, RWA values assess the direct and indirect effects of the predictors on the criterion variables and therefore provide information on the total effect of the predictor variables on the outcome variables. Following this interpretation, Tables 8–10 present the following findings:

This study has some theoretical limitations.

First, from a theoretical point of view we did not consider reciprocal effects between our study variables, particularly between the dependent and independent variables. For instance, in studies on self-regulated learning, intrinsic motivation leads to a higher level of implemented learning strategies; accordingly, assuming and modeling self-organization skills as predictors of motivation should be carefully reflected. Our assumption was that in the context of distance learning students with higher self-organization skills are less overwhelmed and experience competence more frequently than students with lower self-organization skills. Therefore, we assumed students’ self-organization skills being a central predictor of students’ motivation. However, with our study design we cannot examine (and exclude) reciprocal effects. Nevertheless, RWA does model all indirect associations of the specified predictors (Johnson, 2000). That is, our RWA models not only estimate the direct effects of students’ motivation and students’ self-organization skills on the outcomes but also the indirect effects of each predictor via all other predictors, i.e., of a) students’ motivation via students’ self- organization skills on the outcomes; and b) students’ self-organization skills via students’ motivation on the outcomes. Thus, the reciprocal effects of predictors are therefore taken into account in our relative weight analyses.

Second, it is critical that the domain specificity of learning is not considered in the present study. Especially against the background of initial findings showing that the extent of digitally supported instruction during COVID-19 pandemic clearly depends on the subject (Heller and Zügel, 2020), the question arises as to what extent the findings in this paper can claim validity for all or at least for the main subjects equally. Steinmayr et al. (2021) showed that domain-specific teacher variables did not add to the prediction of students’ motivation, competent and independent learning, and learning progress during the school lockdown. How- ever, the latter were not assessed domain-specifically. Hence, we argue that future research should conduct appropriate analyses with domain-specific measurement instruments to shed light on this question.

Third, regarding the type of school, while only secondary school students were used here, it is well known from other studies (e.g., Helm et al., 2021) that distance education during COVID-19 pandemic-related school closures was quite different at the primary level than at the secondary level. For example, a representative parent survey in Austria at the beginning of 2021 (Helm and Postlbauer, 2021) reported that primary students received only 45 min of online lessons per day, whereas students from upper secondary schools received.

4.2 h of online lessons per day. By contrast, in primary schools 78% of the parents’ report that learning assignments are provided in paper form, while in upper secondary schools only 4% report receiving paper pencil assignments, but 83% report receiving assignments via digital learning platforms. Further studies need to investigate these different learning environments and should elaborate on differences in the determinants of learning across school levels (see, e.g., Steinmayr et al. (2021) for such analyses).

In addition, this study has some methodological limitations.

First, our study represents a cross-sectional study; hence, no causal statements can be made. Nevertheless, cross-sectional studies can provide meaningful insights into the possible longitudinal relations of variables if statistical analyzes are rooted in solid theoretical assumptions about predictors and outcomes and if central control variables are modeled. In the present study, we claim both. However, since our theoretical model assumes reciprocal relationships between the characteristics (see earlier discussion in this section), future studies should be longitudinal (see, e.g., Schober et al., 2020).

Second, our sample represents an ad hoc sample that is not representative of the student, parent, and teacher body population but controlled against different characteristics of the respective population. Particularly, the small student sample probably only represents a certain part of the lower secondary students. Therefore, strictly speaking, the findings cannot be generalized beyond our sample. However, in COVID-19 pandemic-related educational research, representative samples are rare (about only one-fifth of the 97 studies identified in the review in Helm et al. (2021) was based on representative samples). As in other current surveys, it can be assumed for our sample that people from educationally disadvantaged backgrounds, in particular, are underrepresented in the data. This could lead to an underestimation of the dispersion in some survey variables and thus their correlation with other variables. It is therefore possible that the effects reported here are too conservative.

Third, measurement invariance analysis indicates that the weak measurement invariance (i.e., equal loadings) of the construct lack of technical equipment at home was not achieved. Hence, a prerequisite for the comparison of the findings between the three informant groups was violated. According to Meitinger et al. (2020) there are various reasons for the lack of measurement invariance. On the one hand, the content of the construct may differ across groups, or the respondents attribute different meanings with the items. On the other hand, measurement invariance might be a result of other sources of measurement error (e.g., method bias). In the present study a lack of metric invariance means that the factor loadings are not equal across groups, i.e., some items are stronger indicators of the latent factor in certain groups of respondents. For the construct “technical equipment at home” the stepwise release of the equality constraints of the factor loadings shows that almost all items and all groups contribute to the missing measurement invariance. This means that the three groups of respondents attribute significantly different meanings to the items, for instance:

• Item 2 “The computer/laptop/tablet in our household is/are up to date” had a comparable low loading in the student group.

These differences show that while items 2 and 3 are good indicators of the construct in the parent and teacher group, they are not so in the student group. In contrast, item 4 seems to work well in the student group, but very bad in the parent group. One interpretation of these findings is that, when asked about the appropriateness of their technical equipment at home for distance learning, from a students’ perspective the number of opportunities to work on an electronical device is of higher importance then the quality of the internet connection and the whether the computer/laptop/tablet is up to date. For parents and teachers, it seems to be the other way around. It makes sense for teachers to think about internet connectivity first, as their focus is online teaching in distance learning. It also makes sense that the condition of the laptop is important for parents, since parents are probably especially needed by their children when there are technical problems with the user devices. These findings indicate that future studies aiming to examine students’ technical equipment for distance learning at home should first construct a measurement scale consisting of items that are interpreted in the same way by different respondent groups or should contain carefully designed bespoke instruments for different groups.

Fourth, because we did not record information about which school the students, teachers, and parents belonged to (for reasons of anonymity), we also cannot account for the hierarchical structure in the data. This could lead to biased standard errors. However, intra-class correlation (ICC) values can be assumed to be low in the case of distance learning since classes have been disbanded and learning is more dependent on individual characteristics and home conditions. Findings from studies on distance learning point to rather low values below 10% (e.g., Jaekel et al., 2021). In addition, prior research from regular schooling shows that for motivational outcomes ICC values are low anyway (e.g., Kunter et al., 2005). Nevertheless, this lack of information is a key limitation that needs to be tackled in future research.

Fifth, for reasons of anonymity and ease of data collection, we refrained from collecting matched data. That is, we did not design the data collection process in a way that would have allowed links to be made between students, parents, and teachers. Therefore, a direct comparison of ratings of the same referent object from multiple perspectives is not possible in this study. This means, that the students who were assessed from the parents’ and teachers’ perspective (in the parent and teacher sample) do not necessarily coincide with the students of the student sample. Future studies should use matched samples to test the validity of the present findings.

Finally, it should be noted that student outcomes were assessed through self- and peer-ratings. This is especially critical for outcome learning success during lockdown. A meta-analysis by Hansford and Hattie (1982) concluded that self-ratings and performance measures are scarcely associated with each other or overlap only 4%–7%. It is therefore unclear to what extent the predictors identified here are also predictive of objectively assessed student performance measures.

Studies have repeatedly shown that children’s and adolescents’ abilities to self-organize and self-direct learning are predictive of the quality and outcomes of learning processes—not only in distance education (Boekaerts and Corno, 2005; Huber and Helm, 2020b; Steinmayr et al., 2021). Hence, in regular classes and in all different scenarios in which learning takes place without the supportive structures of school (e.g., lifelong learning, distance education due to natural disasters), it is particularly important to promote these skills intensively. For school practice, this could mean focusing more than before on types of learning environments that support self-directed learning and meta-learning, that is, learning how to regulate learning processes alone and with teachers. Findings from a student survey in Austria (Lenz and Helm, 2021) show that learners who were taught according to the concept of open and cooperative learning prior to COVID-19 pandemic are significantly more likely than students from traditional classes to report that they made higher progress during distance education. They also rated the quality of the teacher–student relationship during lockdown significantly higher than students from traditional classes. Huber (2021) discusses further implications for practice and policy in education, for example, the role of technology in promoting individualization and interactivity.

For researchers, the findings imply that both descriptive and explanatory analyses of distance education may arrive at partially different conclusion depending on which group of actors is used as the source of information. Only a few predictors, such as learners’ self-direction skills, were found to be highly significant in predicting learning outcomes across all informant groups. Future studies should therefore critically reflect on the validity of single informant studies against this background by discussing possible perspective validity and bias, as well as issues of reliability. As stated earlier, longitudinal studies with representative samples (taking demographic, socioeconomic and further relevant characteristics into account) are needed.