The COVID-19 pandemic has been the most significant challenge that educational systems have faced (Daniel, 2020). According to the United Nations Educational Scientific and Cultural Organization (UNESCO), from April 2020, over 1.5 billion students worldwide from primary to tertiary levels could not attend school due to COVID-19; therefore, a massive number of communities were forced to seek quick solutions through digital learning platforms (Teräs et al., 2020).

In a matter of a few days, institutions adapted their traditional face-to-face in-class teaching methods to online learning. This fast change conducted by institutions revealed gaps and shortcomings in how new ways of teaching and learning were adopted by educational institutions (Teräs et al., 2020). The closure of schools due to the COVID-19 pandemic affected teaching, assessment, and learning (Tarkar, 2020). Teräs et al. (2020) raised the question, “how can the educational systems and individual learners cope with the exceptional situation?” (p. 864).

Although acquiring the training to succeed in a distance learning environment is possible, it requires special arrangements (Daniel, 2020). The COVID-19 pandemic forced academic institutions to “mobilize resources to overcome [barriers] in the short term, paying special attention to the digitalization of learning processes and offering specific technical training to professors, administrative staff, and students” (García-Morales et al., 2021, p. 6). In agreement with Abdrasheva et al. (2022), the rapid change toward online teaching had an impact on educational quality.

The lack of previous experience of teachers in virtual teaching environments was a limitation that generated difficulties in the preparation of the material class, active student participation, and assessment problems (García-Morales et al., 2021). Even though e-learning was a solution to keep teaching and learning possible, there was a main concern regarding the engagement of students with the new digital channels (Tarkar, 2020). From the outset, many studies emerged in terms of the various actions that were developed in educational institutions to face this challenge.

Regarding the studies carried out, it is possible to find interventions (Karsenti et al., 2020), experiences of the actors in educational processes (Roy et al., 2020), the relationship with the adaptation of face-to-face methods, the combination of digital and traditional resources (Fabriz et al., 2021; Medina-Gual et al., 2021), or peer tutoring (Pineda and Ruiz, 2021).

The COVID-19 pandemic had an impact on education worldwide. However, Ober et al. (2023) highlighted that there is currently still a lack of understanding of the impact specifically on the learning process of students during the pandemic period (2019–2020). The research available at the time of this article is generally related to speculations (Pokhrel and Chhetri, 2021), studies of young people’s perceptions of their learning (Napierala et al., 2022), changes regarding three scenarios such as on-site synchronous, online, only and blended (Graça and Lopes, 2023), and advantages or disadvantages of face-to-face learning versus e-learning (Eslamian et al., 2023).

Along with the previous ones, the impact of confinement began to be measured in dimensions such as mental health (Urrutia et al., 2021, 2022; Wu et al., 2023), social behaviors (Cleonares et al., 2021) of teachers and students, and the development of learning and skills of the latter (Hammerstein et al., 2021; León et al., 2023; Uğraş et al., 2023), among which mathematical thinking and reading at different educational levels are highlighted (Ardington et al., 2021; Lewis et al., 2021; König and Frey, 2022) and where some results show inequality according to vulnerable conditions (Schuurman et al., 2023) and income and academic levels (Moscoviz and Evans, 2022), among others. Some of these studies noted improvement (Starling-Alves et al., 2023), and others do not show effects at higher levels (Hodges et al., 2020; Gore et al., 2021; Tomasik et al., 2021; Ludewig et al., 2022).

According to Sanz and Tena (2023), “in the summer of 2020, when it became clear that the impact of the pandemic on educational provision would last longer than initially expected, new analyzes increased the forecast of learning losses caused by the closure of schools” (p. 18). What the COVID-19 pandemic created was a challenge for educational institutions, as Daniel (2020) states: “Many in the COVID-19 student cohort will be concerned about suffering long-term disadvantages, compared to those who studied ‘normally’, when moving on to another level of education or entering the labor market” (p. 92). Therefore, the impact of the pandemic prompted new analyses of the increase in learning losses and the causes of this phenomenon.

Thus, extrinsic, intrinsic, and relational elements were identified in the students’ learning processes. The extrinsic elements include access to technology (devices), connectivity or Internet access, ad hoc physical space, and teaching experience, among others. One of the first challenges was ensuring that teachers, students, and parents had access to the appropriate devices (Khirwadkar et al., 2020). “Involving teachers and students in the development, implementation, and use of educational technology can have an impact on the success of technology in supporting meaningful teaching and learning” (Teräs et al., 2020, p. 865). However, a different study mentioned that during the COVID-19 pandemic, it was not just about teachers being prepared but during those months, teachers had to deal with a 24/7 schedule, including addressing students’ personal issues (Borba, 2021). In the intrinsic elements, attitudes (people’s feelings toward online learning), student autonomy and self-regulation are identified. According to Bringula et al. (2021), higher levels of motivation, together with educational access and the proper environment, benefit online learning.

Relational elements, on the other hand, refer to interaction, feedback, and trajectory correction processes. For example, a study conducted with more than 1,000 educators in Turkey indicated that the lack of interaction between students and educators hindered the opportunity to provide feedback and teach according to individual interests. Other concerns were the barrier of being unable to carry out reliable learning assessments and the lack of knowledge to evaluate the student’s knowledge and skills (Korkmaz and Toraman, 2020).

Nowadays, research continues to be carried out. The results demonstrate the impact on learning, focusing on specific knowledge or skills such as reading performance and mathematical thinking (Lewis et al., 2021). In this way, the following section shows these kinds of research.

Learning mathematics involves consistent and regular guidance from teachers through collaborative opportunities with students. For example, this could be done through breakout groups (Khirwadkar et al., 2020). According to Scheiner (2016), mathematics has a spiral cumulative structure, which means that learning requires a specific didactic, very often fraught with difficulties, because “It is widely acknowledged that the complex phenomena of knowing and learning processes in mathematics need pluralistic frameworks in order to adequately address the many facets of mathematical learning” (p. 165).

Mathematics education thrives in face-to-face teaching because it requires students to understand the abstract concepts of mathematics. The way teachers unfold mathematical knowledge requires implementing pedagogical strategies, such as problem-solving and collaborative work, among others (Khirwadkar et al., 2020).

As stated by Bringula et al. (2021), individual development regarding learning mathematics is related to mathematics self-concept, which refers to “one’s academic self-perceptions of one’s general ability in school. It also refers to the interests of students toward a particular course” (p. 1). Mathematics has to do with the belief of each person about the ability they have to learn the subject, perform well, and succeed in tests, including personal ability compared to others (Bringula et al., 2021).

The discussion regarding mathematics education and new technologies has been around for more than 30 years (Borba, 2021). Most mathematics research has been supported by empirical, quantitative data to prove a given teaching method better than others. Güzeller and Akın (2012) compare two kinds of delivery for teaching mathematics. One is web-based mathematics instruction (WBMI), and the other is traditional mathematics instruction (TMI). Results were significantly different, with better results on the WBMI. In 2014, Kim et al. (2014) showed that motivation, self-efficacy, and cognitive processes were key to online mathematics progress.

Highlighting that academic emotions represented 37% of the variation in students’ achievement, Cho and Heron (2015) concluded that motivation influenced achievement and that both motivation and emotions were related to satisfaction. Therefore, a set of recommendations was given to teachers: “enhance students’ self-efficacy, design supporting tools in online courseware, provide course orientation, provide SRL support through social media, and restructure the format of the course” (p. 4).

In addition to the above, the challenges teachers faced in the area of mathematics in the time of COVID-19 were added. Online teaching created an opportunity for interactions within the mathematical community regarding ideas on teaching and learning (Khirwadkar et al., 2020).

Teaching mathematics online faces a few obstacles: the large number of students in the class, the lack of a device for every student, the quality of the Internet connection, teachers’ lack of experience, and the type of techniques performed by the students (Aloufi et al., 2021).

Recently, qualitative research has played a more important role in proving a point, complemented with quantitative information. The focus should not only be on teachers but also on the students’ experience of this new reality (Borba, 2021).

As stated by Diliberti et al. (2023), studies have documented a large decline in students’ mathematics achievement during and after the COVID-19 pandemic. The reason might have been the lack of preparation prior to the pandemic because other studies are showing that teaching mathematics in virtual contexts is positive, but it requires the training of teachers, resources invested, and technology (Aloufi et al., 2021).

Prior to the COVID-19 pandemic, Al-Saeed and Al-Abed (2018) conducted a study to identify the effectiveness of using virtual classes in mathematics and logical thinking among tenth-grade students. They found statistically significant differences between the experimental and the control groups; the test results were in favor of the experimental group. In a different study conducted in Saudi Arabia in 2019, a positive trend was found in students and teachers using virtual classes; furthermore, the level of satisfaction is related to the level of technology used (Al-Ahmari, 2019).

However, during the pandemic, it seems that virtual classes presented a more negative scenario. A different study conducted in Pakistan stated that online education was not a good option in poor countries because of a lack of technical and financial resources regarding connectivity (Aloufi et al., 2021). Similarly, 56% of students in Zambia mentioned they did not have sufficient access to resources and technology to deal with online classes. Therefore, they considered that the best way to learn mathematics was with face-to-face interactions between teachers and students. The conclusion of the study considers the importance of infrastructure in these kinds of contexts or countries to reach better results in the learning process. The results showed that the main problem in the learning process was limited access to technology or infrastructure (Mukuka et al., 2021).

Aloufi et al. (2021) concluded that “the reason for the appearance of the general result with a high degree may be attributed to the technical features that characterize the educational platform that supports the interactive teaching approach in virtual classes” (p. 108). Similarly, Borba (2021) mentioned that the risk increased in regions without well-paid teachers and a lack of proper devices. The availability of more devices is crucial for ensuring the continuity of the learning process. Still, teachers play a major role in improving and sustaining the mathematics self-concept of online learners (Bringula et al., 2021).

For Bringula et al. (2021), mathematics performance went beyond being technically prepared. The real problem was the sudden change from having the opportunity to an online environment to a mandatory, unique approach to learning.

Under the authority of Khirwadkar et al. (2020), enactivism is important for learning mathematics, which “claims that cognition emerges from a network of interactions among agents and their environment.” (p. 44). New methods for teaching mathematics are required to be created. “Viewed through an enactivist lens, turning to an online platform was not about adapting face-to-face pedagogical strategies by simply following a prescribed curriculum towards some optimal goal through a prescriptive logic (…). It was rather about finding other pedagogical strategies while maintaining the viability of the mathematics education system and moving towards the proscriptive logic” (p. 45).

Regarding reading comprehension, as stated by Dong et al. (2019), “reading comprehension is the ability to obtain meaning from the given text through the interaction between the word decoding process and the application of prior knowledge” (p. 1), and involves understanding written language and, therefore, is composed of a set of linguistic elements as well as cognitive skills (Foorman et al., 2018). In the same way, Ludewig et al. (2022) highlighted that reading performance represents the ability of students to extract relevant information from narrative and informative texts, which allows them to understand, use, and reflect on written texts in areas of life that are relevant to the individual and required by society. Thus, like Neira et al. (2015), we assume that this skill is essential for university academic success, so it is expected that students enter higher education already with sufficient reading skills.

Reading achievement involves multiple levels of comprehension, such as decoding and language factors (Foorman et al., 2018), executive functions (Butterfuss and Kendeou, 2018), and inferential dimension (Ahmed et al., 2016). The decoding and language factors that explain reading comprehension are phonological awareness, decoding fluency, and language measures, such as listening comprehension, vocabulary, and syntax (Foorman et al., 2018). Regarding executive functions, the central components are inhibition, change, and updating. These functions can help explain complex interactions between the reader, the text, and the discourse context (Butterfuss and Kendeou, 2018).

Concerning the inferential dimension, the study by Ahmed et al. (2016), evaluating the direct and inferential mediation reading comprehension model (DIME of Cromley and Azevedo, 2007) in middle and high school students, concludes that it is important to highlight the importance of knowledge and vocabulary. It also increased the importance of inference. Thus, higher education students must master text comprehension, where they need to have advanced decoding, that is, recognition of a greater number of words and inferential mediation. In addition to the above, students’ learning styles, such as teachers’ instruction and students’ self-regulated learning, still generate debate in the context of reading comprehension (Dong et al., 2019).

However, reading achievement, such as mathematical thinking, was another of the competencies that were affected during the pandemic at different academic levels. Studies such as that of Lewis et al. (2021) conclude that, on average, during the school year affected by the COVID-19 pandemic, primary and secondary students made progress during the 2020–2021 school year. The historical trends show greater progress before than the one of the pandemic, 3 to 6 percentile points. Similar to the results of the previous study, the study by Schult et al. (2022) reports that a cross-sectional study conducted in secondary schools in Germany indicated a drop in reading achievement in 2020. The reading comprehension test applied included between 32 and 38 items that evaluated reading processes such as recovering information and ideas expressed explicitly, making inferences, interpreting ideas and information, and evaluating content and textual elements.

In the same way, a nationwide study in the USA using the reading test scores of 5 million students in grades third to eighth from the first 2 years of the pandemic revealed scores that were 0.09 to 0.17 lower standard deviations, relative to peers in the same grade in fall 2019, with the largest impacts on grades third to fifth (Kuhfeld et al., 2023).

Concerning the case of Mexico, there is evidence of learning loss due to the COVID-19 pandemic in reading and arithmetic from two household surveys carried out in 2019 and 2020 in children aged between 10 and 15 years (Hevia et al., 2022). Nevertheless, there is little information related to high school students ranging from 16 to 18 years, the group of students under observation in this research, since these students are the ones entering university for the last 3 years. Therefore, this article aims to measure the impact of the pandemic on the learning process of students entering higher education, specifically on mathematical thinking and reading comprehension, both important skills for every undergraduate student. Our objective is to respond to the following questions:

The questions presented above allow us to infer the following hypotheses:

A cross-sectional study of descriptive scope was developed. This study was based on a non-probabilistic convenience sample, and the chosen cases were young people who decided to take the exam to enter university. For this purpose, a study was carried out where data were collected from young people who were in their last year of high school studies during the years 2016 to 2023 and opted to apply to the university.

For the mathematical reasoning module, the mean score in the pre-pandemic group was 1,123 points with a standard deviation of 100 points, while the average result for the post-pandemic group was 1,069 with a standard deviation of 102 points. Figure 1 shows the change in the distribution from one group to the other. Table 3 has the distribution statistics for both pre-pandemic and post-pandemic distributions.

The binomial model estimates the posterior distribution for the rate parameter θ with a mean value of 0.705 for the pre-pandemic group and a mean value of 0.614 for the post-pandemic cohort. The ratio of the θ parameters has an average of 0.871 with a standard deviation of 3.96×10^-4. The maximum density interval for the ratio is (0.870, 0.872), which suggests a decrease in the average mathematical thinking achievement of approximately 12.8%. Table 4 gives a summary of the distribution for the parameters θ and the ratio. Additionally, the analysis was performed by separating the post-pandemic cohort by year. This showed a yearly decrease in the θ parameter ranging from 0.701 in 2021, 0.594 in 2022, and 0.541 in 2023.

The reading comprehension module has an average pre-pandemic score of 1,137 points and a standard deviation of 90.9 points. The post-pandemic group has a 1,103 mean with a standard deviation of 85.5 points. The change in distribution is shown in Figure 2. As with the mathematical thinking distributions Table 3 has the distribution statistics for both pre-pandemic and post-pandemic distributions.

The model estimates for the posterior distribution of the rate parameter θ have a mean of 0.729 for the pre-pandemic group, while the post-pandemic group has a mean value of 0.672 for the post-pandemic cohort. The ratio of the θ parameters has an average of 0.922 with a standard deviation of 3.79×10^-4. The maximum density interval for the ratio is (0.921, 0.922), suggesting a decrease in this competency of approximately 8%. Table 5 gives a summary of the distribution for the parameters θ and the ratio. The same yearly post-COVID analysis was conducted for this module. In this case, the θ parameter did not experience a consistent decrease, going from 0.701 in 2021, to 0.644 in 2022, and 0.666 in 2023.

Utilizing the posterior distributions of the ratios for both competencies, we can compare them to determine which one was more significantly impacted by the pandemic. The posterior distribution for the difference in ratios has a mean of −0.051 and a standard deviation of 5.516×10^-4. The highest density interval with a 95% probability was estimated to be (−0.0519, −0.0497). This suggests that there was a greater effect on math competency than on reading comprehension. It is important to consider that math scores were already lower. This pre-existing disparity suggests that the COVID-19 pandemic might have further exacerbated existing inequalities.

The aim of this research was to respond to three hypotheses as to whether emergency remote teaching decreased the average mathematics proficiency of students entering college, whether emergency remote teaching decreased the average reading achievement of students entering college, and whether the impact of emergency remote teaching was greater on mathematics skills than reading achievement. The three of them were found to be true. Emergency remote teaching decreased the average mathematics proficiency and reading achievement of students entering college, and mathematics skills reported a greater impact than reading.

According to Hypothesis 1, emergency remote teaching decreased the average mathematics proficiency of incoming college students by 12.8%, as documented by Diliberti et al. (2023). These results are perhaps a consequence of what was studied by Aloufi et al. (2021), who state that online mathematics teaching faced some obstacles, such as the number of students in the class, the lack of a device for each student, the quality of Internet (Mukuka et al., 2021), and teachers’ lack of experience in distance education, unlike the study by Güzeller and Akın (2012), whose results favor the teaching of mathematics in the web-based instruction model. This is due to its planning as opposed to emergency remote teaching (Hodges et al., 2020).

Regarding Hypothesis 2, emergency remote teaching decreased the average reading achievement of students entering college by 8%. That is, the ability to identify, interpret, and evaluate the form and content of various written materials in the areas of study, literature, and social participation decreased. The result coincides with the works of Ludewig et al. (2022), who report a substantial decrease in average reading performance, which corresponds to one-third of a year of learning, even after controlling for changes in student composition. In the same vein, the study by Kuhfeld et al. (2023) suggests that students are at risk of having reading difficulties and will need specific support to develop and strengthen these fundamentals.

Hypothesis 3 was also true because the impact of emergency remote instruction was greater on math skills than on reading achievement. Our results confirmed what was stated by Abdrasheva et al. (2022), who mentioned that the rapid change toward online teaching had an impact on educational quality, both in mathematics and reading comprehension, the former being greater as in the case of what was reported by Schult et al. (2022).

Bringula et al. (2021) found out that the most problematic aspects of online learning are Internet connection and electrical power availability. Khirwadkar et al. (2020) believed that making sure teachers, students, and parents had access to the proper devices was one of the first challenges. However, we believe those were not the main issues that generated the decrease in academic performance after COVID-19. The most probable causes, according to the profile of students examined, were closer to the ones mentioned by Borba (2021) regarding students’ personal issues faced during those months. In the same way, Bringula et al. (2021) considered that motivation, attitudes toward online learning, and autonomy might be the main elements for the results.

There are studies conducted in countries with adequate technological infrastructure for online education that have found no significant impact on students’ performance (Kim et al., 2014; Aloufi et al., 2021). That might be the case for the group of students included in this research, and therefore, their academic performance was supposed to have remained the same, but that was not the reality. The main driver for this variation of trend might be the sudden change in the learning method, similar to the conclusions offered by Bringula et al. (2021). We believe that when technology and the willingness of the students to learn through online methods are involved, in addition to having prepared professors, the results should not have an impact. However, when adopting an online education system as something mandatory or sudden, such as the one created by COVID-19, the situation goes beyond technical or methodology issues but more about emotions and feelings, similar to the findings of Cho and Heron (2015) and Kim et al. (2014), who concluded that motivation and emotions influence achievement.

This study is one of the first to provide quantitative information about the impact of COVID-19 on education for high school students. We believe it helps the field because it progresses from the speculative studies known to the time this research was completed. Therefore, we wanted to contribute to filling the gap when it comes to the lack of systematic studies demonstrating the impact of learning through a standardized evaluation and focusing on specific classifications of knowledge or basic skills, such as reading comprehension and mathematical thinking. Furthermore, it concentrates its efforts on analyzing mathematics and reading learning for high school students; we had the study by Lewis et al. (2021), who directed their research for third- through eighth-grade students.

We believe the results of this research are relevant because of the importance of mathematical and reading skills for university success. According to Neira et al. (2015), reading skills are required for excellence in higher education. If similar results were obtained in different contexts and countries, the risk and damage to the world would increase because of the importance of higher education in the economic future of any country (Tarkar, 2020).

In addition to the above, this study also allows specific interventions to be made with these students who enter university to contribute to their professional careers and to fill the academic gap this group of students has because of COVID-19.

It is of utmost importance to mention that this study has limitations. The first one is that even though emotional factors and the lack of the proper resources might be related to the decrease in the academic results found for this article, this research does not directly measure these factors and represents a limitation of the study. Further research might be directed to academic environments that showed positive results using online teaching and confirm if the need for emergency remote teaching, which included a different set of motivations and emotions, might impact academic results. Another limitation is that the standardized test is not one that all high school students willing to get into college have to take, and it could be that only the students with higher academic achievements took the test. One last limitation could be the different formats used to take the test before and during the pandemic when a web-based format was required. We invite future research to focus on the follow-up of the academic progress of the students entering college during the pandemic years to find out if there is any academic gap and, if so, the most effective interventions conducted by academic institutions.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethical approval was not required for the study involving human participants in accordance with the local legislation and institutional requirements. Written informed consent was obtained from the individuals for participation in the study and for the publication of any potentially identifiable images or data included in this article.

CO-B: Conceptualization, Investigation, Methodology, Writing – original draft, Writing – review & editing. HR-P: Conceptualization, Investigation, Methodology, Project administration, Writing – original draft, Writing – review & editing. SM-P: Data curation, Formal analysis, Writing – original draft, Writing – review & editing, Investigation.