Epidemic viruses have had a profound impact on human health throughout history. From the SARS to Ebola to COVID-19, these highly infectious agents have not only caused widespread illness and death but also profound psychological impacts on individuals and communities. COVID-19 was first detected in December 2019 in Wuhan, China. Shortly afterward, it quickly spread across the globe, becoming a pandemic by March 2020. On September 21th, 2023, there were 770,778,396 confirmed cases of COVID-19 and 6,958,499 deaths in the world (1), while as of September 21th, 2023, China had recorded 99,309,232 confirmed cases and 121,679 deaths (2). Although the World Health Organization (WHO) has declared that the COVID-19 pandemic is no longer a “public health emergency of international concern” (3), recent data from the Beijing Center for Disease Control and Prevention (CDC) shows that COVID-19 has surpassed influenza as the most frequently reported infectious disease in China since the 37th week of 2023 (September 11–17) (4). Therefore, the COVID-19 remains a highly infectious disease, at least in China.

In fact, the COVID-19 pandemic has highlighted the significant impact of epidemics on mental health. Studies show that individuals are experiencing anxiety, depression, and stress (5, 6). Certain populations, such as healthcare workers (7, 8), older adults (9, 10), children and adolescents (11, 12), low-income communities (13), and those with preexisting mental health disorders (14, 15) are particularly vulnerable to the psychological consequences of the pandemic. However, it is undeniable that university students are indeed one of the groups heavily affected by the adverse consequences of the pandemic. Studies have shown that COVID-19 has posed challenges for university students in various areas, such as social participation, sense of achievement, pursuit of goals, and academic advancement, in addition to the uncertainty it has brought (16, 17). A meta-analysis conducted on literature from 2020 to 2021 revealed that 26.0% (95%CI: 23.3–28.9%) of university students experienced depressive symptoms (18). Another meta-analysis study, conducted on literature published prior to March 28, 2021, also found that the prevalence of depression and anxiety symptoms among college students was found to be 33.6% (with a 95% confidence interval [CI] of 29.3–37.8%) and 39.0% (with a 95% CI of 34.6–43.4%), respectively (19).

In order to prevent or/and reduce the occurrence of psychological problems, it is important to understand what negative psychological disorders that individuals are experiencing during the pandemic (20–25). In fact, several questionnaires have been developed and validated across different countries to measure psychological effects of COVID-19. These include the Fear of COVID-19 Scale (26), the COVID-19 anxiety syndrome scale (27), the COVID-19 Stress Scales (28) and the COVID-19 Perceived Risk Scale (29) and so on. The measures focus on assessing various aspects such as fear, depression, anxiety, stress in relation to COVID-19. Some scales, such as the Fear of COVID-19 Scale, are widely used in China (30, 31). Some researchers, however, have proposed that the majority of scales developed in the early phases of the pandemic primarily measure acute stress responses that were commonly observed at that time (32, 33). Therefore, it is imperative to regularly verify the reliability and validity of assessment tools specifically designed for the university student population within the general community. These tools should accurately reflect the current characteristics of the pandemic and be easily applicable to the unique needs.

The COVID-19 Impact Scale (CIS), developed in 2022, is a self-report measure that specifically assesses symptoms of multiple disorders (32). It comprises of 10 items and has been found to exhibit a single-factor structure. The CIS has an excellent reliability in terms of internal consistency and the convergent and divergent validity. The CIS was initially created using an Korea University students sample (N = 2,152), and has since been translated and validated in Turkish (33) and Spanish (34). The CIS has several advantages. Firstly, it is a valid and reliable scale and demonstrates better explanatory power compared to existing scales. For instance, research has found that the CIS has incremental validity even after controlling for the FCV-19S scale (32). Secondly, the CIS effectively encompasses a broader spectrum of emotional reactions and functional difficulties despite its brevity, enhancing its comprehensiveness in measuring the impact of the COVID-19 pandemic. Scales developed during the early stages of the pandemic primarily emphasized pathological responses, such as fear (26) and anxiety (27). In summary, the CIS is a valuable tool for research and practical applications due to its superior explanatory power of COVID-19 effects and comprehensive assessment.

Accordingly, the purpose of this study was to develop a Mandarin Chinese version of the CIS and assess its factor structure, internal consistency, test–retest reliability, along with its validity. To that end, the three aims of Study were as follows. Firstly, the CIS was translated into Mandarin Chinese. Secondly, its factor structure was tested. It is expected that the Mandarin Chinese version of the CIS possess a unidimensional structure similar to those established in the previous study (32, 33). Thirdly, its internal consistency reliability, test–retest reliability, and convergent validity, criterion validity and incremental validity were estimated. It is expected that its provide a reliable estimate and convergent validity. Additionally, a positive correlation between the CIS and FCV-19S, along with negative affect, is predicted, while a negative correlation with subjective well-being is expected for criterion validity. Moreover, the CIS is hypothesized to uniquely predict psychological distress after controlling for FCV-19S for incremental validity.

Cronbach’s α coefficient was used to assess the internal consistency of the Mandarin Chinese CIS, while test–retest reliability was evaluated using Intraclass correlation coefficients (ICCs). Shoukri, Asyali, and Donner proposed that sample sizes of 55 or more are sufficient to achieve a reliability level above 0.8 at a significance level of 0.05 with 80% power (38). Our sample size (n = 86) was therefore acceptable for conducting test–retest reliability based on these criteria.

Lavaan package with R studio was used to conducted CFA (39). Since each of scores was normally distributed (skewness values <3, kurtosis values <8) (40), parameters were performed by using the maximum likelihood estimation method. Previous study suggested that the sample should have a minimum size of 100 or/and the ratio of participants per parameter estimated (e.g., items or factors) should be at least 5:1, but ideally higher (41). Therefore, our sample sizes (N = 504) met both criteria for conducting CFA. Model fit was assessed using chi-square, comparative fit index (CFI), Tucker-Lewis fit index (TLI), standardized root mean square residual (SRMR) and root mean square error of approximation. As recommended by Hu and Bentler (42), models that meet the following criteria demonstrate a good fit to the data:CFI of 0.95 or higher, SRMR of 0.08 or lower, and RMSEA of 0.06 or lower. Meanwhile, models that show CFI between 0.90 and 0.94, SRMR between 0.09 and 0.10, and RMSEA between 0.07 and 0.10 are considered to have an acceptable fit.

Criterion validity was tested by examining Mandarin Chinese CIS relationship with the FCV-19S and DASS-21 and the SWLS. Pearson correlation coefficients were calculated. Moreover, regression analyses were conducted to examine whether the Mandarin Chinese CIS contributed independently to the prediction of negative affect states measured by DASS-21 after adjustment for gender, age and FCV-19S. Components that were able to predict the change of psychological symptoms were regarded as having incremental validity. Finally, convergent validity was assessed through construct reliability (CR) and average variance extracted (AVE).

As far as we know, the CIS has only been tested for psychological properties in Korea and Turkey (32, 33). Therefore, cross-cultural and linguistic validation studies are needed. The CIS was translated into Mandarin Chinese and its psychometric properties were examined in the study. As expected, the psychometric integrity of the Mandarin Chinese CIS was supported by our findings. This conclusion was based on three main criteria.

The first criterion was the reliability of the Mandarin Chinese CIS. As for the test–retest reliability, a ICC value of 0.60–0.80 indicates good test–retest reliability, and a value above 0.80 is considered excellent test–retest reliability (43). The study found good test–retest reliability (0.640), providing support for temporal reliability of the Mandarin Chinese CIS. Regarding internal consistency reliability, Cronbach’s α lower than 0.60 were considered poor, values between 0.70 and 0.80 acceptable, and values over 0.80 indicated good reliability (44). Mandarin Chinese CIS demonstrated excellent internal consistency (0.89), which is in consistent with the findings of previous research (32, 33).

The second criterion was the validity of the Mandarin Chinese CIS. For measures assessing criterion validity, Pearson correlation coefficients between 0.50 and 1.00, between 0.030 and 0.50, and < 0.30 were considered large, moderate and small, respectively (45). As such, the criterion validity of the Mandarin Chinese CIS was supported by moderate positive correlations with negative affect, as well as small negative correlations with subjective well-being. These correlations were in the expected direction and had magnitudes that were very similar to those found in the original study where the CIS was developed (32). Of note, previous research has not found correlation between CIS and subjective well-being. The discrepancy between our findings and those of Min et al. could potentially be attributed to differences in sample characteristics, which may influence the outcomes of the same variable (33). When assessing incremental validity, regression analyses indicated that the Mandarin Chinese CIS had incremental validity in predicting negative affect beyond other factors such as gender, age, and FCV-19S. This suggests that the CIS provided unique information and explained a significant portion of variance that other factors did not account for. This finding is consistent with previous research (32, 33). To evaluate convergent validity, CR with a recommended value of 0.70 or higher and AVE with a recommended value of 0.50 or higher were used (46). the AVE value in the current study was below cut-off points (0.50), but the CR value was ≥0.80. some researcher propose that if the AVE value is less than 0.50, but the CR value is equal to or greater than 0.80, the convergent validity can still be considered acceptable (47).

The third criterion was the factor structure of the Mandarin Chinese CIS. The CFA showed evidence for the unidimensional structure of the Mandarin Chinese CIS, which compatible with previous studies (32, 33). However, some items had lower factor loadings (range 0.42–0.91), which were similar to the previous study (0.40–0.82) (33), but lower compared to the initial study (0.72–0.86) (32). While some researchers have proposed a threshold value of 0.5 for factor loadings (46), the acceptable cutoff for factor loading depends on the research’s specific purpose. For example, Stevens suggested that a factor loading greater than 0.40 is sufficient (48). To improve the usefulness of CIS, more research is necessary to provide more evidence on its factor structure in the future.

In addition to the limitations mentioned above, this study also has several other potential shortcomings. Firstly, the participants were exclusively college students, and the majority of them were Mandarin Chinese. As a result, the generalizability of the Mandarin Chinese CIS to other age groups could be limited. Secondly, the study only evaluated the Mandarin Chinese CIS’s criterion validity using the DASS-21, FCV-19S, and SWLS as comparison tools. Future research should include a clinical sample to investigate whether the Mandarin Chinese CIS can effectively differentiate between individuals with and without COVID-19. The third limitation of the study was that participants scored relatively low on the Mandarin Chinese assessment. This may have been due to the timing of data collection. On January 8th, 2023, the Chinese government classified COVID-19 infections as “Class B management” (49). As a result, the majority of college students have already been infected with the COVID-19 and are in an immune period at the time of the study. Nevertheless, our study still identified the unique predictive power of Mandarin Chinese CIS for negative emotional states. Future research could corroborate our findings by examining the predictive power of Mandarin Chinese CIS in high periods of the COVID-19, such as during winter. The fourth limitation of the study was the imbalance in the distribution of male and female participants, particularly evident in the test–retest assessments. Our survey collection process was voluntary and anonymous, making it difficult to predict or control for potential gender differences in response rates. Despite our efforts to recruit a representative sample, we encountered challenges that resulted in the gender imbalance. To address this issue, it is necessary for future research to implement targeted recruitment strategies in order to ensure a more balanced representation of male and female participants.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

The studies involving humans were approved by the Institutional Review Board at Shandong Youth University of Political Science. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their electronic informed consent to participate in this study.

QZ: Conceptualization, Formal analysis, Funding acquisition, Writing – original draft. YL: Investigation, Formal analysis, Writing – review & editing. JY: Conceptualization, Writing – review & editing. CL: Data curation, Formal analysis, Writing – original draft. HY: Investigation, Writing – review & editing.