The outbreak of the disease called COVID-19 that appeared in 2019 and was caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has challenged the world's health systems by increasing demand for care of their patients. As of the third year of the pandemic, there have been over 400 million confirmed cases and almost 6 million deaths. Thus, this new disease has triggered the rapid development of a number of measures to detect, prevent, mitigate, and reduce its impacts. Among these measures, non-pharmaceutical interventions like those applied during the Spanish Flu were again applied [1]. However, the efficacy of these initiatives was limited since the disease was previously unknown. Alternatively, vaccines were considered as essential to fight against COVID-19, but their rapid evaluation processes due to the urgency of using them may have limited their popularity [2, 3]. In addition to these issues, vaccine availability and access were constraining factors. In practice, countries adopted diverse mixtures of protection measures to overcome risks of this disease like contagions, hospitalizations, and deaths. These mixtures were mainly proposed by governments and built on the available protection options [4–7]. However, all these conditions led countries to experience complex situations in a number of sectors other than health, as food security, local and global economy, education, tourism, and environmental air pollution [8, 9]. Such difficult situations also involved concerns experienced by individuals about sports and leisure, gender relations, domestic violence/abuse, and mental health [9].

Despite the highly complex and ever-evolving situations caused by COVID-19, it is of interest to identify good practices learned from successful managements of COVID-19. In fact, these lessons may suggest recommended practices to better protect populations from COVID-19 and other similar diseases. Different types of analyzes have been performed to identify and assess impacts of protection measures put in place. The case fatality rate (CFR), computed as the ratio of deaths among all patients confirmed with a disease causing an epidemic, is considered usually a benchmark for assessing and comparing the severity of this epidemic between countries. Variations in this rate during peaks and outbreaks would evidence response impacts against the disease [10]. However, it is hard to get the numbers for computing this rate [11, [12]]. Deaths are right-censored because of a time delay between the appearance of symptoms and death, whereas confirmed patients are not entirely determined, leading to biased numbers. Alternatively, some scholars have studied the effects of a single category of interventions, such as travel restrictions [13, 14], social distancing [15, 16], and personal protective measures [17]. More recently, [6] analyzed the effects of multiple interventions, but only the non-pharmaceutical ones implemented by governments were considered. In this paper, we propose a method to assess crucial effects for any set of interventions, including both pharmaceutical and non-pharmaceutical ones. Furthermore, temporality of interventions is also taken into account, as they may vary due to epidemic evolution. This is the case of variants of this disease that eventually appeared as those called Alpha, Delta, and Omicron.

To compare the management of COVID-19 between countries, we use the notion of reliability given by P(X ≤ Y), which is the probability of being in a success state without having entered a failure state, see e.g., Singh and Billinton [18]. The reliability is typically applied when relations stress-strength are analyzed, which are frequent in fields like medicine, quality control, and engineering [19]. However, this measure can also be used in other contexts. For example, [20] compared mortality rates from populations with unequal incomes in order to get lost life years of compensation since human beings can be seen as systems exposed to failures. To the best of our knowledge, this is the first application of reliability to analyze the management of COVID-19. Considering countries as systems, this measure allows assigning a probability to each country X when it is compared to a reference system. Then, adopting another country Y as a reference system, we obtain a set of probabilities R = P(X ≤ Y) for each couple of countries (X, Y). These probabilities thus allow the establishment of orderings among countries, as R numbers define a stochastic order [21, 22]. In this case, such orderings can be easily deduced because relations among countries can be organized into simple schemes.

Our approach to stochastically order countries is a new deep learning method, as it reveals previously unknown order relations among countries [23].

Under our proposed method, the management of COVID-19 in 67 countries during 2021 and the first months of 2022 is analyzed by considering COVID-19 deaths per million (DPM) and COVID-19 cases per million (CPM), which are commonly used rates for comparing countries [24–26]. This means that the effects of Delta and Omicron, two of the main variants of COVID-19 classified as variants of concern by the World Health Organization, are included in this study [27,28].

The rest of the paper is structured as follows. In the next section, the data used and their main features are presented. Section 3 presents the methodology to be applied. It concerns the description of the computation of reliability and the establishment of orderings. Section 4 shows results for each of the variables to be studied. The last section presents concluding remarks.

Daily data on mortality and cases due to COVID-19 from 67 countries were obtained from https://covid19.who.int/WHO-COVID-19-global-data.csv, with the exception of Ecuador, whose information was obtained from https://www.salud.gob.ec. The period of analysis was from January 1st, 2021 to February 19th, 2022, which is related to COVID-19 vaccine availability for most of the analyzed countries. These data were downloaded on February 22th, 2022. To avoid some unusual values resulting from data released with delay, daily data were transformed into epidemiological weeks. This kind of week is commonly referred to as an epi week or a CDC week, i.e., a seven-day period starting on Sunday. Furthermore, two-letter country codes defined in ISO 3166-1, called alpha-2 codes, were used instead of the full names of countries.

The total number of individuals from the examined countries was obtained from https://population.un.org. Because the period of analysis is short, such total numbers were assumed to be constant over the studied time interval.

Next, the DPM and CPM variables for a country i and a given epidemiological week j are built using the following expressions.

Let us see how to build country orderings by using the notion of reliability given by R = P(X ≤ Y), where X and Y are random variables representing failures of two systems [18]. Note that 0 ≤ R ≤ 1. If R < 0.5 (>0.5), the system represented by X (Y) would be more resistant to failures than the system represented by Y (X), whereas if R = 0.5, both systems behave equally. Since the relations between X and Y are unknown, we propose using the nonparametric R estimate given by the classical Wilcoxon-Mann-Whitney statistic, see [29]. Given samples x₁, …, x_n of X and y₁, …, y_m of Y, this nonparametric statistic is expressed by

where I(x, y) = 0 if x ≥ y and I(x, y) = 1 if x ≤ y. This is an unbiased and consistent estimate of R.

Let A and B be two countries and let V_A and V_B be their corresponding random variables when we focus on a variable V of interest. Then, we define R_{A, B, V} = P(V_A ≤ V_B) and its estimate is denoted by r_{A, B, V}. This means that R_{A, B, V} < 0.5 (>0.5) implies that V_A (V_B) would present more frequently higher values than the ones of V_B (V_A), whereas if R_{A, B, V} = 0.5, both V_A and V_B behave equally. In the case R_{A, B, V} ≥ 0.5, it is said that V_A stochastically precedes V_B Based on this inequality, we define a strict version as follows. V_A strictly stochastically precedes V_B if R_{A, B, V} > 0.5. Using this strict stochastic precedence order, we have that country B performs worse than country A with respect to the variable V if R_{A, B, V} > 0.5.

Let A_i, i = 1, …, s, be a set of countries. When considering the ordering introduced above, a matrix M_V of order s × s is built to represent the outputs R between any couple of countries. The element (i, j) of M_V is 1 if V_Ai stochastically precedes V_Aj; otherwise, this element is 0.

Now, we will deduce a causal diagram associated with M_V, i.e., a directed acyclic graph representing probabilistic causal domains [31] Further, directed cycles are included in this diagram, if needed. This directed graph will exclude redundant edges, i.e., edges that do not block all existing paths between the two most extreme vertexes of such a graph.

To this aim, denoting by the directed edge A_j → A_i if the element (i, j) of M_V is 1, we propose to apply Algorithm 1. Steps 1 and 2 of this algorithm arrange the matrix M_V in such a way that countries with a few 1s by row are placed on top and left of the matrix, whereas those with more 1s are placed on the bottom of the matrix. For each row of this matrix, more 1s means the country's management is relatively better. Step 3 renames countries according to their new positions in M_V, calling them B_i, i = 1, …, s. In this way, B_i has less or the same number of 1s than B_j if i < j. These three steps are key because they allow an easy deduction of a causal diagram in some cases. For instance, if under the diagonal of M_V there are only 1s, we have B₁ → B₂ → ⋯ → B_s because some of those 1s evidently represent all the relations B_i−1 → B_i for i = 2, …, s. This ideal ordering would be expected, but variations may arise. Precisely, starting with this ideal relation in Step 4, Steps 5 and 6 modify it if any of the following three situations occurs. First, when there are 0s in positions (i, i−1). Although the corresponding relation has been previously included as B_i−1 → B_i, it must be changed by B_i → B_i−1. Second, when there are 1s located over the diagonal of M_V, say the position (i, j) with i < j. This means that their corresponding 0s are located in the position (j, i), i.e., under that diagonal. Thus, the relation B_i → B_j holds and must be added to the causal diagram. Third, once the first situation has happened, the change from B_i−1 → B_i to B_i → B_i−1 breaks the causal diagram in Step 4. Hence, it is necessary to design the “right bridges” to reconnect that diagram. Assuming that the diagram is interrupted in the vertex i and M_V has value 1 in the position (i, j) for some j > i, then the relation B_i−1 → B_j is added to the diagram. Proposition 1 guarantees Algorithm 1 works.

Proposition 1. Let V be a variable to be analyzed. Let A_i, i = 1, …, s, be a set of countries and M_V their associated matrix as defined above. Let nAi,V=∑j=1sMV(i,j), i = 1, …, s. We have:

Assume that there is a country that manages better than the other analyzed countries, and another one that manages worse than the other analyzed countries. Rename A_i, i = 1, …, s, as B_i, i = 1, …, s, by considering an order defined in i). We have:

Proof.

Statement i) follows because n_{Ai, V} varies between 0 and s−1.

Assume that there is a country that manages better than the other analyzed countries, say without loss of generality B_s.

Let us prove ii). This assumption means that B_s strictly stochastically precedes B_i, i = 1, …, s−1. Therefore, R_{Bs, Bi, V} > 0.5, i = 1, …, s−1, and thus n_{Bs, V} = s−1. On the other hand, suppose that there is B_j, j ≠ s, which also manages better than the other analyzed countries. This fact implies that Bs′ strictly stochastically precedes B_s, i.e., R_{Bj, Bs, V} > 0.5. Because of R_{Bj, Bs, V} = 1−R_{Bs, Bj, V}, we have R_{Bs, Bj, V} < 0.5, which contradicts the fact that B_s strictly stochastically precedes B_j. Thus, it is concluded that B_s is unique.

The claims about B₁ follow similarly.

Let us prove iii). Consider the set T = {k:i < k ≤ s satisfying B_i−1 → B_k}. This set is not empty because of s ∈ T due to the assumption that there is a country that manages better than the other analyzed countries. The claim then follows.

In this paper, we proposed methods to order countries based on their management of COVID-19 and to cluster such countries by considering similar management behaviors. These methods consider the notion of reliability, P(X ≤ Y), by treating countries as systems exposed to failures. Our proposed methods are more general than typical methods that only assess the impacts of specific measures, as they consider not only interventions given by governments, but also any activities that people undertake to protect themselves.

The application of our proposed methods to 67 countries allowed us to identify the countries that performed better and worse in managing COVID-19 deaths and cases per million.

Notably, several countries demonstrated better management of COVID-19 for both COVID-19 deaths and cases. However, this consistency was almost lost when observing worse management of the disease. Moreover, most countries exhibiting better management of COVID-19 demonstrated complete orderings. These results are related to COVID-19 deaths and cases per million.

In practice, countries that exhibited better management of COVID-19 may recommend their protection strategies to others. We plan to deepen these results in a forthcoming paper by considering key protection measures as vaccination. Also, since the reliability measure R varies with the time interval considered, we plan to assess the management of COVID-19 in different time intervals, to understand how it evolves in each country.

These methods can also be applied to assess country's management of other diseases or sets of diseases.

Publicly available datasets were analyzed in this study. This data can be found here: https://covid19.who.int/WHO-COVID-19-global-data.csv, https://www.salud.gob.ec, and https://population.un.org.

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.