PubMed (see search strategy in Supplementary Material 1) and other electronic databases (Cochrane Library, EMA, AEMPS and Google to identify possible grey literature) were systematically searched (Supplementary Material 1) for efficacy meta-analyses published between January 2020 and December 2022. In addition, two additional searches were conducted: one for individual clinical studies published between January 2020 and October 2023 (to identify potential individual clinical studies not included in the published meta-analyses due to exclusion criteria or because they were published after the meta-analyses), and the other focused on drugs considered potential treatments at the start of the pandemic and were discarded early on. The searches were carried out without language limitations.

The additional systematic reviews followed the same inclusion criteria as those considered for the above but referred to individual clinical studies (randomized clinical trials or observational studies) with mortality outcomes, both for the eight drugs approved by the EMA for the treatment of COVID-19 (cilgavimab/tixagevimab, anakinra, nirmatrelvir/ritonavir, regdanvimab, tocilizumab, casirivimab/imdevimab, remdesivir and sotrovimab). Moreover, for the nine drugs that were discarded early because of unfavorable or insufficient clinical results (eculizumab, danoprevir, APN01, leronlimab, thymosin alfa1, REGN3084/REG3051).

Although only meta-analyses with an impact factor greater than 1 were included in the base case, an additional review of meta-analyses with an impact factor less than 1 in the SJR index was performed.

For the systematic meta-analysis review, the data extracted from the articles to be included were as follows: 1) Year of publication; 2) First author’s surname; 3) Type of meta-analysis (random-effects model, fixed-effects model, Bayesian model, network meta-analysis (NMA), mixed treatment comparison (MTC), indirect treatment comparison (ITC), and others); 4) Design of the clinical studies included in the meta-analysis (randomized clinical trials, observational studies [cohort, case-control, prospective or retrospective, etc.); 5) Comparator in the clinical studies (standard treatment, placebo control, etc.); 6) Main efficacy endpoints (mortality, disease progression, admission to ICU with mechanical ventilation, risk of secondary infection, hospital discharge, etc.); 7) Number of studies included in the meta-analysis; 8) Number of patients included in the meta-analysis; 9) Result of the measurement of the effect of the treatments compared to placebo or standard treatment (relative risk [RR], odds ratio [OR], hazard ratio [HR], risk difference, etc.) specifying - if available - the 95% confidence intervals (95% CI) and the statistical significance of the difference (p); 10) Degree of heterogeneity in the meta-analysis (I²), according to the following criteria: (i) up to 25%, low heterogeneity; between >25% and <75%, moderate heterogeneity; and (iii) if ≥ 75%, high heterogeneity (Higgins et al., 2003); and finally, 11) Effect size, calculated using Cohen’s d; by convention, Cohen’s d of 0–0.4, 0.5–0.7 and ≥0.8 are considered small, medium and large effect sizes, respectively (Chen et al., 2010). Cohen’s d will be calculated using the tool available at the following URL: https://www.escal.site/.

Data extraction was carried out by one investigator (DRR) and reviewed by another investigator (CRT).

As noted above, available clinical trials, observational studies and meta-analyses have yielded conflicting or inconclusive results, making it difficult to determine which drug treatments are truly effective for the treatment of COVID-19. Discrepancies in results between meta-analyses are difficult to resolve, as meta-analyses of meta-analyses are impossible. There is no consensus on the appropriateness of mixing observational studies and randomized clinical trials in the same meta-analysis (Bosdriesz et al., 2020; Kimachi et al., 2021; Ranstam and Wagner, 2022; Shrier et al., 2007; Toews et al., 2024) as is the case in part of the COVID-19 treatment syntheses. Meta-analysis is a statistical method used to combine the results of individual studies, obtaining a larger sample size, which provides greater reliability (precision) of treatment effect estimates (Higgins and Green, 2011). On the other hand, a larger sample size reduces the risk of type II error (Bausch and Cartwright, 2021; Rubio, 1996). Consequently, meta-analyses with larger sample sizes and the latest published meta-analyses (generally with a larger sample size due to the inclusion of the most recently published studies) were analyzed in preference. The risk of bias of individual clinical studies (in aspects such as randomization process, deviations from intended interventions, missing outcome data, measurement of the outcome, selection of the reported results) was analyzed in most of the meta-analyses, but was not analyzed in the systematic review because it is a synthesis of synthesis studies. This systematic review has not been registered.

The three bibliographic searches yielded 348 references. Of these, 148 were excluded because they did not meet the inclusion criteria. The full articles of the remaining 200 references were analyzed for eligibility. Of these, 96 were excluded. Accordingly, 104 articles were selected for inclusion in the synthesis. Please see Figure 1 for the study selection process according to PRISMA guidelines (all articles included in the base case of the synthesis can be found in Supplementary Material 2, in alphabetical order). The 96 references finally excluded were excluded for the following reasons: (i) 23 because they were letters to the editor or articles with insufficient or contradictory data (confirming or not confirming the impact of treatment on mortality); (ii) 4 because they only analyzed adverse effects of treatments; (iii) 3 because they were a protocol, analyzed a drug combination or virological sensitivity; and finally (iv) 66 because they had an impact factor of less than 1 in the SJR index (Supplementary Material 3). No meta-analysis was excluded for having a score less than 5 according to the NHLBI instrument. All meta-analyses analyzed were above this score.

The articles on the drugs discarded early for the treatment of COVID-19 are listed in Supplementary Material 4.

In Table 1, the number of meta-analyses selected in the systematic review is given for each drug or pharmacological group (literature references in Supplementary Material 2). In total, 85 meta-analyses were selected and analyzed (one meta-analysis was able to analyze several drugs). The characteristics of the meta-analyses included in the synthesis are comprehensively detailed in Supplementary Tables S1–S9.

In Figure 2, a timeline diagram of the drugs approved or discarded for COVID-19 treatment is presented, specifying the milestones that marked the EMA’s decisions in this regard. Table 2 (bibliographic references in Supplementary Material 2) contains a summary of the characteristics and results of the clinical trials that justify approval of the indication in COVID-19 by the EMA. The following effects were found in these studies (Table 2): (i) clinical improvement, shorter recovery time, and reduction in COVID-19 hospitalization or mortality with remdesivir (Beigel et al., 2020; Goldman et al., 2020; Gottlieb et al., 2022); (ii) clinical improvement and reduced mortality with tocilizumab (RECOVERY Collaborative Group et al., 2022; Rosas et al., 2021); (iii) reduction of mean viral load and mortality with casirimab/imdevimab (Abbas et al., 2022; CEDER. Center for Drug Evaluation and Research CDER Review, 2021); (iv) reduction of hospitalization, oxygen therapy and mortality with regdanvimab (Celltrion use of regdanvimab for the treatment of COVID-19. Assessment report, 2021); (v) reduction in hospitalization or death with sotrovimab (Gupta et al., 2021); (vi) reduction in hospitalization or death with nirmatrelvir/ritonavir (Hammond et al., 2022); (vii) reduction in the incidence of severe COVID-19 or death with cilgavimab/tixagevimab (Evusheld, 2019); and, finally, (viii) reduction in WHO-CPS progression of COVID-19 with anakinra (Kyriazopoulou et al., 2021).

The effect on mortality of COVID-19 drug treatments was highly variable in the different published meta-analyses, ranging from no association to a small, medium or large mortality effect (Supplementary Figure S1; Supplementary Material 6). As can be seen in Figure 3, the sample size effect of individual meta-analyses or clinical studies of licensed treatments for COVID-19 was associated with the demonstration of mortality reduction. Indeed, a reduction in mortality was observed in meta-analyses with a sample size of more than 5,000 patients (remdesivir, sotrovimab, cilgavimab/tixagevimab, casirivimab/imdevimab and tocilizumab), but not in meta-analyses with fewer than 5,000 patients (nirmatrelvir/ritonavir, regdanvimab and anakinra). It should be clear that the possible role of sample size in the results obtained is a mere hypothesis, simply the description of a result.

In Table 3, the reduction in COVID-19 mortality for the 17 treatments initially proposed is summarized, considering the outcome obtained in the most recent meta-analyses of randomized clinical trials (or, if not available, also including observational studies). Looking first at meta-analyses of randomized clinical trials, mortality was significantly reduced with tocilizumab [OR = 0.78 (95% CI 0.65–0.94)] (Albuquerque et al., 2023), casirivimab/imdevimab [OR = 0.67 (95% CI 0.50–0.91)] (Deng et al., 2023), sotrovimab [OR = 0.20 (95% CI 0.08–0.48)] (Deng et al., 2023) and remdesivir [RR = 0.83 (95% CI 0.71–0.98)] (Huang et al., 2023). In meta-analyses including observational studies, mortality reduction was observed with cilgavimab/tixagevimab [OR = 0.50 (95% CI 0.39–0.64)] (Wang et al., 2023), regdanvimab [OR = 0.14 (95% CI 0.03–0.56)] (Yang et al., 2022) and nirmatrelvir/ritonavir [OR = 0.24 (95% CI 0.15–0.39)] (Cheema et al., 2023). No reduction in mortality was observed with anakinra, sarilumab, antimalarials and the antivirals darunavir/cobicistat, favipiravir, interferons alpha and beta, lopinavir/ritonavir and umifenovir. At present, it should be noted that none of the monoclonal antibodies approved or registered for clinical use maintains proven clinical efficacy against the Omicron variant and its successive evolving sub-variants due to insufficient virus-neutralizing or blocking activity (Coutant et al., 2024; CovidCAREgroup, 2022). Consequently, it can be concluded that, according to the most recent meta-analyses, the treatments with the most evidence of mortality reduction in patients with COVID-19 would be remdesivir and tocilizumab. The magnitude of the effect with both drugs was small (Cohen’s d < 0.5) (Table 3).

An additional analysis of the worst and best outcome in terms of mortality reduction, obtained in meta-analyses and individual clinical studies with a larger sample size, was performed (Table 4). In this sense, “probable” or “inconclusive” efficacy was considered when the result was positive (mortality reduction) or negative (no mortality reduction), respectively, in the study with the largest sample size. Thus, mortality reduction was considered likely with tocilizumab [OR = 0.73 (95% CI 0.56–0.93)] (Rubio-Rivas et al., 2021), casirivimab/imdevimab [OR = 0.21 (95% CI 0.06–0.68)] (Gao et al., 2023), cilgavimab/tixagevimab [OR = 0.50 (95%CI 0.39–0.64)] (Wang et al., 2023), sotrovimab [OR = 0.40 (95% CI 0.25–0.63)] (Amani and Amani, 2022) and remdesivir [OR = 0.79 (95% CI 0.73–0.85) (Low-flow oxygen patients)] (Mozaffari et al., 2023). The effect size was variable, being small (Cohen’s d 0.5) with tocilizumab, cilgavimab/tixagevimab and remdesivir, medium (Cohen’s d between 0.5 and 0.7) with sotrovimab, and large (Cohen’s d ≥ 0.8) with casirivimab/imdevimab. Again, due to doubts about the efficacy of monoclonal antibodies (Coutant et al., 2024; CovidCAREgroup, 2022), the treatments with the most evidence of mortality reduction in patients with COVID-19 would be remdesivir and tocilizumab, according to meta-analyses and larger studies.

Considering the reduction in Covid-19-related hospitalization or death from any cause (generally up to day 28) as a combined efficacy endpoint, casirivimab/imdevimab [RR = 0.28–0.29; p < 0.0001] (Ronapreve, 2024 300 mg + 300), sotrovimab [RR = 0.21 (95% CI 0.09–0.50)] (Xevudy, 2021), nirmatrelvir/ritonavir [-5.62% (95% CI -7.21; −4.03%)] (Hammond et al., 2022) and remdesivir [RR = 0.13 (95% CI 0.03–0.59)] (Gottlieb et al., 2022) were effective. The magnitude of the effect was large (Cohen’s d ≥ 0.8) with sotrovimab and remdesivir.

All eight medicines authorized for the treatment of COVID-19 have at least one randomized, double-blind, placebo-controlled clinical trial (Figure 2). Several drugs initially proposed but not licensed for the treatment of COVID-19 (sarilumab, hydroxychloroquine/chloroquine, darunavir/cobicistat, favipiravir, IFN-alpha/IFN-beta, lopinavir/ritonavir and umifenovir) have been discussed previously (Section 3.5; Table 3). Insufficient demonstrative efficacy data may be the reason why these drugs have not been approved for the treatment of COVID-19 (Figure 2; Supplementary Material 4). The data available for other drugs not authorized for COVID-19 are reviewed in more detail below: eculizumab, danoprevir, APN01, leronlimab, camrelizumab and thymosin α1.

Eculizumab is a recombinant humanized IgG2/4κ monoclonal antibody that binds to human complement protein C5 and inhibits terminal complement activation (Bekemv 300 mg concentrado para solución para perfusión, 2021). It has only one feasibility study that concluded it could improve survival (randomized clinical trials would be necessary to confirm this) (Annane et al., 2020), two case series (Diurno et al., 2020; Burwick et al., 2022) and one observational (cohort, retrospective) study comparing 10 eculizumab-treated patients with 65 controls, a small sample size that precluded valid conclusions (Ruggenenti et al., 2021).

Danoprevir is an NS3/4A protease inhibitor used to treat HCV genotype (GT) 1b infections (Miao et al., 2020). The results of a cohort study evaluating the efficacy of treatment with danoprevir plus ritonavir in 11 patients with COVID-19 have been published (Chen et al., 2020). The study design precludes conclusions about the efficacy of danoprevir.

APN01 is the recombinant form of human angiotensin-converting enzyme 2 (rhACE2) that may prevent SARS-CoV-2 entry into the host cell and reduce lung injury (EUnetHTA, 2021). Three clinical studies of COVID-19 treatment with APN01 are registered in the clinicaltrials.gov database, but results are only available for one of them (NCT04335136) (COVID-19, 2023). This is a randomized, double-blind, placebo-controlled clinical trial. Eighty-eight patients were treated with APN01, and 90 received a placebo. Mortality at 28 days was 10.2% and 13.3%, respectively, with no statistically significant difference (OR = 0.63, 95% CI 0.23–1.70, p = 0.3588).

Leronlimab is a C-C chemokine receptor type 5 blocking monoclonal antibody originally developed to treat human immunodeficiency virus infection (Yang et al., 2021). Five clinical studies of COVID-19 treatment with leronlimab are registered in the clinicaltrials.gov database, but results are only available for one of them (NCT04343651) (COVID-19 (APN01-COVID-19), 2024). This is a randomized, quadruple-blind, placebo-controlled clinical trial. Fifty-six patients were treated with leronlimab, and 28 received a placebo. There was no mortality at day 14 in either group. There were also no differences in symptom score, time to clinical resolution, or length of hospitalization.

Camrelizumab is a monoclonal antibody proposed as a potential treatment for COVID-19, but no clinical studies have been identified that have evaluated its efficacy in this indication.

Thymosin α1 has been used in the treatment of viral infections as an immune response modifier for many years (Liu et al., 2020). In a retrospective study, compared to the untreated group (N = 40), treatment with thymosin α1 (N = 36) significantly reduced mortality in severe COVID-19 patients (11.11% vs. 30.00%, p = 0.044) (Liu et al., 2020).

A total of 151 meta-analyses have been identified. This high number is due to the health and social relevance of the COVID-19 epidemic and, therefore, to the tendency of the many medical journals to publish meta-analyses that could clarify which treatments would be the most effective in this context. For this reason, the systematic review was divided into two analyses according to the supposed quality of the studies according to the impact factor of the medical journals. The full references of the initially excluded studies are attached in Supplementary Material 3. As can be seen in Supplementary Material 7, most meta-analyses of immunomodulatory drugs (anakinra, tocilizumab, sarilumab) did not show a statistically significant effect or had a small magnitude with respect to mortality. Regarding antimalarials (hydroxychloroquine, chloroquine), there was no effect on mortality in 16 of 18 meta-analyses. Finally, regarding antivirals, a small effect on mortality was found in one meta-analysis of favipiravir, in two meta-analyses of interferon and in two meta-analyses of remdesivir.