An empirical comparison of the harmful effects for randomized controlled trials and non-randomized studies of interventions - Dai, Minhan; Furuya-Kanamori, Luis; Syed, Asma; Lin, Lifeng; Wang, Qiang (Frontiers in Pharmacology, 2023)
Randomized controlled trials (RCTs) are the gold standard to evaluate the efficacy of interventions (e.g., drugs, vaccines), yet the sample size of RCTs is often limited for safety assessment. Non-randomized studies of interventions (NRSIs) had been proposed as an important alternative source for safety assessment. In this study, we aimed to investigate whether there is any difference between RCTs and NRSIs in the evaluation of adverse events. We used the dataset of systematic reviews with at least one meta-analysis includingboth RCTs and NRSIs and collected the 2 by 2 table information (i.e., numbers of cases and sample sizes in intervention and control groups) of each study in the meta-analysis. We matched RCTs and NRSIs by their sample sizes (ratio: 0.85/1 to 1/0.85) within a meta-analysis. We estimated the ratio of the odds ratios (ROR) of an NRSI against an RCT in each pair and used the inverse variance as the weight to combine the natural logarithm of ROR (lnROR). We included systematic reviews with 178 meta-analyses, from which we confirmed 119 pairs of RCTs and NRSIs. The pooled ROR of NRSIs compared to RCTs was estimated to be 0.96 (95% confidence interval: 0.87, 1.07). Similar results were obtained with different sample size subgroups and treatment subgroups. With the increase of sample size, the difference in ROR between RCTs and NRSIs decreased, although not significantly. In conclusion, there was no substantial difference in the effects between RCTs and NRSIs in safety assessment when they have similar sample sizes. Evidence from NRSIs might be considered as a supplement to RCTs for safety assessment.