The inclusion of intervention in the study, QLQX, is based on the Chinese New Drug Certificate (National medicine permission number: Z20040141), and the patent “A pharmaceutical composition and preparation method for the treatment of chronic heart failure” (patent number: ZL02146573.8). The quality control, prescription composition and chemical composition involved in the included studies are shown in Supplementary Table 1.

This study was pre-registered (PROSPERO, No. CRD42021248464) and conducted following the PRISMA statement (14). The checklist was shown in Supplementary Table 2. The literature search was conducted in the databases of PubMed, Cochrane, Embase, Chinese National Knowledge Infrastructure (CNKI), China Science and Technology Journal (VIP), and Wanfang. The retrieval time was from the database establishment to 20 July 2021. MeSH terms combined with free search words were used for the literature search. The search strategy of the PubMed database was shown in Supplementary Table 3.

Inclusion criteria met the following requirements: (1) study type: preclinical experiments (PEs) or randomized controlled trials (RCTs); (2) cellular or animal models preclinically compatible with CHF, and patients clinically diagnosed with CHF; (3) QLQX alone or combined with RT as the intervention, and in preclinical experiments, the mechanism of QLQX was explored; (4) control group received RT or placebo; (5) there was no restriction on the outcome indicators of PEs, and the outcome indicators of RCTs were included as follows: clinical efficacy rate, cardiac function, 6-min walk distance (6-MWD), brain natriuretic peptide (BNP), N-terminal pro-brain natriuretic peptide (NT-proBNP), and adverse events. The exclusion criteria were as follows: (1) studies that included non-CHF patients; (2) repetitive studies, review, protocol, comment, case report, etc.; (3) PE or RCT included other Chinese medicine or related interventions in addition to QLQX; (4) data of studies was not available even though contacted with original authors.

According to the Systematic Review Centre for Laboratory animal Experimentation (SYRCLE) (15), a risk of bias tool, PEs were assessed according to the following items: (1) sequence generation; (2) baseline characteristics; (3) allocation concealment; (4) random housing; (5) blinding of performance; (6) random outcome assessment; (7) blinding of detection; (8) incomplete outcome data; (9) selective outcome reporting; (10) other sources of bias. According to the Cochrane criteria, we assessed the quality of the included RCTs using the Risk of Bias 2.0 (RoB 2.0) (16) in six aspects: (1) bias arising from the randomization process; (2) bias due to deviations from intended intervention; (3) bias due to missing outcome data; (4) bias in measurement of the outcome; (5) bias in selection of the reported result; (6) overall. Under the supervision and coordination of a researcher (QH), two researchers (BS and YW) assessed the risk of the included PEs, and another two researchers (WY and YJ) assessed the risk of the included RCTs according to “low risk,” “high risk,” or “some concerns,” respectively.

The following data was extracted: (1) basic information of the included PEs (the first author, publication year, animal species, sex, number of animals, weight, intervention, experiment duration) and RCTs (the first author, publication year, sample size, age information of the patients, intervention, trial duration); (2) all outcome measures of experiments; (3) outcome measures including clinical efficacy rate, cardiac function, 6-MWD, BNP, NT-proBNP, and adverse events. Under the supervision of a researcher (HC), two researchers (HSL and JW) extracted the data of the PEs, and the other two researchers (CL and HZ) extracted the data of the RCTs.

The Stata 17.0 software (Stata Corp., College Station, TX, USA) was applied to statistical analysis: (1) random-effect model was employed, as different RT medicine or dosage may affect the therapeutic effect; (2) Cohen’s d and 95% CI were used for continuous variables; (3) relative risk (Relative Risk) and 95% CI were used for categorical variables; (4) A p-value < 0.05 indicated statistically significant; (5) Q statistics and I² were used to evaluate the heterogeneity of each pooled analysis, and the p-value in Q statistics < 0.05 or I² > 50% meant high heterogeneity; (6) the sensitivity analysis was carried out when any meta-analysis presented high heterogeneity; (7) as for the difference of interventions or classification of outcome measures, subgroup analyses were inclined to carry on.

This study used GRADE (Grades of Recommendation, Assessment, Development and Evaluation) (17), a transparent and structured quality rating system, to grade each of the outcome indicators in meta-analysis with four levels of evidence: high, moderate, low, and very low. The accessment followed the GRADE handbook. The reasons for downgrading levels of evidence certainty in the included studies were shown below: high risk of bias in results (18), high level of heterogeneity (19), low generalizability of the study (20), no statistical significance of effect size (21), and publication bias in the study (22).

A total of 2,235 studies were retrieved through the initial search, and 1,482 studies were obtained after removing duplicate studies. By screening the titles or abstracts, 177 studies were obtained, followed by the full-text accessment, and 7 PEs and 24 RCTs were included. The flow chart of the study screening is shown in Figure 1.

Seven PEs (23–29) and 24 RCTs (30–53) included 294 rats and 2,731 patients respectively. PE studies range from 4 to 8 weeks including 21–60 animals in each study. The clinical trial duration ranges from 2 to 24 weeks with patients from 64 to 491 each. In the PEs, all experiments were pharmacodynamic studies investing the biochemical effects of QLQX in animal CHF models. In the included RCTs, 1 trial (33) used the placebo in the control group. The characteristics of included studies are shown in Tables 1, 2.

Seven PEs (23–29) were assessed by the SYRCLE risk of bias tool (Supplementary Table 4). All PEs were assessed as the “some concerns” in the item of “random housing” in all PEs due to lack of reporting. Three PEs (27–29) existed a “some concerns” risk in the “sequence generation” section due to lack of group method reporting, and other two PEs (25, 26) without baseline information were assessed as the “some concerns.” A total of 24 RCTs (30–53) were assessed by RoB 2.0 risk of bias tool (Supplementary Table 5). All RCTs were assessed as the “low” risks in the item of “bias due to deviations from intended intervention” and “bias arising from the randomization process.” From the item of “bias in measurement of the outcome” or “bias in selection of the reported result,” three RCTs (38, 42, 49) were assessed as “high” risks because of incomplete data and results.

Fourteen RCTs (32, 34–37, 40, 41, 44–47, 49, 51, 53) reported the clinical efficacy rate. As shown in Figure 3A, QLQX plus RT significantly increased clinical efficacy rate [RR = 1.16, 95% CI (1.12, 1.22), p < 0.05, GRADE: moderate, as shown in Table 3], with no heterogeneity [Q (13) = 5.43, P = 0.00, I² = 0]. Egger test results showed that there was no published bias (β₁ = 2.02, SE of β₁ = 1.636, z = 1.23, P = 0.2180 > 0.05). The L’Abbe and funnel plots are respectively presented in Figures 3B, C.

Fourteen studies reporting the results of LVEF, 7 studies reporting LVEDD and three studies (43, 44, 53) reporting LVESD were pooled and analyzed. As shown in Figure 4, the intervention group significantly improved the indicators of LVEF [SMD = 1.20, 95% CI (0.97, 1.43), p < 0.05; Q (13) = 46.66, p = 0.00, I² = 71.31%, GRADE: moderate], and LVEDD [SMD = −1.04, 95% CI (−1.39, −0.70), p < 0.05; Q (10) = 60.77, p = 0.00, I² = 84.40%, GRADE: low] compared to the control group. However, there was no significant difference between QLQX plus RT and RT on decreasing LVESD in patients with CHF [SMD = −0.82, 95% CI (−1.78, 0.14), p > 0.05; Q (2) = 26.92, p = 0.00, I² = 92.44%, GRADE: low]. Heterogeneity analysis indicated that differences in drug dosage and sample size might cause high heterogeneity.

Six studies (35, 37, 39, 45, 46, 53) reported the 6-MWD. The forest plot in Figure 5 showed that intervention group improved the 6-MWD significantly [SMD = 1.55, 95% CI (0.89, 2.21), p < 0.05, GRADE: low]; but there was a higher heterogeneity [Q (5) = 59.55, p = 0.00, I² = 91.67%]. The heterogeneity analysis indicated that different durations led to high heterogeneity.

Eight studies (31, 38, 39, 43–45, 47, 49) reported serum BNP levels. However, sensitivity analysis found that two studies (44, 49) caused a higher heterogeneity [Q (7) = 152.55, p = 0.01, I² = 98.16%] and were excluded from the pooled analysis. The forest plot in Figure 6A reported that QLQX plus RT significantly reduced the level of serum BNP in patients with CHF [SMD = −0.78, 95% CI (−1.06, −0.51), p < 0.05, Q (5) = 12.29, p = 0.00, I² = 59.27%, GRADE: low]. Six studies (34, 37, 42, 46, 51, 53) QLQX significantly decreased serum NT- proBNP levels [SMD = −2.15, 95% CI (−3.60, −0.71), p < 0.05, Q (5) = 168.27, p = 0.00, I² = 98.06%, GRADE: low] as shown in Figure 6B.

Seven studies (41, 44, 45, 47, 49, 51, 53) reported adverse events. The forest plot Figure 7 indicated that compared with the RT, QLQX treatment showed a decrease in adverse events [RR = 0.46, 95% CI (0.25, 0.87), p < 0.05, Q (6) = 12.01, p = 0.02, I² = 51.05%, GRADE: low].