Seven databases, such as PubMed, embase, Cochrane Library, China National Knowledge Infrastructure database (CNKI), VIP database for Chinese Technical Periodicals (VIP), Wanfang Data and China Biomedical Medicine database (CBM), were searched from inception to November 11, 2020. Search terms were used as follow: “refractory mycoplasma pneumoniae pneumonia”, “RMPP”, “traditional Chinese medicine”, “children”, “adolescents”, “randomized controlled trial” etc. Details of all databases of search strategies were shown in Supplementary Table S1. Reference lists and conference proceedings were also checked manually for additional relevant data. We contacted relevant corresponding authors for missing or unreported information when needed. No restrictions were applied on language or publication date.

Studies meeting the following conditions were included for further analysis: 1) Study design: prospective, randomized controlled trials (RCTs); 2) Participants: all the enrolled children and adolescents within 18 years old were required to meet any proper diagnostic criteria of RMPP, such as “Expert consensus on the diagnosis and treatment of Mycoplasma pneumoniae pneumonia in children” (Chen et al., 2015); 3) Interventions and Comparisons: patients receiving CHM (no restriction on dosage, formula, and form) combined with WM in intervention groups, while WM alone as the control; 4) Outcomes: at least one primary outcome of interest with reliable and available data.

Cases meeting the following conditions were excluded: 1) case-control or cohort trials, quasi-randomized studies; 2) cell or animal experiments, case reports, meta-analyses, reviews; 3) studies with poor design, unavailable data, or unreported target outcomes; 4) children and adolescents who received other joint interventions such as acupuncture, cupping, moxibustion, massage, acupoint application.

The primary outcomes were clinical efficacy rate, antipyretic time, and cough disappearance time. The secondary outcomes included the disappearance time of lung rale and lung X-ray infiltrates, inflammation index and adverse effects.

From each study, the collected data are: 1) study characteristics, including publication year, location, as well as the name of the first author; 2) participants information, such as age, gender, sample size, course of disease, and diagnosis standards; 3) details of interventions, including treatment protocols, TCM therapies, and duration; 4) outcome measures.

The Cochrane risk of bias tool (Higgins et al., 2011) was adopted to the evaluation of the selected studies’ methodological quality based on considerations below: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting, and other bias. The bias risk assessment was independently conducted by two investigators with three potential responses: low, high and unclear. A third author was also invited to resolve any conflict.

Review Manager 5.3 was applied for statistical analysis. The risk ratio (RR) with 95% confidence intervals (CI) was used to express the results of dichotomous data. Mean difference (MD) or standardized mean difference (SMD) with a 95% CI was adopted for continuous data. Heterogeneous results could not be avoided due to the use of different herbal medicines in all selected studies. For this reason, summary effect measures were calculated by the random-effects method. Statistical heterogeneity was quantified through the I-squared (I ² ) statistic, whereby an I ² > 50% was deemed substantial heterogeneity (Higgins et al., 2003). Subgroups analyses were performed and categorized according to types of WM and different TCM therapies described in each study, respectively, which could not only reduce heterogeneity to some extent but also compare the clinical efficacy of different TCM therapies in RMPP. The differences at a statistically significant level were defined by p < 0.05. Publication bias was assessed through a funnel plot and calculated by Egger’s tests, when at least ten studies were available. Sensitivity analyses were conducted by changing the random-effects method to a fixed-effect model and deleting each study in sequence to evaluate the stability of the results and explore the sources of heterogeneity.

Some positive findings from meta-analysis might be caused by a random error (by chance) rather than the true effects of the intervention and these results might often increase the likelihood of overestimation (Type I errors) or underestimation (Type II errors) (Pereira and Ioannidis, 2011; Afshari et al., 2017), which could be resolved by Trial sequential analysis (TSA) (Kang, 2021). Therefore, we performed TSA by using TSA 0.9.5.10 Beta software to reduce the false-positive results caused by random errors and determine whether more RCTs were needed to obtain robust conclusions. The error rate was set as 5% for type I and 10% for type II.

A total of 425 articles were retrieved with the predefined search strategy. 299 studies remained after we removed duplicates. By screening titles and abstracts, 244 articles were excluded, and 55 articles were yielded to conduct a full-text evaluation. Among them, 38 articles were excluded, which included non-RCTs (n = 13); trials with inappropriate interventions (n = 13); trials in which outcomes did not meet the inclusion criteria (n = 8); studies lack of adequate drug information (n = 2); articles without access to full text (n = 2). Finally, 17 RCTs (Qian, 2011; Chen, 2014; Meng, 2015; Tian, 2016; Wang and Wu, 2016; Yuan et al., 2016; Lian, 2017; Sun et al., 2017; Tan and Yang, 2017; Zhong et al., 2017; Wang et al., 2018; Wang, 2018; Yang, 2019; He et al., 2020a; He et al., 2020b; Zhang, 2020a; Zhang, 2020b) were eligible for meta-analysis (Figure 1). A list of excluded studies by reading full-text was provided in Supplementary File S2.

All studies were single-center and parallel-design RCTs conducted in China between 2009 and 2019. A total of 1,451 participants were recruited, with 726 in intervention groups and 725 in control groups. WM in our analysis could be divided into two categories: nine studies (Chen, 2014; Meng, 2015; Tian, 2016; Yuan et al., 2016; Sun et al., 2017; Zhong et al., 2017; Wang et al., 2018; He et al., 2020a; Zhang, 2020b) using macrolide antibiotics, and the other eight (Qian, 2011; Wang and Wu, 2016; Lian, 2017; Tan and Yang, 2017; Wang, 2018; Yang, 2019; He et al., 2020b; Zhang, 2020a) using macrolide antibiotics joint with corticosteroids. Ten studies (Qian, 2011; Chen, 2014; Tian, 2016; Yuan et al., 2016; Lian, 2017; Wang, 2018; Yang, 2019; He et al., 2020a; He et al., 2020b; Zhang, 2020a) administrated CHM to participants in decoction form, three (Wang and Wu, 2016; Sun et al., 2017; Wang et al., 2018) in granule, three (Meng, 2015; Tan and Yang, 2017; Zhang, 2020b) in injection and one (Zhong et al., 2017) in oral liquid. TCM therapies were described in all studies, nine (Qian, 2011; Yuan et al., 2016; Lian, 2017; Sun et al., 2017; Tan and Yang, 2017; Zhong et al., 2017; He et al., 2020a; He et al., 2020b; Zhang, 2020b) with removing heat-phlegm and toxicity therapy, four (Chen, 2014; Tian, 2016; Wang, 2018; Zhang, 2020a) with promoting blood circulation therapy and the rest (Wang and Wu, 2016; Sun et al., 2017; Wang et al., 2018; Yang, 2019) with invigorating Qi and consolidating superficies therapy. The treatment duration ranged from one week to four weeks. Table 1 listed the characteristics of selected studies. Supplementary Tables S3, S4 provided detailed information on CHM.

The results of methodological bias were presented in Figure 2. All included studies mentioned the randomized allocation. Ten clearly described the randomization method, including random number table (Yuan et al., 2016; Lian, 2017; Sun et al., 2017; Tan and Yang, 2017; Zhong et al., 2017; Wang et al., 2018; Yang, 2019; He et al., 2020a) or computer-generated numbers (Tian, 2016) or ballot (He et al., 2020b), which were considered as the low risk, while seven (Qian, 2011; Chen, 2014; Meng, 2015; Wang and Wu, 2016; Wang, 2018; Zhang, 2020a; Zhang, 2020b) did not provide the detailed randomization methodology. Allocation concealment was unclear and none of them were double-blinded. It was easy for participants to identify which group they were in, since the packages of CHM and WM were different. All articles had complete data and consistent outcomes as described in method section. And other biases were unclear.

Ten studies (740 patients) (Qian, 2011; Meng, 2015; Tian, 2016; Wang and Wu, 2016; Yuan et al., 2016; Lian, 2017; Zhong et al., 2017; Wang, 2018; He et al., 2020a; Zhang, 2020a) mentioned antipyretic time. Overall analyses revealed that CHM combinated with WM showed more advantages in shortening antipyretic time than WM alone [MD = −2.60, 95% CI (−3.06, −2.13), p < 0.00001, I ² = 81%] (Figure 3B). In subgroup analyses (Table 2), we found that this outcome might be associated with TCM therapy, as removing heat-phlegm and toxicity therapy brought more benefits compared with promoting blood circulation therapy and invigorating Qi and consolidating superficies therapy.

Twelve studies (1,008 patients) (Qian, 2011; Meng, 2015; Tian, 2016; Yuan et al., 2016; Lian, 2017; Sun et al., 2017; Zhong et al., 2017; Wang, 2018; Wang et al., 2018; He et al., 2020a; Zhang, 2020a; Zhang, 2020b) were identified. A significant difference was found between groups, since the cough disappearance time in children and adolescents treated with CHM and WM was shorter than those treated with WM alone [MD = −2.77, 95% CI (−3.12, −2.42), p < 0.00001, I ² = 49%] (Figure 3C). Results of subgroup analyses (Table 2) yielded that removing heat-phlegm and toxicity therapy might work faster than the two others.

Ten studies (858 patients) (Qian, 2011; Yuan et al., 2016; Lian, 2017; Sun et al., 2017; Zhong et al., 2017; Wang, 2018; Wang et al., 2018; He et al., 2020a; Zhang, 2020a; Zhang, 2020b) evaluated this outcome measure. Meta-analysis showed that intervention with CHM and WM resulted in a greater decrease than WM alone [MD = −2.65, 95% CI (−3.15, −2.15), p < 0.00001, I ² = 72%] (Figure 4A). Categorized by TCM therapy, the subgroup analyses (Table 2) indicated that promoting blood circulation was the most advantageous in shortening lung rale disappearance time.

The lung X-ray infiltrates disappearance time was assessed in four studies (392 patients) (Wang and Wu, 2016; Sun et al., 2017; Wang et al., 2018; Zhang, 2020b). Statistically significant differences between the groups were noted, which demonstrated that CHM combined with WM was superior [MD = −2.75, 95% CI (−3.33, −2.17), p < 0.00001, I ² = 0%] (Figure 4B). In subgroup analysis (Table 2), TCM therapy of removing heat-phlegm and toxicity was more beneficial than invigorating Qi and consolidating superficies.

Three studies (342 patients) (Zhong et al., 2017; Wang et al., 2018; Zhang, 2020b) mentioned the changes in tumor necrosis factor-α (TNF-α). The analysis result exhibited significant difference between two groups [MD = −5.49, 95% CI (−7.21, −3.77), p < 0.00001, I ² = 58%] (Figure 4C), making it clear that CHM combined with WM was remarkably better than WM alone in reducing inflammation. In addition, the level of TNF-α seemed to be lower in the TCM therapy of removing heat-phlegm and toxicity than invigorating Qi and consolidating superficies (Table 2).

One study (Wang et al., 2018) reported no adverse reactions during treatment. One (Chen, 2014) reported minor adverse reactions in two groups, but the number of cases was not shown. Seven studies (Lian, 2017; Sun et al., 2017; Tan and Yang, 2017; Wang, 2018; Yang, 2019; He et al., 2020a; He et al., 2020b) reported the number of adverse events and details were shown in Supplementary Table S2. All reactions were at a mild level, posing no influence on the therapy. No difference was represented between two groups [RR = 0.97, 95% CI (0.60, 1.57), p = 0.91; I ² = 0%] (Figure 4D). The adverse reactions such as nausea, vomit, diarrhea, stomach ache, and rash were found in both groups, indicating those symptoms might be labeled in the direction of macrolide antibiotics and corticosteroids.

Meta-regression analysis was attempted to explain the heterogeneity in antipyretic time and lung rale disappearance time. Neither type of WM (regression coefficient β = −0.677, SE = 0.487, p = 0.207) nor TCM therapy (regression coefficient β = 0.498, SE = 0.457, p = 0.312) had an association with antipyretic time. In terms of lung rale disappearance time, the type of WM (regression coefficient β = −1.027, SE = 0.341, p = 0.02) was more relevant than TCM therapy (regression coefficient β = 0.528, SE = 0.225, p = 0.051), indicating type of WM might be the source of heterogeneity.

Funnel plots and Egger’s test were performed for the assessment of publication bias. The funnel plot (Figure 5A) revealed a slight asymmetry and Egger’s test (t = 3.05, p = 0.01) indicated possible publication bias in clinical efficacy rate. To further confirm whether the bias had an impact on clinical efficacy rate, trim and fill method was performed. One study was added and the corrected RR was 1.493 [95% CI (1.349, 1.642)], remaining significant. Visual assessment of funnel plots (Figures 5B–D) and Egger’s test showed no publication bias for antipyretic time (t = −1.02, p = 0.338), cough disappearance time (t = 0.71, p = 0.495) and lung rale disappearance time (t = 0.30, p = 0.771).

All results showed good consistency whether in the fixed-effects or random-effects model. The overall results did not alter after deleting one study each time. For further verification, we implemented sensitivity analyses of clinical efficacy rate, antipyretic time, cough disappearance time and lung rale disappearance time by STATA 16.0. Supplementary Figure S1 indicated that the outcomes were not reversed by removing any study, which had relatively good stability.

When we deleted one study each time and reanalyzed the rest, we found the heterogeneity still existed except for TNF-α results of Zhong et al. (2017). The heterogeneity dropped from 58 to 0% after this study was excluded. Therefore, this RCT may be a source of considerable heterogeneity, in which azithromycin with intravenous injection was applied while trials of Wang et al. (2018) and Zhang (2020b) exerted azithromycin orally.

The required information size (RIS), defined as the number of events or patients from the included studies necessary to accept or reject the statistical hypothesis (Wetterslev et al., 2009), is important to increase the quality of meta-analysis. Thus, We applied TSA boundaries to calculate the RIS and assess the robustness of our results. Relative risk reduction (RRR) was derived from the meta-analysis. In Figure 6, the Z-score curve (blue line) is found to cross the RIS boundary (vertical red line), TSA boundary (red polylines) and conventional boundary (dotted black lines), which indicated that the conclusion on clinical efficacy rate was robust with the existing evidence.

This review is subject to some limitations. First, all included studies were performed in China and failed to describe the blinding method or allocation concealment. The poor quality of methodology might contribute to an exaggerated curative effect and decreased reliability of the evidence, thereby lowering the credibility and generalization of the results. Second, given the small sample size and limited number of studies in certain outcomes, the results might be insufficient to ensure a significant difference. Third, although sensitivity analysis and TSA demonstrated the reliability of our results, heterogeneity was observed in some results, such as antipyretic time and lung rale disappearance time. This might correlate with CHM intervention approaches, such as CHM composition, dose, and different forms administration. Thus, these results with substantial heterogeneity should be interpreted with caution. Last, drug safety is a key factor in clinical applications, but only nine RCTs described adverse events. Therefore, the safety of CHM in RMPP should be validated in future.

The validity of the conclusion in meta-analysis was highly dependent on the quality of the RCTs included, thus more high-quality studies evaluating the efficacy and safety of CHM for RMPP are needed. Based on the above limitations, some recommendations are suggested for further studies: 1) Clinical trials should be registered in advance on clinical trials registry platform. 2) Methodological quality of future trials including randomization, allocation concealment, and blinding should be improved by using random number tables and opaque envelopes,etc. 3) The safety assessments of CHM need more attention. 4) Clinical trials of CHM should standardize syndrome differentiation and treatment, in order to investigate the clinical application of CHM better.