Although deaths in children have declined substantially in the past 30 years, more than 5 million still die every year (1). Electronic fetal heart rate (FHR) monitoring was introduced to detect fetuses' pathological states as early as possible in the obstetrics practice in the late 1950s. However, the misinterpretation and ambiguity of FHR patterns may increase unnecessary interventions, such as operative deliveries and cesarean sections (2–4). Different guidelines over the past decades have recommended some modifications for interpreting FHR tracings, but beliefs in the etiology of basic FHR patterns (including the baseline, the variability, accelerations, decelerations, and sinusoidal patterns) have remained essentially unchanged (5, 6). In these FHR patterns, the baseline is a pre-requisite for evaluating the other patterns (7). Gynecologists and obstetricians usually estimated the baseline by visual analysis, but the unreliability of visual interpretation with a high degree of inter-and intra-observer variability is found (7–12). Therefore, computer-assisted analysis has been sought to mitigate the variability of visual explanation (13–17).

Several studies have evaluated the performance of different computer-assisted methods. For example, in 2016, Jezewski et al. evaluated 11 different algorithms using two inconsistency coefficients based on three properties (i.e., number, location and area) of accelerations/accelerations (18). They found that the algorithm of Arduini et al. (9, 19) outperforms other methods by achieving the lowest mean inconsistency coefficients on a private dataset with 41 FHR signals. This nonlinear filtering method is similar to the algorithm proposed by Mantel et al. (20). The difference is that Arduini's baseline is computed in 10 min windows with 5 min shift, whereas Mantel's baseline is calculated for the whole FHR tracing. Considering Mantel's method, Houzé de l'Aulnoit et al. further evaluated 11 newer algorithms by comparing the computed baselines with that estimated by clinicians on a dataset with 90 FHR signals (13). This study found that Lu and Wei's algorithm (14) achieves better results than other methods by achieving a new morphological analysis discriminant index (MADI) of 7.3%. Recently, a weighted median filter was proposed by Boudet et al. to compute the FHR baseline, and more agreement (represented by a MADI of 4.0%) with clinicians' consensus than Lu and Wei's method was shown on this dataset with 90 FHR recordings (15). Similar to Lu and Wei's method, an algorithm for the baseline estimation based on singular spectrum analysis and empirical mode decomposition was also proposed by Lu et al. (16) and evaluated on another public dataset with 552 FHR recordings. This method also was objectively evaluated on the dataset with 90 FHR recordings (13) by achieving a MADI of 15.6%. Unlike signal processing methods, the CTGNet based on deep learning was proposed in our previous study (21) and evaluated on a larger dataset with 234 FHR recordings. This method was compared with 12 signal processing methods and the lowest metrics (including the root-mean-squared difference between baselines and MADI) were obtained. These methodological studies illustrate the excellent performance of the CTGNet. However, its clinical application still requires a comparative study with large-scale multicenter data.

To evaluate the clinical usability of the CTGNet, we compare the FHR baseline predicted by the CTGNet with that estimated by clinicians using a large dataset with 1,267 FHR recordings acquired with fetal monitors of five device manufacturers.

Table 1 summarizes the comparisons of the baselines predicted by the CTGNet and those estimated by clinicians. In 99% of FHR recordings from GMU_DB, JNU_DB and SMU_DB, differences do not exceed 5 bpm, whereas the percentages of differences less than 5 bpm dropped to 58%–85% in FHR recordings from LCU_DB and UHB_DB. For the baseline difference ≥5 bpm, proportions are <0.5% for datasets from China (i.e., GMU_DB, JNU_DB and SMU_DB), 13.6% for LCU_DB, and 42.2% for UHB_DB, respectively. In general, the ratio of differences <5 bpm and ≥5 bpm is 4:1 among the 1,267 FHR recordings.

According to Kappa values (i.e., excellent agreement: Kappa >0.75, good agreement: 0.4 ≤Kappa ≤0.75 and poor agreement: Kappa <0.4), agreement in the baseline estimation is excellent for different datasets. However, Kappa values for datasets from China (i.e., GMU_DB, JNU_DB and SMU_DB) are >0.98, whereas those for LCU_DB and UHB_DB are 0.885 and 0.771, respectively. For 1,267 FHR recordings, the Kappa value and the correlation coefficient between manual measurement and CTGNet calculation are 0.873 and 0.969, respectively.

Figures 2A–E are Bland-Altman plots demonstrating the interchangeability of the clinical measurement and the CTGNet for the baseline estimation. The mean differences are <1. Limits of agreement for GMU_DB, JNU_DB, SMU_DB, LUC_DB and UHB_DB, respectively, are −0.82/0.93, −1.34/0.93, −1.63/3.39, −13.10/14.52 and −9.05/9.90. Maximum differences for GMU_DB, JNU_DB, SMU_DB, LUC_DB and UHB_DB, are 2.81 bpm, 3.43 bpm, 16.52 bpm, 48.88 bpm and 18.86 bpm, respectively.

These most divergent examples in each database are shown in Figures 3A–F. It can be observed that the difference is large when the signal loss rate increases. Anyway, on the whole dataset including 1,267 FHR recordings, the mean difference, deviation and limits of agreement are 0.36, 3.64 and −6.81/7.48, respectively.

Reliable FHR interpretation is the base of the fetal state assessment. The poor recognition performance of FHR patterns can propagate the error to subsequent steps, thereby decreasing classification accuracy. In all these FHR patterns, the baseline is a precondition for evaluating of the other patterns. Visual estimation of the FHR baseline is subject to inter-and intra-observer variability. Computer-assisted baseline estimation has been proposed as a promising way to reduce this variability. In order to evaluate the performance of our computer-assisted method (i.e., CTGNet), a comparison of FHR baseline estimation by the CTGNet and a consensus of clinicians presents in this study.

Baselines assigned by computer-assisted methods had been compared with those estimated by clinicians in several studies (7–12). In the studies of Arduini et al. and Ayres-de-Campo et al., limits of agreement (LoA) were no more than −6.45 and 7.07 in ≤150 FHR tracings. In addition, Kappa value and ICC coefficient also were used to evaluate agreement in baseline determination between a computer-assisted method and several experts in previous studies. Obtained Kappa values varied from 0.18 to 0.97, while the ICC coefficient was within the range of 0.83–0.98 (Table 2). All these results were obtained on several small datasets with ≤150 FHR tracings acquired with EFM from ≤3 different manufacturers. In the present study, a larger dataset with 1,267 FHR tracings acquired with EFM from 5 different manufacturers was used to evaluate agreement in baseline determination between the CTGNet and clinicians. This dataset included ∼50% high-quality tracings with a signal loss rate of <10% per 10 min and ∼50% low-quality recordings with a signal loss rate of <50% per 30 min. Kappa values were >0.98 for these high-quality tracings from GMU_DB, JNU_DB and SMU_DB, while the Kappa value was 0.771 for the lowest-quality FHR recordings (n = 552) from UHB_DB. Regardless, an excellent agreement was obtained on the whole dataset (n = 1,276). These results indicate possibilities for the clinical application of CTGNet in FHR baseline estimation.

The high loss rate of FHR tracings severely affected the performance of CTGNet. In the present study, the Kappa value was reduced from 1.0 for the FHR recordings from JNU_DB with a low signal loss rate to 0.771 for the low-quality recordings from UHB_DB with a high signal loss rate. Although high-quality recordings with a signal loss of ≤20% are recommended by the FIGO guidelines to assess the FHR patterns (5), low-quality tracings with a mean signal loss of 28%–55% are shown in clinical practice (27, 28). For example, the mean signal loss of 13% and 30% were found during the first and the second stage of labor, respectively (29). Therefore, the CTGNet can be further trained on datasets with low-quality tracings to improve its robustness.

On these five datasets, we further compared the performance of our method (30) with existing signal processing methods. The method proposed by Mantal et al. achieved the best performance on the GMU_DB and JNU_DB datasets. Our method achieved the best results on the JNU_DB and LCU_DB datasets. The method proposed by Boudet et al. achieved the best results on the UHB_DB dataset. In the comprehensive evaluation of the five datasets, Boudet's method and our method ranked first and second respectively, and their baseline differences were both less than 3 bpm (Table 3).

The CTGNet for the FHR baseline estimation provided an excellent agreement with clinicians. However, this occurs in FHR recordings with low and medium signal loss rates. In the future, the CTGNet can be further improved by training it with more low-quality tracings.