Clinical data of all children with NEC admitted to the neonatal unit of Children's Hospital of Soochow University from 1 January 2012 to 31 December 2021 were retrospectively analyzed, and the study was pre-approved by the institutional review board. The inclusion criteria for this study were the modified Bell staging ≥ stage II. The exclusion criteria were the following: children with immunodeficiencies, inherited metabolic disorders, congenital malformations of the intestine, and patients with pneumoperitoneum detected during preoperative examination, regardless of the presence of NEC. Additionally, cases of infants transferred from other hospitals who underwent a change in treatment from conservative to surgical without adequate clinical data were excluded. Enrollment in all cases was independently reviewed by two experts, with a third expert consulted in case of disagreements. All clinical and demographic data utilized in this study were retrospectively extracted from medical records. These data encompass patient characteristics and clinical parameters recorded at the time of NEC diagnosis, as well as subsequent clinical deterioration or preoperative status prior to surgical intervention. The study was approved by the Ethics Committee of the Children's Hospital of Soochow University (2023CS189; Suzhou, China), informed consent was waived by the ethics committee. The study was adhered to the tenets of the Declaration of Helsinki.

Variables including general data, maternal prenatal general data, underlying disease and comorbidities, treatment before diagnosis of NEC, laboratory testing and imaging, nutritional management before NEC diagnosis were collected and analyzed. Cases with more than 50% missing data for any variable were excluded from the analysis. Additionally, variables with more than 50% missing values were also excluded. For the remaining missing values, mean imputation was applied for continuous variables, and mode imputation was used for categorical variables.

Certain indicators, such as WBC, PLT, lymphocyte count, CRP, and procalcitonin, were categorized due to their clear normal ranges in neonates on the first day of life, facilitating the detection of abnormalities. In contrast, variables like hemoglobin and albumin were kept as continuous data to better capture subtle changes and their correlation with disease progression. This approach also allows the CatBoost algorithm to model nonlinear relationships more effectively, thereby improving the model's predictive accuracy.

Binary labels were constructed based on the dataset after dividing the population into two groups: conservative treatment NEC and surgical NEC using the revised Bell staging to define the different treatment modalities in the model. The conservative group involved conservative management, which included cessation of feeding, parenteral nutrition, and empirical antibiotic therapy. Surgical indications were defined by radiographic evidence of significant bowel perforation (pneumoperitoneum) or, in the absence of imaging findings, clinical deterioration despite prolonged conservative management. Infants who died in the conservative group were classified into the surgical group to mitigate potential bias, as these infants were likely candidates for surgery but were unable to receive it in a timely manner due to the progression of their condition. Surgical intervention was defined as including abdominal drainage and exploratory laparotomy. If the disease outcome was in the surgical group, the label was 1, otherwise, the label was 0. This binary labelling approach was used for both the training set and the validation set.

The goal of this study was to accurately predict whether a child would progress to NEC requiring surgery based on the clinical data (clinical indicators, test and examination results). The outcome was a binary categorical problem in which the target variable was the binary short-term progression of NEC. The explanatory variables were the parameters included in the measurement. Prediction was performed using two machine learning algorithms, including LR and CatBoost. The LR and Catboost models were implemented using Python packages.

The dataset was randomly divided into two subsets: 70% of the data was allocated for training the models, while the remaining 30% was reserved for the validation cohort. This division allowed for internal validation and ensured that model evaluation was conducted on data not seen during training, thus minimizing the risk of overfitting.

Model performance was assessed exclusively on the validation cohort to evaluate the models' generalizability to unseen data. Key performance metrics included the area under the ROC curve and its corresponding 95% confidence intervals. The AUC score was used to quantify the models' ability to distinguish between infants requiring surgical vs. conservative treatment for NEC. Additionally, other performance measures, such as sensitivity, specificity, and accuracy, were calculated to provide a comprehensive evaluation of the models' predictive capabilities.

A modern tool for interpretable AI called SHapley Additive exPlanations (SHAP) was used to determine the most important factors and their contribution to the prediction. This so-called SHAP value enabled the interpretation of the results of a machine learning model using game theoretic concepts: the Shapley value. The SHAP value quantified the marginal contribution of each feature to the final prediction. This study used the CatBoost machine learning algorithm for the final model, which was interpreted using the SHAP value.

The Graphics User Interface (GUI) was developed using the Python-based Tkinter module.

Statistical analysis was performed using Python version 3.6.5 and SPSS 26.0 software. Count data were expressed as cases and percentages (%), and the comparison between the two groups was made using the χ² test or Fisher's exact probability method. Measurement information was expressed as median (interquartile spacing) [M (P25, P75)], and comparison between groups was performed using the Mann–Whitney U test. Covariance was assessed on the results of univariate logistic regression analysis (all tolerances ≥ 0.1 and VIF values ≤ 10). Variables with statistically significant difference in univariate logistic regression analysis were included in logistic regression analysis and Catboost regression analysis. Predictive efficacy was evaluated using the AUC, and the optimal threshold was established by maximizing the Youden index (sensitivity + specificity − 1). A two-tailed value of P < 0.05 was considered statistically significant.

This study involved 449 NEC cases in the cohort, all Bell stage II or higher, for predictive modelling. Thirty-five children with pneumoperitoneum on preoperative examination, thirty-one children with more than 50% missing case records, three children with congenital intestinal malformations (congenital megacolonopathy, intestinal atresia, and intestinal malrotation), and one child with spontaneous intestinal perforation were excluded during the study period. A total of 379 children were finally enrolled in this study, with 142 receiving surgical treatment (including 39 infants who died in the conservative treatment group) and 237 receiving conservative treatment. Details of the study design are shown in Figure 1. These children were randomly assigned to either the training or validation set (7:3). The clinical characteristics of the two groups of patients are listed in Table 1.

Among the general data, the gestational age and birth weight of NEC infants treated with surgery were significantly lower than those of NEC infants treated with conservative treatment. Specifically, the gestational age in the conservative treatment group ranged from 25.14 to 41.14 weeks (median: 31.57 weeks), while in the surgery group, it ranged from 27.00 to 41.71 weeks (median: 30.86 weeks). Regarding birth weight, the conservative treatment group showed a range from 750 g to 5,000 g (median: 2,000 g), while the surgery group ranged from 600 g to 4,500 g (median: 1,500 g). The proportion of preterm infants in the surgical group was higher than that in the conservative group. As regards the maternal prenatal general data, the proportion of cesarean delivery, antenatal corticosteroids treatment, and premature rupture of membranes ≥18 h in the surgery group were higher than those in the conservative group. As regards the underlying diseases or comorbidities before the diagnosis of NEC and treatment measures, the surgical treatment group had significantly higher proportion including frequent sleep apnea, type II respiratory failure, hyponatremia, anemia, use of drugs to close PDA, NRDS, sepsis, pulmonary alveolar surfactant, aminophylline, caffeine, and vasoactive drugs compared with the conservative group. As regards the laboratory results of blood test, the surgical treatment group had significantly higher parameters including WBC < 5 × 10⁹/L, PLT < 100 × 10⁹/L, lymphocyte count < 2.0 × 10⁹/L, CRP ≥ 20 mg/L, and PH < 7.25 compared with the conservative group. And the surgical treatment group had significantly lower parameters including hemoglobin, albumin, and base excess compared with the conservative group. The results of abdominal x-ray examination showed that patients with intestinal wall thickening and portal vein gas in the surgery group were more than those in the conservative group. In terms of nutritional management before NEC diagnosis, The surgical group had significantly fewer cases of initiating breast feeding before the onset of NEC and fewer cases of feeding speed > 25 ml/(kg·day) compared to the medical treatment group.

A significant difference in the following variables between the two groups was found by univariate analysis: birth weight, gestational age, premature infants, cesarean delivery, antenatal corticosteroids treatment, premature rupture of membranes >18 h, frequent sleep apnea, type II respiratory failure, hyponatremia, anemia, use of drugs to close PDA, NRDS, sepsis, alveolar surfactant, aminophylline, caffeine, vasoactive drugs, WBC < 5 × 10⁹/L, hemoglobin, PLT < 100 × 10⁹/L, lymphocyte count < 2 × 10⁹/L, albumin, CRP ≥ 20, PH < 7.25, base excess, intestinal wall thickening, portal vein gas, breastfeeding initiation, and feeding speed > 25 ml/(kg·day). The Collinearity test was performed on the results of univariate analysis, and the results showed that VIF values were less than 10, suggesting no significant collinearity between the variables.

The Catboost model (AUC = 0.910) predicted NEC surgery better than Logistic regression (AUC = 0.848). The discriminative performance of these 2 models is shown in Figure 2, where the best cut-off value with the largest Youden index was taken to calculate the sensitivity, specificity, precision and accuracy. The sensitivity of Catboost and LR were 61.5% and 64.1%, the specificity was 82.6% and 82.5%, the precision was 0.857 and 0.735, and the accuracy was 0.833 and 0.798, respectively. The above results demonstrated that the Catboost model performed better in the prediction of surgical intervention in NEC between the above 2 models.

The final model was identified during the feature dimension reduction of the Catboost model. The 29 feature models were significantly better than the 10 feature models (AUC = 0.872) and 15 feature models (AUC = 0.902) in predicting NEC surgical intervention, but not significantly better than the 14 feature models (AUC = 0.905) in Figure 3. The 14 feature models have better net benefits and higher threshold probabilities compared with the 29 feature models. Therefore, our attention focused on the characteristics of the 14 Catboost model. These included base excess, WBC < 5 × 10⁹/L, vasoactive drugs before NEC, gestational age, serum albumin, sepsis, birth weight, PH < 7.25, hemoglobin, portal vein gas, feeding speed > 25 ml/(kg day), type-2 respiratory failure, use of drugs to close PDA before NEC, and frequent sleep apnea, which were used as the final model for further analysis. The AUC, specificity, sensitivity, precision, and F1 score of the Catboost model for predicting NEC were 0.905, 0.833, 0.641, 0.858 and 0.734, respectively.

The output results of the final model were interpreted with the help of the SHAP method, and the average SHAP value was used to reflect the contribution of the features to the model. The influence of individual features on the model output and their importance ranking are shown in descending order in Figures 4A,B. Dot is made for SHAP value in the model for each infant, the colors of the dots demonstrate the actual values of the features for each one, as red means a higher feature value and blue means a lower feature value. The interpretation of the predictions using SHAP values are shown in Figure 4C. Red lines indicates features that drove a higher prediction probability of NEC surgery, and blue lines indicates features that drove a lower prediction probability. In addition, the length of the lever was proportional to the corresponding factors to predict the degree of contribution.

A Graphics User Interface (GUI) software was developed to facilitate clinicians to use the Catboost model. The GUI software screenshots are shown in Figure 5. The GUI software can be downloaded from https://nec-models.oss-cn-shanghai.aliyuncs.com/surgicalNECpredicted.exe and https://nec-models.oss-cn-shanghai.aliyuncs.com/model.pkl (please ensure that both files are placed in the same folder for the software to function properly). Clinicians or other users only need to input 14 feature values to directly obtain the prediction results to understand whether a child with NEC needs surgical intervention. This is beneficial to the choice of surgical timing and early intervention to improve the prognosis.