This study retrospectively enrolled patients with pulmonary infection who were admitted to our hospital between April 2020 and December 2023. All the patients underwent a CT examination.

The diagnostic criteria were presented as follows. NTM was diagnosed according to the Euro-American 2020 edition (24). MTB was diagnosed according to the rapid tuberculosis diagnostic criteria with 2023 World Health Organization edition (25). To differentiate the NTM and MTB, we conducted the T-SPOT test using a testing kit produced by DEAOU (Guangzhou) according to the kit instructions. Next-generation sequencing (NGS) of alveolar lavage fluid and bacteria culture were also performed to differentiate the NTM and MTB. The collection of alveolar lavage fluid and next-generation sequencing information including sample processing and DNA extraction, and library generation and sequencing can be found in the previous study (26). Integrating the results of multiple tests, the NTM and MTB patients were classified.

The inclusion criteria included (1) age ranged from 18 to 80 years; (2) diagnosed with NTM or MTB infection; (3) had bacterial culture results; (4) had NGS test results; (5) had T-SPOT test results; (6) with at least 2 set of lung CT images available. We excluded these patients with lung cancer, fungal infection, pneumoconiosis, and mixed infections of TB and NTM. The process of patient screening and enrolling was presented in Figure 1. After screening, 99 MTB patients and 76 NTM patients were enrolled in the final analyses.

It should be noted that there was a data imbalance regarding the MTB and NTM patient number, which may cause the potential for producing overfitting problems. Due to the requirements of periodical for follow-up after diagnosis, at least 2 images of NTM/MTB patients were taken. Considering the slow rate of radiological abnormal changes in NTM, longer follow-up time was required. Hence, more CT images of one NTM patient were collected than MTB. Therefore, although there was unbalance in the proportion of patients, CT images were as balanced as possible.

This study aimed to construct 3 models (clinical model, radiomics model, multimodal model) for differentiating NTM from MTB using 5 machine learning algorithms (Logistic, XGBoost, AdaBoost, RandomForest, and LightGBM), and to explore which algorithms was more suitable. The purpose of classified multi-model method was to select the best model, rather than directly modeling to get the final model. In this study, we used the training/verification mechanism of 5-fold cross-validation to summarize the performance of each model in many trainings, focusing on the overall performance of each model. The overall workflow of the models’ development, validation, and comparison was presented in Figure 2.

Logistic regression models are more traditional single-model classification algorithms that aim to identify the connection between features and the likelihood of a specific (binary) outcome. It is widely employed by medical professionals for its ability to calculate odds ratio (27).

Models such as RandomForest, XGBoost, LightGBM, and AdaBoost are ensemble models based on decision trees. These ensemble models can get a strong model through taking the strengths of all single models. This strong classifier achieves a relative best performance.

RandomForest algorithm is the embodiment of group intelligence, which creates different training sets by randomly sampling rows (bagging) and columns (feature bagging) from the dataset. The decision tree of RandomForest algorithm is grown by bootstrap. RandomForest reduces training variance and improves model generalization and integration. And it can be utilized without parameter tuning, offering variable importance information for classification and high predictive accuracy (28).

Conversely, XGBoost, LightGBM, and AdaBoost are based on the idea of gradient boosting. XGBoost performs a second-order Taylor expansion of the loss function and employs various techniques to minimize overfitting. It utilizes the structure score (gain) of the tree to determine split points, improving tree quality, and implements parallel and distributed computation for increased efficiency. For detailed algorithm information, consult the literature (29).

The LightGBM algorithm employs a histogram approximation algorithm to generate a histogram that discretises continuous features. It splits trees at leaf nodes with the largest lift. And using the GOSS technique, the samples with larger gradient are preferred, so the optimal split can be found faster and the training efficiency is improved (30, 31).

AdaBoost is a relatively new nonlinear ML algorithm that builds a tree by adjusting sample weights and combines multiple trees to become a strong classifier. It does not require feature screening, can perform automatic feature selection, and has a low risk of overfitting. For specific steps, refer to the relevant literature (32).

We collected the clinical data of patients including the age (years), gender, white blood cell (WBC, 10⁹/L), erythrocyte sedimentation rate (ESR, mm/h), C-reactive protein (CRP, mg/L), Interleukin-6 (IL-6, pg./mL), and procalcitonin (PCT, ng/mL). The difference in these variables between NTM and MTB groups was first compared to identify the differential variables. These differential variables were then enrolled in Logistic regression analysis to determine the independent factor associated with the disease types. Then Receiver Operating Characteristic (ROC) analysis and area under curve (AUC) were used to assess the discriminating performance of these independent factors. Decision Curve Analysis (DCA) was used to evaluate their obtained clinical net benefit for discriminating disease.

Based on the independent clinical factors, a clinical features-based model for discriminating NTM from MTB was first constructed and validated using 5-fold cross-validation. The cross-validation can partially resolve the overfitting problem. In addition, we added L2 penalty to control the complexity of the model to prevent it from being too complicated. We also applied early stop-ping, learning rate adjustments, and drop-out to prevent overfitting. The detailed parameters of the Logistic algorithm were as follows: C = 1.0; max-iter = 100; penalty = l2; tol = 0.0001. The parameters of the XGBoost algorithm were as follows: learning-rate = None; max-depth = None; min child-weight = None; reg-lambda = None. The parameters of the AdaBoost algorithm were as follows: learning-rate = 1.0; n-estimators = 50. The parameters of the RandomForest algorithm were as follows: criterion = gini; max-depth = None; min impurity-decrease = 0.0; n-estimators = 20. The parameters of the LightGBM algorithm were as follows: boosting type = gbdt; learning rate = 0.1; max depth = −1; n-estimators = 100; num-leaves = 31. The discriminating performance of models from 5 algorithms was evaluated by ROC and DCA analyses.

The CT images of all patients were also obtained. CT scans were performed using 64-slice CT scanners with the following parameters: tube voltage 120 kV; automatic tube current modulation 300 mA; detector collimation 64 × 0.625 mm; thread pitch 0.993; section thickness 2 mm; section interval 2 mm. The scanning area ranged from the apex pulmonic to the bottom of the lung.

All CT images were assessed by two radiologists with 5 years of diagnostic experience in CT, respectively. They were blinded to the histopathological and clinical data of patients. The 2 radiologists manually segmented the region of interest (ROI) of CT images by using 3D slicer software, and the intraclass correlation coefficient (ICC) between the 2 radiologists indicated the consistency of extracted features. ICC > 8 suggested a good consistency between observers.

For feature extraction, all images and ROIs were batched into 3D slicer software. The extracted radiomics features included 162 first-order features, 216 gray-level co-occurrence matrix (GLCM) features, 126 gray-level dependence matrix (GLDM) features, 144 gray-level run-length matrix (GLRLM) features, 144 gray-level size zone matrix (GLSZM) features, and 45 neighboring gray-tone difference matrix (NGTDM) features.

A total of 837 radiomics features were extracted. We then used the Least absolute shrinkage and selection operator (LASSO) analysis to remove the redundant features. By introducing a penalty coefficient (λ), the coefficients of most features will be compressed to 0. The retained features in the final optimal LASSO model were selected for further analysis. LASSO analysis is a regularization method for regression analysis. By introducing L1 regularization into the regression model, the coefficients of some features are reduced to zero, thus realizing variable selection. LAASO can reduce the complexity of the model, and improve the prediction performance of the model. Therefore, application of LASSO analysis can partially resolve the overfitting problem. Then, the differences of the retained features after LASSO analysis between NTM and MTB groups were compared. The differential features between the 2 groups were enrolled in the logistic regression analysis to explore the independent factors. The clinical value of independent factors in disease discrimination was assessed by ROC and DCA analysis.

Based on the independent features, a radiomics features-based differentiation model was constructed in a training set using 5 machine learning algorithms including Logistic, XGBoost, AdaBoost, RandomForest, and LightGBM. Their performance was also verified in the validation set by ROC and DCA analyses. Detailed information for the model construction and validation can be found in the Methods 2.3 section.

The key clinical and radiomics features, that are independently related to the disease types, have been identified in the above analyses. Then these various indicators were enrolled in logistic regression analysis to further determine the independent variables. Based on the key clinical and radiomics features, a multimodal differentiation model was constructed using 5 machine learning algorithms. Their performance was also verified in the validation set by ROC and DCA analyses. Detailed information for the model construction and validation can be found in the Methods 2.3 section. Especially, the importance ranking of features within the multimodal model was explored using 3 optimal algorithms.

Then, the differentiation performance of the 3 models (clinical-based, radiomics-based, multimodal-based) was compared from aspects of AUC, cutoff, accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and Kappa values. Although we have collected data for 4 years to establish and evaluate our models, there was limit to accurately assessing their performance due to the lack of test data, especially for NTM. Therefore, it was necessary to obtain a new external dataset from public data or data from other institutions. Unfortunately, we failed to obtain the test data from other institutions due to some reasons. Currently, no publica data on NTM was provided.

Therefore, another external dataset containing NTM and MTB cases who met the diagnostic criteria were retrospectively found. From January 2024 to December 2024, totally 59 patients (20 NTM and 39 MTB patients) were collected. These data differed from the patient data used when constructing the learning model, and were the newly generated data after performing learning model. Only one image was collected per patient in the new external dataset. We also validated the performance of our models for differentiating NTM and MTB using the new external test dataset. The differentiation performances of 3 models were also compared in the new external test dataset.

In addition, we compared the differentiation performance of multimodal model in the new external test dataset with other measures including NGS detection method, radiologist assessment with 5-year experiences, and existing machine learning model. In terms of radiologist assessment, one radiologist identified the pathogen that infected the patients in the testing dataset by only scanning CT images without any other reminding. The radiologist was informed that the patients were only infected by one type of pathogen, either NTM or MTB. The existing machine learning model was searched from the published articles. Currently, few studies reported the machine learning model for the differentiation of NTM from MTB. Finally, a deep learning model based on CT images by Wang et al. was selected for the performance comparison (33). For the differentiation performance comparison among different methods, AUC, accuracy, sensitivity, specificity, MTB-precision, and NTM-precision were set as the indicators.

The continuous variables were presented as median and quartiles as they did not confirm to the normal distribution, and their differences between the 2 groups were compared with the Mann–Whitney U test. The categorical variables were expressed as frequency and percent, and their distribution difference between the 2 groups was analyzed by χ² test. Univariable and multivariable logistic regression analyses were used to explore the association between variables and disease types. ROC and DCA were used to assess the discriminating performance. p < 0.05 was considered statistical significance.

In this study, we enrolled 175 patients infected by NMT or MTB to our analysis the median age was 63 [51, 70] years old, with average age of 58 years old. The median white blood cell was 5.6 [4.7, 7.2] (10⁹/L). The median CRP was 4.4 [1.3, 21.0] (mg/L). The median ESR was 32.0 [12.5, 59.5] (mg/L). The median IL-6 was 26.65 [4.38, 56.22] (pg/mL). Male patients accounted for 54% of all patients. Of the 76 patients diagnosed with NTM, 35 patients (46.1%), 23 patients (30.2%), 15 patients (19.7%), and 3 patients (4.0%) were infected by M. avium, M. intracellulare, M. abscessus, and M. kansasii strains of NTM, respectively.

The baseline characteristics of MTB (N = 99) and NTM (N = 76) patients are presented in Table 1. It followed that age (p < 0.001), gender (p = 0.002), and IL-6 level (p < 0.001) showed significant differences between MTB and NTM groups. NTM group had higher median age (66 vs. 57), but lower WBC (5.4 vs. 5.9), CRP (4.4 vs. 5.0), ESR (27 vs. 32), and IL-6 (15.6 vs. 39.5) levels than MTB group.

Based on 3 significant baseline characteristics, we next explored their association with the disease types. The univariable logistic regression presented their significant association (Figure 3A, all p < 0.05), and the multivariable regression analysis further displayed their independent association (Figure 3B, all p < 0.05). Their differential performance of TB and NTM was then assessed by ROC analysis (Figure 3C), finding that age had the highest AUC value (0.672), followed by IL-6 (0.667). DCA analysis showed that the 3 clinical features obtained similar clinical net benefits for discriminating TB and NTM (Figure 3D).

Subsequently, we aimed to construct a clinical feature-based differentiation model. Before model construction, all patients were assigned to training and validation sets. The 3 clinical feature-based differentiation models were first constructed in the training set using 5 deep learning algorithms, and then the differentiation performance of the model was verified in the validation set. The results showed that XGBoost and RandomForest had the most favorable differentiation performance in the training set (Figure 4A). In the validation set, the RandomForest algorithm achieved the highest AUC value (Figure 4B). Integrating the results of the training set and validation set, XGBoost algorithm may show the overfitting and RandomForest algorithm had more stable differentiation performance. Hence, 3 clinical feature-based models by the RandomForest algorithm were regarded as the optimal deep learning model.

Besides the key clinical features, the CT images can also provide some valuable information for disease differentiation. To reveal the potential value of CT images, we explored significant radiomics features contributing to disease differentiation behind the CT images. A total of 837 radiomics features were extracted. To identify the most valuable features among 837 features, we performed the LASSO analysis to remove the redundant features by compressing their coefficients to 0. The optimal LASSO model (Figures 5A,B) was obtained when the standard error of minimum distance presented the λ = 0.067, containing 7 non-zero radiomics features. The detailed information on these 7 radiomics features is shown in Table 2.

Among 7 features, X709 (GLSZM-Gray Level Non-Uniformity Normalized) showed no difference between the 2 groups (Table 2). The remaining 6 features were further found to correlate with the disease type in univariable logistic regression analysis (Table 3, all p < 0.05), and multivariable regression analysis showed that X75 (GLSZM, Gray Level Variance), X210 (GLCM, Correlation), and X751 (FIRSTORDER, Maximum) were independent factors of disease type (all p < 0.05). It should be stated that the confidence interval of X210 was significantly abnormal, hence X210 was removed from our analysis. Finally, only X75 and X751 features were entered into our further analyses.

ROC analysis showed that X75 had a higher AUC value for discriminating the disease than the X751 (Figure 6A, P for Delong test<0.001), with a sensitivity of 0.713 and specificity of 0.591. The obtained clinical net benefit of X75 was also superior to that of X751 (Figure 6B). Based on X75 and X751, a 2 radiomics features-based differentiation model was constructed using 5 deep learning algorithms in our training set, and the differentiation performance of the model was verified in the validation set. In the training set, XGBoost and RandomForest algorithms achieved the highest AUC values, while they achieved the lowest differentiation performance in the validation set (Figure 6C). It followed that the Logistic algorithm achieved a relatively stable performance both in training and validation set. Hence, 2 radiomics features-based model from the Logistic algorithm was regarded as the optimal deep learning model.

The above results have demonstrated the importance of 3 clinical features (age, gender, IL-6 level) and 2 radiomics features (X75, X751) in the differentiation of lung disease, respectively. We further combined these 5 features and confirmed their independent role in the disease types. The multivariable regression analysis showed that age, IL-6, X75, and X751 were all independently related to the disease types (Figure 7A). Further, we constructed a multimodal differentiation model based on the 4 features in the training set, followed by verification in the validation set. Considering the model performance in training and validation sets, the LightGBM algorithm achieved a relatively favorable and stable differentiation performance (Table 4). Therefore, the multimodal model constructed by the LightGBM algorithm was regarded as the optimal model.

Our results showed that the best 3 clinical features-based models, 2 radiomics features-based models, and the multimodal model were from RandomForest algorithm, Logistic algorithm, and LightGBM algorithm, respectively. Therefore, we next used the 3 algorithms to rank the importance of 4 features within the multimodal model. The results showed (Figure 7B) that age and X75 were the top 2 features among the 3 algorithms.

Finally, we compared the differentiation performance of 3 optimal models constructed by corresponding algorithms only in the validation set due to the stability consideration. The optimal multimodal model from the LightGBM algorithm (Figure 8A) had the highest AUC value (0.804) compared with the optimal clinical model (0.756) and radiomic model (0.718). In addition, the optimal multimodal model (Table 5) had the highest accuracy (0.724), sensitivity (0.875), and NPV (0.738). However, the multimodal model had the lowest specificity (0.693). The PPV values of the optimal multimodal model was in between the clinical model and the radiomics model. The clinical net benefit of 3 models seemed to be similar (Figure 8B). To better visualize the net benefit, we next performed the DCA analysis only based on the Logistic regression (Figure 8C) among the whole population, finding that the multimodal model had better clinical net benefit.

We further verified the prediction performance among 3 clinical data based-model, 2 radiomics based-model, and multimodal model for differentiating NTM from MTB using a new external testing dataset. There were 36 male and 23 female patients in the new external testing dataset. The median age, IL-6, X75, and X751 were 61 [52, 67] years old, 10.9 [6.4, 33.2] (pg/mL), 4.77 [3.68, 5.42], and 766 [751, 771], respectively. The comparison results showed that the multimodal model had the highest AUC, accuracy, sensitivity, and NPV, but had the lowest specificity (Table 6). These results were similar to those from the validation set, which suggested the stability of our findings.

We also compared the differentiation performance of our multimodal model with the only NGS detection method, only radiologist assessment, and existing machine learning model constructed by Wang et al. (33) for differentiating NTM from MTB. The results (Table 7; Figure 9) showed that our multimodal model had the highest AUC, accuracy, and sensitivity. The NGS detection also had favorable sensitivity. Radiologist assessment had the highest specificity and favorable accuracy. The published model by Wang et al. using CT images only had the highest precision for identifying NTM (yes vs. no). Our multimodal model improved accuracy than NGS, radiologist, and existing machine learning model, with an increased accuracy of 26, 4, and 6%, respectively. It also significantly improved sensitivity than radiologist and existing machine learning model, with an increased sensitivity of 4.5 and 15%, respectively, but was similar with that of NGS detection. These results highlighted the superiority of our multimodal model for differentiating NTM from MTB compared with existing approaches or existing machine learning model. Our model significantly improved the differentiation performance and accuracy, and provided favorable sensitivity at the same time.