This multicenter, retrospective, case–control study was conducted at Xuhui Central Hospital, Fudan University, and Putuo People’s Hospital, School of Medicine, Tongji University, from January 2016 to January 2024. The study was approved by the Ethics Committee of Xuhui Central Hospital (2024–003), Fudan University, in accordance with the Declaration of Helsinki. Informed consent was obtained from all participants.

A total of 5,214 patients with CAP were consecutively enrolled in the study, including 3,127 individuals in the discovery cohort from Xuhui Central Hospital and 2,087 individuals in the validation cohort from Putuo People’s Hospital. Demographic characteristics and clinical information were gathered from the electronic medical records system of the hospital.

The inclusion criteria (Farhat et al., 2024; Zhu et al., 2024) and exclusion criteria (Glöckner et al., 2022; Farhat et al., 2024; Zhu et al., 2024) were applied as previously described and are summarized as follows.

The inclusion criteria were as follows: community-acquired infection; age ≥18 years; new or worsening lung infiltrates observed on chest imaging; at least one of the following: fever (temperature ≥38.3 °C), cough, purulent sputum production, or focal chest signs on auscultation; and routine blood tests results obtained at the time of presentation.

The exclusion criteria were as follows: acquired or therapy-induced immunodeficiency; active tuberculosis or hospital-acquired infection; pregnancy; use of intravenous antibiotics, antiviral drugs, or glucocorticoids within the past two weeks; and the presence of other respiratory conditions, such as lung cancer, chronic obstructive pulmonary disease, or bronchiectasis. Figure 1 illustrates the inclusion and exclusion process of the study cohort.

On the basis of the “Diagnosis and Treatment of Adults with Community-acquired Pneumonia” guidelines and the Infectious Diseases Society of America/American Thoracic Society (IDSA/ATS) guidelines, patients with CAP in both model cohorts were classified into mild and severe categories (Metlay et al., 2019). Severe CAP is defined by the presence of either one major criterion or three or more minor criteria. Major criteria: (1) Respiratory failure requiring invasive mechanical ventilation; (2) Septic shock requiring vasopressor support. Minor criteria: (1) Respiratory rate ≥ 30 breaths/min; (2) PaO₂/FiO₂ ratio ≤ 250; (3) Multilobar infiltrates; (4) Confusion or disorientation; (5) Blood urea nitrogen ≥ 20 mg/dL; (6) white blood cell count < 4,000/mm³; (7) platelet count < 100,000/mm³; (8) core temperature < 36 °C; (9) Hypotension requiring aggressive fluid resuscitation (systolic blood pressure < 90 mmHg).

All patients with CAP were categorized into five risk classes (I–V) based on the Pneumonia Severity Index (PSI) score (Fine et al., 1997).

Laboratory tests were conducted in duplicate at the Department of Clinical Laboratory, Shanghai Xuhui Central Hospital and Putuo People’s Hospital, as previously described (Cheng et al., 2021), and are summarized as follows.

Blood samples were collected from each patient prior to the administration of any medications. A 2 mL blood sample was drawn using standard venipuncture in the antecubital fossa and collected in an ethylenediaminetetraacetic acid tube. Routine blood tests were performed with a Mindray BC-series automatic blood cell analyzer (Shenzhen, China) within 30 minutes of blood collection. Twenty-four variables were recorded from the routine blood tests: neutrophil count, neutrophil%, red blood count (RBC), platelecrit (PCT), platelet count (PLT), platelet distribution width (PDW), hemoglobin (HG), eosinophil count, eosinophil%, basophil count, basophil%, MPV, lymphocyte count, lymphocyte%, hematocrit (HCT), monocyte count, monocyte%, platelet large cell ratio (PLCR), WBC, red blood cell distribution width-standard deviation (RBWSD), red blood cell distribution width-coefficient of variation (RBWCV), mean corpuscular volume (MCV), mean corpuscular hemoglobin concentration (MCHC), and mean corpuscular hemoglobin (MCH).

Models were built from the 24 features from the routine blood tests via 12 machine learning algorithms, including AdaBoost, decision tree (DT), light gradient boosting (LGB), k-nearest neighbors (KNN), generalized linear model (GLM), logistic regression (LR), random forest (RF), gradient boosting (GB), support vector machine (SVM), extreme gradient boosting (XGB), naïve Bayes, and TabNet. The models were optimized through grid search and manual tuning.

The average area under the receiver operating characteristic (ROC) curve (AUC) from fivefold cross-validation was used to determine the optimal hyperparameters. Model performance was assessed using metrics such as the AUC, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, and F1 score. The DeLong nonparametric test was used to compare AUC values across the models and identify the best-performing model. The cut-off thresholds used in the ROC curve analyses were determined based on the Youden Index, which identifies the point on the ROC curve that maximizes the sum of sensitivity and specificity (i.e., sensitivity + specificity-1).

The model parameters were as follows (Cheng et al., 2025): XGB (objective=‘multi:softmax’, colsample_bytree=1.0, learning_rate=0.2, max_depth=7, n_estimators=100, subsample=0.8), RF (n_estimators=100, max_depth=10, random_state=42), DT (max_depth=10, random_state=42), SVM (n_splits=5, shuffle=True, random_state=42), GNB (learning_rate=0.1, max_depth=7, min_samples_leaf=1, min_samples_split=2, n_estimators=50, subsample=1.0), TabNet (max_epochs=100, patience=20, batch_size=32, virtual_batch_size=128, num_workers=0, drop_last=False, weights=1), KNN (n_neighbors=5), LGB (max_depth=10, learning_rate=0.11, n_estimators=100, random_state=42), LR (max_iter=1000, random_state=42), GLM (family=sm.families.Binomial), DT(‘criterion’: ‘entropy’, ‘max_depth’: 5, ‘min_samples_leaf’: 4, ‘min_samples_split’: 2), and GB (learning_rate’: 0.1, ‘max_depth’: 7, ‘max_features’: ‘log2’, ‘min_samples_leaf’: 1, ‘min_samples_split’: 2, ‘n_estimators’: 50, ‘subsample’: 1.0).

Feature reduction helps eliminate noise, streamline models, and enhance their interpretability. In our study, SHapley Additive exPlanations (SHAP) was employed to quantify the importance of each feature in the two-class machine learning model. Features were ranked based on their mean absolute SHAP values in descending order.

To determine the optimal feature subset, we progressively reduced the number of features from the full set (24 variables) down to a single feature, sequentially evaluating model performance at each step. For each reduced feature set, a new model was trained, and its AUC was compared to that of the full-feature model using the DeLong nonparametric test. A p-value < 0.05 was considered statistically significant. The smallest feature subset that maintained comparable predictive performance (i.e., no significant drop in AUC) was selected as the optimal feature set for the final model. This model was then evaluated in the validation cohort to assess its generalizability.

To facilitate clinical application, the final model was implemented as a web application using the Streamlit Python framework. In the application, users can input the feature values, and the model will provide the probability for each category along with the most likely category.

This calculation was performed under the assumption of a balanced case–control design. To estimate the minimum required sample size, we conducted a power analysis using PASS software. The input parameters were set as follows: expected sensitivity and specificity of 0.80, each with a margin of error of 0.05, and a two-sided significance level (α) of 0.05. Based on these criteria, the calculated minimum sample size was 246 participants per group to achieve sufficient statistical power for evaluating the diagnostic performance of the biomarkers.

Normality was assessed with the Shapiro–Wilk test. Statistical differences between groups were analyzed with appropriate tests: the independent-samples Student’s t test for normally distributed continuous variables, the Kruskal–Wallis test for nonnormally distributed continuous variables, and the chi–square test for categorical variables. The relationships among the variables were assessed via Pearson analysis. Continuous variables are expressed as the means ± SDs, and categorical variables are reported as frequencies and percentages. Data analysis and graphing were performed in GraphPad Prism Software version 9.0 (GraphPad Software, Inc., San Diego, CA, USA), SPSS software (version 19.0; SPSS Inc., Chicago, IL, USA), R 4.0.2 (R Core Team), Python (version 3.11), and PyCharm (version 2023.3.5). A p value < 0.05 was considered to indicate statistical significance.

A total of 5,214 patients with CAP were enrolled in the study, comprising 2,684 mild patients and 2,530 severe patients. There were no significant differences (P>0.05) between the mild and severe groups across key indicators. Significant differences were observed in all routine laboratory indicators between mild and severe CAP patients (P < 0.05) except the eosinophil count and basophil count (P > 0.05), as shown in Supplementary Table S1.

A total of 3,217 patients were enrolled in the discovery cohort, consisting of 1,612 patients with mild disease and 1,615 with severe disease. There were also no significant differences (P>0.05) between the mild and severe groups across these indicators. In this cohort, significant differences were observed in all routine laboratory indicators between mild and severe patients (P < 0.05) except for monocyte percentage, eosinophil count, and basophil count (P > 0.05). These findings are consistent with the results observed in the entire cohort, as detailed in Supplementary Table S2.

The validation cohort, consisting of 2,087 participants (1,072 with mild disease and 1,015 with severe disease), was recruited. Similar to the findings in the discovery cohort, significant differences in all routine laboratory indicators were found between mild and severe patients (P < 0.05) except the eosinophil count and basophil count (P > 0.05), which is also in line with the findings from the whole cohort, as shown in Supplementary Table S3.

All 24 features from the routine blood tests were employed to train classification models based on 12 machine learning algorithms, including AdaBoost, DT, LGB, KNN, GLM, LR, RF, GB, SVM, XGB, naïve Bayes, and TabNet.

In terms of the AUC (Figure 2A), the performance of the AdaBoost (AUC = 0.95), GB (AUC = 0.95), LGB (AUC = 0.95), LR (AUC = 0.95), RF (AUC = 0.95), TabNet (AUC = 0.95), XGB (AUC = 0.95), and GLM (AUC = 0.95) models was significantly better (P < 0.05, Figure 2B) than that of the other models, whose AUC values ranged from 0.91 to 0.93. Additionally, the area under the precision–recall curve (AUPRC) of the models based on the AdaBoost, GB, LGB, LR, RF, TabNet, XGB, and GLM algorithms were also higher than those of the other models (Figure 2C), confirming the superior performance of the former. A detailed description of the performance of these 12 models is provided in Figure 2D and Supplementary Table S4.

Given these results, the AdaBoost, GB, LGB, LR, RF, TabNet, and XGB models and the GLM were selected for inclusion in the subsequent classification model development steps.

To further enhance the clinical applicability of the models, we conducted feature reduction. Features were selected using the SHAP method, and on the basis of the resulting feature importance ranking, the feature set was progressively reduced from 24 features to a single feature to determine the optimal set for maximizing predictive performance (Figure 3A, Supplementary Table S5). For the AdaBoost, GB, LGB, LR, RF, TabNet, and XGB, and the GLM, reducing the number of features for constructing the models to 10, 10, 13, 10, 9, 10, 14, and 10, respectively, resulted in no significant differences in performance with the corresponding models built from the full feature set (P > 0.05, Supplementary Table S5). Interestingly, the AUCs of the AdaBoost, GB, LGB, LR, RF, TabNet, and XGB models and the GLM were consistently 0.95 with the reduced feature sets.

The final model was subsequently selected according to number of features incorporated as well as the values of performance metrics such as sensitivity, specificity, PPV, NPV, accuracy, and F1 score (Supplementary Table S6). In this way, the RF model, which included only 9 selected features (Figure 3B), including LYMPH%, RDW-CV, MONO#, HGB, RBC, NEUT%, WBC, NEUT#, and HCT, and demonstrated strong performance across the abovementioned metrics, was chosen as the final model.

The 9-feature RF model achieved an AUC of 0.95 (Figure 3C) and an AUPRC of 0.94 (Figure 3D), with a PPV of 0.89, NPV of 0.88, accuracy of 0.89, and F1 score of 0.89, as detailed in Supplementary Table S6. A confusion matrix (Figure 3E) was created to visualize the performance of the RF model, showing that the sensitivity was 0.89 and the specificity was 0.90. Decision curve analysis (DCA) revealed that the final 9-feature RF model had greater clinical utility than the other models, as presented in Figure 3F. Finally, a heatmap (Figure 3G) was used to visualize the correlations among the original 24 variables.

All models developed in the discovery cohort required validation with the independent validation cohort to assess robustness. Therefore, we employed the previously designated validation cohort to verify the performance of the 9-feature RF model. Consistent with the findings in the discovery cohort, the RF model demonstrated remarkable performance in assessing disease severity in patients with CAP. In the validation cohort, the RF model achieved an AUC of 0.95 (Figure 4A) and an AUPRC of 0.94 (Figure 4B), with a PPV of 0.88, an NPV of 0.87, an accuracy of 0.88, and an F1 score of 0.87. The confusion matrix (Figure 4C) also demonstrated the performance of the RF model, showing that the sensitivity was 0.86 and the specificity was 0.89.

The DCA plot in Figure 4D shows that the final 9-feature RF model provided consistent net benefits across the entire range of threshold probabilities, highlighting its desirable clinical utility. A heatmap (Figure 4E) was used to visualize the correlations among the 24 variables in the validation cohort.

Finally, we conducted a direct head-to-head comparison between our final 9-feature RF model and the PSI score. As shown in Supplementary Figure S3A, the PSI score achieved an AUC of 0.76, with a cutoff at risk class IV, yielding a sensitivity of 80.76% and a specificity of 80.34%. The DCA plot in Supplementary Figure S3B further demonstrates that the PSI score consistently provided net clinical benefits across the entire range of threshold probabilities, underscoring its practical utility in clinical decision-making. Overall, the performance of our model exceeded that of the PSI score.

To improve the interpretability and transparency of the model, the SHAP method was used to evaluate the contribution of each feature to the output of the optimal model. The global SHAP provide an overview of the model’s behavior, from which SHAP summary plots (Figure 5A) are generated, showing the ranked contributions of each feature on the basis of their average SHAP values in descending order.

Additionally, SHAP dependence plots were generated to aid in understanding how individual features influence the output of the RF model. Figure 5B illustrates the relationship between the real values and the SHAP values for the 9 selected features: SHAP values greater than zero indicate a positive class prediction, meaning that the model classifies the patient as having severe CAP.

The local SHAP demonstrates how a specific prediction is made for an individual by incorporating individual input data. Figure 5C illustrates the prediction for a CAP patient who was classified into the “severe” category with a probability of 100% according to the RF model. Figure 5D illustrates the prediction for a CAP patient who was classified into the “severe” category with a probability of 0% according to the RF model.

The final RF model was integrated into a web application to enhance its practicality in clinical settings, as shown in Figure 6. Once the user has entered the values for the 9 features required by the model, the application automatically predicts the likelihood of severe CAP in the patient. Additionally, a force plot for the CAP patient in question is displayed, illustrating the features that influence the model toward predicting the severity of CAP. The blue features on the right are those that push the prediction toward “severe,” whereas the red features on the left push it toward “mild.” The web application is accessible online at https://predictionmodel-for-aki.streamlit.app.