This retrospective cohort study analyzed data from the Second Affiliated Hospital of Anhui Medical University (AMU) and the Medical Information Mart for Intensive Care IV (MIMIC-IV, version 3.0). The AMU database comprises records from a large regional medical center, whereas MIMIC-IV is a widely used, open-access database comprising data collected at the Beth Israel Deaconess Medical Center. Ethical approval was obtained from AMU (approval code YX2024-204) and access to MIMIC-IV was granted after completion of the required training (record 39691989). All patient data were de-identified and encrypted to ensure privacy, and the ethical documentation is available online. This study adheres to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis–Artificial Intelligence (TRIPOD-AI) reporting guidelines, and the completed checklist is provided in the Supplementary Material .

ICU-BSIs were diagnosed according to Centers for Disease Control and Prevention (CDC)/National Healthcare Safety Network (NHSN) guidelines (Horan et al., 2008). The inclusion criteria for suspected ICU-BSIs were: (a) blood cultures obtained ≥48 hours after ICU admission; (b) first ICU admission; and (c) age ≥18 years. The exclusion criteria were: (a) the presence of a related infection at another site; and (b) prior or repeated ICU admissions.

Figure 1 shows the study workflow. We included 5,399 patients: 3,496 from the MIMIC-IV dataset and 1,903 from the AMU dataset. AMU patients were randomly split into training and test sets (hereafter referred to as the internal validation set) in a 4:1 ratio. The training set was used for model development, whereas the test set and the MIMIC-IV dataset served as the internal and external validation sets, respectively. The model incorporated demographic factors, the Charlson Comorbidity Index (CCI), laboratory measurements, and treatment-related variables; the outcome was 28-day mortality ( Supplementary Table S1 ). Based on established criteria for sample-size calculation (Riley et al., 2019), the sample size was adequate ( Supplementary Table S2 ).

Variable selection was guided by clinical value and accessibility, acknowledging that ideal predictors should support early diagnosis and prognostication with adequate specificity and sensitivity. Variables were selected according to the following principles: (1) to maintain model parsimony, pairwise associations were minimized (absolute Spearman’s ρ < 0.5; Supplementary Figure S1 ); (2) variables with >50% missingness were excluded ( Supplementary Figure S2 ); and (3) to ensure model stability and practicality, variables were chosen based on their clinical relevance, objectivity, and accessibility. Because this was a retrospective analysis, outliers and missing values were addressed before model development. For variables with <50% missingness, multiple imputation was performed using the Multivariate Imputation by Chained Equations (MICE) framework with m = 50 datasets: Predictive Mean Matching for continuous variables, logistic regression (LR) for binary variables, and polytomous (multinomial) regression for categorical variables; a fixed random seed (123) was used to ensure reproducibility. Analyses across imputed datasets were pooled using Rubin’s rules. This approach reduced bias, preserved statistical power, and enhanced model stability.

Given that patients’ laboratory parameters can exhibit substantial short-term fluctuations in patients with ICU-BSIs, variables were time-aligned to the blood culture collection timestamp, and change-related features were engineered to enhance patient-level personalization and reduce redundancy. In this study, t₀ denotes the blood culture collection time, serving as the baseline reference point, while t₁ represents the time of variable extraction, which may occur either before or after t₀ . Accordingly, when t₁ > t₀ , it indicates the post–blood culture period, and when t₁ < t₀ , it corresponds to the pre–blood culture period. The mean relative rate of change of a variable X between the two time points is defined as:

Note: Δt=t ₁-t ₀

Based on this definition: X_v1 denotes the mean relative rate of change during the post–blood culture period (t₁ > t₀ ); X_v-1 denotes the mean relative rate of change during the pre–blood culture period (t₁ < t₀ ). X_v-1 represents the normalized rate of change rather than the maximum absolute value. Since the calculation produces multiple rates of change over time due to temporal rolling, we selected the rate with the largest absolute value. In this framework, positive values indicate an increase in the corresponding indicator, whereas negative values indicate a decrease.

The follow-up period was defined as 28 days from the time of blood culture collection. Patients who remained alive at 28 days were classified as survivors. Data beyond the 28-day period were not incorporated into the model, and subsequent survival status was not further tracked.

Continuous variables were presented as medians and interquartile ranges (IQRs), and group comparisons were performed using the Mann–Whitney U test. Categorical variables were reported as frequencies and percentages, with group comparisons conducted using the chi-square test or Fisher’s exact test, as appropriate. The eXtreme Gradient Boosting (XGBoost) algorithm was chosen for model development because of its strong predictive performance in sepsis and related disease diagnosis and prognosis (Hou et al., 2020; Wang et al., 2025; Zhou et al., 2024). Hyperparameters were optimized using grid search and cross-validation to ensure model stability. Shapley Additive Explanations (SHAP) were used to interpret and visualize relationships between input variables and outcomes; SHAP values quantified each variable’s contribution to the prediction of 28-day mortality. SHAP-derived variable importance is shown in Figure 2 . Statistical significance was defined as P < 0.05. All statistical analyses and modeling were performed using R (version 4.0.3) and Python (version 3.7.0).

The AMU and MIMIC-IV cohorts exhibited similar 28-day mortality rates (25.6% vs 25.1%). Across both cohorts, the 28-day mortality group was characterized by older age; higher CCI scores; increased serum creatinine (SCr), prothrombin time (PT), and procalcitonin (PCT) levels; and lower platelet counts (P < 0.001). However, analysis of pathogen distribution showed a higher prevalence of Gram-positive bacteria (22.2%, P = 0.011) and Gram-negative bacteria (12.3%, P = 0.006) in the AMU 28-day mortality group ( Table 1 ), whereas the MIMIC-IV 28-day mortality group predominantly exhibited Gram-negative bacteria (4.7%, P = 0.003) and fungal infections (3.6%, P < 0.001) ( Supplementary Table S3 ).

A total of 33 variables were included for model construction. To mitigate overfitting and underfitting and optimize performance, a grid search combined with cross-validation was used to determine the final set of hyperparameters ( Supplementary Table S4 ). In the internal training and validation sets, the XGBoost model demonstrated strong discrimination, with AUROC values of 0.92 (95% CI 0.90–0.94) and 0.85 (95% CI 0.80–0.90), respectively (P = 0.012; Figure 3A ). Further analysis showed that AUROC and accuracy increased as variables were sequentially incorporated according to their importance, but both plateaued after 10 variables, indicating that additional variables did not materially improve predictive performance ( Supplementary Figure S3 ). Therefore, a simplified 10-variable model was retrained to enhance clinical applicability.

As shown in Figure 3B and Supplementary Figure S4 , the simplified XGBoost consistently outperformed the other approaches, achieving AUROCs of 0.93 (95% CI 0.91–0.95) in the training set, 0.81 (95% CI 0.76–0.85) in the internal validation set, and 0.71 (95% CI 0.70–0.73) in the MIMIC-IV external validation set.

In the Gram-positive and Gram-negative subgroups, the simplified model achieved AUROCs of 0.91 (95% CI 0.87–0.94) and 0.90 (95% CI 0.86–0.93) in the AMU dataset, and 0.72 (95% CI 0.66–0.77) and 0.72 (95% CI 0.62–0.82) in the MIMIC-IV dataset, respectively. These values were all higher than those observed in the overall cohort.

To enable fair comparisons, we computed AUROC, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for all models at the optimal cutoff. Across datasets, the simplified XGBoost model consistently outperformed the comparator models—LR, k-nearest neighbors (KNN), random forest (RF), support vector machine (SVM), and artificial neural network (ANN) in terms of AUROC, sensitivity, specificity, and NPV ( Supplementary Table S5 ).

Decision curve analysis (DCA) showed that the simplified XGBoost model yielded stable and relatively high net benefit on internal data, particularly in the training set; on the external dataset, net benefit was lower and remained positive, and therefore clinically useful, only at low threshold probabilities ( Supplementary Figure S5A ). Overall calibration results showed that, in the training set, the simplified XGBoost model’s calibration curve was closest to the ideal line and performed best; in the internal validation set it deviated only slightly, indicating acceptable calibration. In the external validation set (MIMIC-IV), the curve lay largely below the ideal line, suggesting overall risk overestimation, although the curve was smooth and the magnitude of deviation was relatively small ( Supplementary Figure S5B ). For the comparator models (LR, KNN, RF, SVM, and ANN), adherence to the ideal line was acceptable in the low-to-intermediate predicted-probability range but worsened at higher probabilities, and systematic overestimation with reduced stability was observed in the external validation set ( Supplementary Figures S6A–E ). Overall, the simplified XGBoost model showed better probability calibration than the other models in the training and internal validation sets; on external data it still tended to overestimate risk, but the bias was smaller.

To further investigate the impact of individual variable changes on 28-day mortality in patients with ICU-BSIs, partial dependence plots (PDPs) were generated for the 10 most influential continuous variables ( Figure 4 ; Supplementary Figure S7 ). SHAP values were used to quantify each variable’s effect on the predicted probability of 28-day mortality, with positive values indicating increased risk and negative values indicating a protective effect. The top 10 variables, ranked in descending order of importance, were: duration of antibiotic use; average rate of change in platelet count; SCr; duration of invasive mechanical ventilation (IMV); average rate of change in SCr, average rate of change in white blood cell count (WBC), CCI; total bilirubin; PCT; and average rate of change in hemoglobin. Figures 4A–D shows that dynamic rates of change in laboratory parameters were closely associated with 28-day mortality in patients with ICU-BSIs. A decreasing platelet trajectory was associated with a marked increase in mortality risk, whereas stable or rising trends were linked to lower risk (A). WBC dynamics followed a nonlinear pattern: decreases were associated with relatively lower risk, but excessive increases were accompanied by higher risk (B). Changes in serum creatinine demonstrated an approximately linear positive association with mortality, indicating that worsening renal function was strongly related to poor prognosis (C). Similarly, increases in total bilirubin were associated with a gradual rise in mortality risk, whereas stable or lower levels suggested a more favorable prognosis (D).

As shown in Supplementary Figure S7 , the SHAP dependence analysis revealed interpretable threshold effects of several key clinical variables on 28-day mortality among patients with ICU-BSIs. (1) A. Antibiotic duration: Treatment lasting more than two days was associated with a significant reduction in mortality risk, suggesting a protective effect of adequate therapy. (2) B. Scr: > 110 μmol/L was linked to a marked increase in mortality risk, reflecting the adverse prognostic impact of renal dysfunction. (3) C. IMV: Mortality risk progressively increased once IMV > 24 hours, indicating that longer respiratory support was associated with worse outcomes. (4) D. CCI: > 3 were associated with a substantial rise in mortality risk, consistent with the influence of comorbidity burden on adverse outcomes. (5) E. PCT: > 0.1 ng/mL were associated with increased mortality, consistent with its established role as a biomarker of infection severity. (6) F. Hemoglobin: Stable levels corresponded to the lowest risk, whereas marked fluctuations—either increases or decreases—were associated with elevated mortality risk.

Furthermore, this study explored interactions among common clinical variables, revealing the following relationships with 28-day mortality. Supplementary Figure S8 presents SHAP-based interaction dependence plots (y-axis: SHAP value; higher values indicate a greater positive contribution to the risk of 28-day mortality). The findings were as follows: (1) S8A: A decreasing platelet change rate was associated with higher risk, and this effect was intensified when the WBC change rate further decreased; increases in the platelet change rate alone had a limited risk-reducing effect. (2) S8B: An increasing WBC change rate was associated with greater risk, with a stronger effect at higher PCT levels; risk rose sharply when WBC decreased while PCT increased, indicating a synergistically adverse interaction. (3) S8C: Fluctuations of SCr within the normal range had little impact; once SCr exceeded the normal threshold, even slightly, risk increased rapidly, and when accompanied by a positive SCr change rate the risk further escalated, indicating a threshold effect and interaction. (4) S8D: Longer duration of IMV was associated with higher risk, whereas a longer course of antibiotics was associated with lower risk, especially when IMV duration was prolonged, suggesting a mitigating effect. Taken together, these plots indicate nonlinear and interactive effects of key variables on 28-day mortality: decreased platelets combined with unfavorable WBC changes, high PCT concurrent with unfavorable WBC changes, SCr above threshold with continued increase, and prolonged IMV with insufficient antibiotic duration were each associated with higher risk, whereas adequately sustained antibiotic therapy partially offset the risk associated with prolonged IMV in some scenarios.

To evaluate the performance of individual variables and their rate-of-change transformations, Supplementary Figure S9 compares the AUROC of single variables in the two cohorts. The x-axis shows 1–specificity (FPR) and the y-axis sensitivity (TPR); the diagonal dashed line denotes the no-discrimination reference, and the colored bands indicate 95% confidence intervals. In AMU (panel A), most curves lie above the diagonal, indicating moderate discrimination, with single-variable AUROCs of approximately 0.55–0.72; rate-of-change features generally outperform their corresponding baseline-level variables. In MIMIC-IV (panel B), a similar pattern is observed, although AUROCs on the external cohort cluster around 0.57–0.64, indicating weaker discrimination. The relative ranking of variables is broadly consistent with AMU, suggesting external consistency and robustness.

Although the model demonstrates relatively high interpretability, its clinical applicability remains constrained by the inherent computational complexity of the machine learning framework adopted in this study. To address this limitation, we developed a web–based prognostic prediction system for ICU-BSIs, which has been made openly available on GitHub (https://github.com/jaser1314/simulation-software-for-zhou) to facilitate further exploration and validation by researchers. The program is designed to be executed locally, thereby avoiding the use or transmission of any patient-level data, and it supports cross-platform deployment (iOS, Windows, and Android), enhancing its usability across diverse computational environments.