This study was approved by the Institutional Ethics Committee of the First Affiliated Hospital of Chongqing Medical University (reference number: 2021–366). The ethics committee waived the requirement for informed consent because of the retrospective nature of this study. Patient data were sourced from medical record systems and analyzed anonymously to protect patient privacy.

We included a cohort of patients who had culture-positive fungi during their admission to seven ICUs (GICU, general ICU; SICU, surgical ICU; RICU, respiratory ICU; NICU, neurology ICU; NSICU, neurosurgery ICU; CSICU, cardiothoracic surgery ICU; CCU, cardiovascular ICU) at the First Affiliated Hospital of Chongqing Medical University from January 1, 2015, to December 31, 2019. Culture-positive fungi refer to specimens obtained from ICU patients that were cultured positive for fungi by laboratory physicians in the microbiology room, and an official report was issued. Subsequently, all patient data, including basic information (age, gender and comorbidities), characteristics of fungi (microbiology and time to positivity of ICU), laboratory results (all results shall be obtained within 24 h after the fungal culture is positive), and clinical data (days in the ICU, department, Acute Physiology and Chronic Health Evaluation (APACHE) II Score, diagnosis on ICU admission, and clinical characteristics), were extracted from our internal electronic medical records (Table 1). The 28-day mortality rates after ICU admission were recorded. Data were collected by three investigators and were checked by two other investigators to avoid bias. Notably, these features were chosen on the basis of their availability in all patients rather than on any a priori assumptions about their ability to predict fungal acquisition, although the goal of our prediction model was to select the most influential factors in the collected data for the prediction of ICU-AF.

According to guidelines, infection after 48 h of hospitalization is defined as HAP (11). Therefore, we included patients whose first culture was positive for fungi longer than 48 h after ICU admission in the ICU-AF cohort and less than 48 h in the non-ICU-AF cohort. The cohort process and exclusion criteria are shown in Figure 1.

All antifungal treatment decisions were jointly made by two or more deputy chief physicians with >15 years of clinical experience in critical care medicine. Among these, empirical antifungal therapy prior to fungal culture is based on guidelines (6).

ML methods are computer algorithms that automatically recognize complex patterns on the basis of empirical data. The goal is to enable algorithms to learn from past or present data and to use this knowledge to make predictions or decisions regarding unknown future events (16). In the current study, we used the random forest (RF) ML algorithm. It is a “tree-based” algorithm in which multiple decision trees are constructed using random classifications of independent features that are used to predict outcome labels for random subsets of samples (17). On the one hand, the RF technique is a regression tree technique that uses bootstrap aggregation and randomization of predictor variables to achieve a high degree of predictive accuracy and is often used in medical field analysis to construct classification prediction models (18–20). On the other hand, RF may be more suitable for feature selection during classification tasks in bioinformatics and related sciences, where it has a relatively low tendency to overfit and produces more robust results (21).

We randomly assigned 1,434 cases to the sample, with 50% of the cases used as the training set and the rest as the test set. We also ensured that there was no gender or age bias between the training set and testing set.

For the training set, we first used the Least Absolute Shrinkage and Selection Operator (LASSO) to reduce the dimension of features according to whether the patient is ICU-AF. We performed feature reduction using LASSO on the training set. LASSO performs feature selection during model construction by penalizing the respective regression coefficients. As this penalty increases, more regression coefficients shrink to zero, thus resulting in a more regularized model (22). In this process, 49 significant features with nonzero coefficients were obtained. We then used them in the RF prediction model. By using a ten-fold cross-validation analysis, we selected the best model parameter on the basis of the accuracy of each fold of the model. At the same time, we ranked the features in this model by setting a threshold of 0.05 to select the features in reference to previous articles (23, 24). These features were retained, and the randomized forest model was trained to predict patient ICU-AF by using ten-fold cross-validation. Finally, the model was tested using the test set. Both downscaling and ten-fold cross-validation were used to prevent overfitting. Overfitting can occur when excessive features affect the predictive performance of a model. However, the use of nested k-fold cross-validation allows us to perform model training independently of hyperparameters optimization, which prevents overfitting or incorrect generalization estimates (25). The R language was used to draw the density map between each feature and APACHE II, and the lm function was used to fit the regression model. The “pheatmap” package implements heatmap to display sample survival and feature performance.

In the ten-fold cross-validation of the model, we trained different model parameters and selected the model parameters with the best accuracy in one fold for application to the test set. The ability of the model to discriminate between acquired fungi was determined using the area under the curve (AUC), and the stability of the model was determined on the basis of sensitivity and specificity. From our learning models, we chose the model with the best discrimination ability.

All statistical analyses were performed using Stata 24.0 software. To divide the training and test sets, we used analysis of variance (ANOVA) to analyze whether there was a difference in the age distribution between the training and test sets, and the chi-square test was used to analyze whether there was a difference in the gender distribution between them. The main specification of ML is that the models constructed from selected features perform well for predicting patient outcomes, AUC, sensitivity, specificity, and accuracy and are only used to determine the performance of the models (26). Therefore, many previous studies have used ML and logic methods (27, 28). In our research, factors associated with antifungal therapy and empirical antifungal therapy for acquired fungi were analyzed using univariate and multivariate conditional logistic regression models for all features of the ML model. Its odds ratio (OR) and 95% confidence interval (CI), p < 0.05 was considered significant.

For the submission of the manuscript, we enrolled 2,147 cases. A total of 1,434 cases with complete data were obtained after exclusion and screening (Figure 1, step 1). The cases were randomly and equally divided into the training (N = 717) and test (n = 717) sets. The distribution of outcome labels for patients in the training and test sets showed no significant differences (p = 0.37, chi-square test, not shown). The features of the two data sets are shown in Table 1; age (ANOVA, p = 0.60, not shown) and gender (chi-square test, p = 0.15, not shown) had no statistical difference, and the other features are shown in Table 1. On the basis of whether the fungi were ICU acquired, LASSO was performed to reduce the dimension of features. Thereafter, by using ten-fold cross-validation, the average accuracy of the random forest model was 0.907 ± 0.042, among which the third-fold accuracy we applied was the highest, which was 0.972 (Figure 2). We took the third-fold model parameter as our optimal model parameter. We selected six features with importance ≥0.05 in the optimal mode, namely, times of arterial catheter, times of enteral nutrition, times of corticosteroids, times of broad-spectrum antibiotics, times of urinary catheter and times of invasive mechanical ventilation (Figure 3A).

By using these features for ML analysis and testing on an independent test set, the results showed that the AUC for predicting ICU-AF was 0.981 in the test set, with a sensitivity of 0.960 and specificity of 0.990 (Figure 3B). Disease severity in ICU patients was represented by the APACHE II Score, which was analyzed separately with the continuous time of these six features. Only the times of invasive mechanical ventilation showed a significant linear correlation with the APACHE II Score (p = 0.031) (Figure 3C). The duration time of these features showed no significant differences in the 28-day mortality (Figure 3D).

Considering the univariate and multivariate conditional logistic regression analyses of antifungal therapy in ICU-AF, the results showed that among these six features, times of arterial catheter (p = 0.011, OR = 1.057, 95%CI = 1.053–1.104) and times of invasive mechanical ventilation (p = 0.007, OR = 1.056, 95%CI = 1.015–1.098) were independent risk factors for antifungal therapy in ICU-AF (Table 2). In the sample on antifungal therapy, times of arterial catheter (p = 0.004, OR = 1.098, 95%CI = 0.855–0.970) was an independent risk factor for empirical antifungal therapy (Table 3).

This study applied an unconventional method to study susceptibility to ICU-AF: First, we used efficient ML methods to analyze clinical data to reduce the bias of manual analysis. Second, we focused on the early warning of fungal infection, namely, ICU-AF, and this approach is more in line with the needs of treating ICU patients. Finally, we investigated the role of the ICU-AF early warning model in antifungal therapy and empirical antifungal therapy for guiding the management of ICU-AF.

This study has the following limitations. First, this was a retrospective, single-center study. It should be noted that this single-center study involved seven different ICU wards (GICU, RICU, SICU, NICU, NSICU, CSICU, and CCU), and some specific characteristics (such as major abdominal surgery and disturbance of consciousness) were diverse. However, even across seven different ICUs, each patient had these six features. As a routine treatment procedure for patients in the ICU, the duration of these six features was obtained via detailed nursing records and reflected the length of time that patients received treatment in the real world and the homogeneity of fungal infection risk factors across all ICUs. Second, there were more than 40 salient features in the optimal model (Figure 3A). However, we selected only six features with importance >0.05. When multiple features appear in the results, it is crucial to extract better and more convenient feature models for clinical applications. In addition to using 0.05 as a threshold to screen six features, we also explored the important role of these six features in ICU-AF on the basis of clinical practice. Other features (such as central venous catheter, abdominal surgery and SOFA scores) that were reported to be connected with ICU-AF (43, 44), could probably have hidden roles. They still have potential value for future discussion. The prevailing view supports that the six features, analyzed in the current study are good predictors of ICU-AF (7, 37–42). Controlling these six operations is an effective way to reduce ICU-AF. Blaize et al. found that controlling the use of corticosteroids could reduce the risk of invasive pulmonary fungal infections in COVID-19 patients admitted to the ICU (45). Thirdly, regarding the question of whether ICU physicians can distinguish fungal colonization from fungal infection. The AUC of the optimal model for the fungal infection test obtained in this study was 0.670 (Supplementary Figure S1). In the clinical cohort, these were indistinguishable at the time of diagnosis; thus, we advanced the field of view to the acquired fungus. Finally, increasing the amount of training data can enable us to obtain more information and make diverse learning in most cases, as well as increase the chances of achieving better results. Some important studies use 70% or 80% of samples in the training set (46–48). We randomly assigned 1,434 cases to the sample, with 50% of the cases used as the training set and the rest as the test set, to improve the efficiency of model validation. Meanwhile, it was also ensured that there was no gender and age bias between the training set and the test set. Although this ratio is also a common ratio for dividing datasets in previous studies, such as in some studies on tumor diseases (49, 50), we will continue to collect and expand sample size data in future research to improve the sample ratio in the training set.

In summary, ML classifier models in clinical cohorts have the potential to predict the risk of ICU-AFI. The most important risk factors for ICU-AF are the six time-related clinical parameters (arterial catheter, enteral nutrition, corticosteroids, broad-spectrum antibiotics, urinary catheter, and invasive mechanical ventilation) that provide early warnings for the early prevention of fungal infection. Furthermore, this model, although needs to be more clinically validated, has the potential to help ICU physicians assess whether empiric antifungal therapy should be administered to ICU patients who are susceptible to fungal infections.