The development cohort included 1148 patients who were recruited from Medical Information Mart for Intensive Care IV (MIMIC IV version 1.0) (14, 15). MIMIC IV is a relational database containing the real information of patients admitted to the ICUs of Beth Israel Deaconess Medical Center in Boston, MA, USA, from 2008 to 2019. The principal investigator completed the Human Research Course (Record ID: 37097306) and obtained access to this database, and the project was approved by the institutional review boards of the Computational Physiology Laboratory of the Massachusetts Institute of Technology and Beth Israel Deaconess Medical Center and was granted a waiver of informed consent. All data were extracted with structured query language (SQL) from BigQuery.

The validation cohort included 121 patients treated in the Department of Intensive Care Unit, Fourth Hospital of Hebei Medical University, Shijiazhuang, China. This study was approved by the Ethics Committee of the Fourth Hospital of Hebei Medical University (approval number: 2021KS034).

The inclusion criteria in this study were as follows: (1) AKI patients meeting the KDIGO-AKI criteria; and (2) patients who received CRRT after diagnosis. Patients younger than 18 years were excluded, and when the same patient were admitted multiple times, only data for the first admission was included. In addition, in the validation cohort, patients whose family members voluntarily stopped treatment within 24 h after receiving CRRT were also excluded.

AKI was defined as any of the following Kidney Disease Improving Global Outcomes (KDIGO) criteria (16): increase in SCr ≥0.3mg/dl (≥26.5mol/l) within 48 h; increase in SCr ≥1.5 times baseline, which is known or presumed to have occurred within the prior 7 days; or a urine volume < 0.5ml/kg/h for 6 h.

The primary outcome was defined as death within 28 days after receiving CRRT. Patients in the validation cohort whose family members voluntarily stopped treatment for more than 24 h were considered dead.

The following variables were extracted from the relevant literature and clinical records:

Demographic characteristics: Age (17–21), sex (20, 21), height, and weight (21).

Comorbidities: Congestive heart failure (CHF) (18), atrial fibrillation (AF), chronic liver disease (CLD), chronic obstructive pulmonary disease (COPD), chronic coronary syndrome (CCS) (18), hypertension, diabetes, and malignant cancer (18, 19).

Last vital signs within 2 h prior to receiving CRRT: Heart rate (HR) (18), mean arterial pressure (MAP) (18, 21), and temperature (T).

Results of the last laboratory test within 24 h prior to receiving CRRT: White blood cell count (WBC), hemoglobin (HB) (17, 20), red cell volume distribution width (RDW) (22), platelet count (PLT) (18, 20, 21), sodium (20), potassium (20), calcium, phosphate (18, 23, 24), total bilirubin (TBIL) (18, 20, 21), albumin (18, 20, 21), creatinine (18, 21), baseline creatinine (20, 21), pH (17, 20), oxygenation index (21), base excess (20), and lactate (20, 21). Oxygenation index is calculated by equation PaO₂/FiO₂.

Interventions 24 h prior to receiving CRRT: Mechanical ventilation (18, 20, 21), vasopressor use (20, 21), sedative use, and analgesic use.

Central venous pressure (CVP) (missing rate: 74.7%), mean platelet volume (25) (missing rate: 100%), troponin (missing rate: 73.9%), N-terminal pro B type natriuretic peptide (NT-proBNP) (missing rate: 97.4%), and creatine kinase (missing rate: 70.2%) were not extracted due to excessive amounts of missing data (missing rate >50%), and there appears to be no evidence of their relationship with prognosis in this group of patients.

In the development cohort, there were missing data for most variables. Variables with excessive amounts of missing data were excluded. We assumed that the data were missing at random and filled in missing data using multiple imputation with chained equations. We performed fifty multiple imputations and merged the dataset into the development dataset. All analyses were performed with R software (version 4.1.1; R Foundation for Statistical Computing).

We used a Q-Q plot to assess the normality of the continuous variables, and cubic spline functions were used to assess the linearity of the relationship. Continuous variables that did not conform to normal or linear distributions were converted to categorical covariates based on their clinical significance. The continuous variables are expressed as the mean (standard deviation), and the categorical covariates are reported as numbers and percentages.

All variables were included in the logistic regression model, and we added an interaction term between mechanical ventilation and oxygenation index. The variables were screened using an all-subset regression, with the best model judged by adjusting the r-squared and Bayesian information criterion (BIC). The screened models were tested for multicollinearity by calculating the variance inflation factor (VIF).

Finally, we used the best model to construct a nomogram that could provide clinicians with an intuitive and quantitative tool for predicting the outcomes of AKI patients undergoing CRRT.

The model discrimination was evaluated with the C-index and area under the receiver operator characteristic curve (AUC). The model calibration was evaluated with Brier scores and calibration plots. Decision curve analysis (DCA) curves were used to assess the clinical applicability of the model (26, 27).

Internal validation was performed with the enhanced bootstrap technique, in which regression models were fitted in 1,000 bootstrap replicates, drawn with replacement from the development cohort. The model was refitted in each bootstrap replicate and tested using the original sample to estimate optimism in the model performance. External validation was performed with the validation cohort.

In total 1,148 patients from the MIMIC IV database were eventually included in our study (Figure 1). The 50 datasets obtained by multiple imputation techniques were merged into the final development cohort (Table 1). The best models were screened by adjusting the r-squared value and BIC (Figure 2).

The VIFs of the screened variables were all <5. Seven variables (age, vasopressor use, RDW, lactate, WBC, PLT, and phosphate) were finally included in our model, which was used to plot the nomogram (Figure 3) and make the web-based prognostic calculator (Figure 4, https://libo220284.shinyapps.io/DynNomapp/).

The predictive performance of our model as measured by the C-index was 0.812 (Table 2 and Figure 5A) in the development cohort, indicating that the model had relatively good discriminative capacity. Our model showed high agreement between the actual and predicted probabilities in the development cohort, with a Brier score of 0.173 (Table 2 and Figure 5B). In addition, the DCA curve demonstrated that our model was clinically useful in the development cohort (Figures 5C,D).

Our model also achieved good internal validation performance after 1,000 bootstrap replicates, with a C-index of 0.811 and a Brier score of 0.173 (Table 2).

In total 121 patients were eventually included in the external validation cohort (Table 3 and Figure 1). The predictive performance of the nomogram as measured by the C-index was 0.768 (Table 2 and Figure 6A) in the external validation cohort, indicating that the model had relatively good discriminative capacity and generalizability in different settings. The nomogram also showed acceptable agreement between the actual and predicted probabilities in the external validation cohort, with a Brier score of 0.202 (Table 2 and Figure 6B). In addition, the DCA curve demonstrated that our model was clinically useful in different settings (Figures 6C,D).

Several prediction models of the outcome of AKI patients with CRRT have been developed, although their clinical use is rare.

Kim et al. (12) developed the MOSAIC model for patients with AKI undergoing CRRT. Unfortunately, this model only incorporated APACHE II outcomes and SOFA scores, and although these data were extremely accessible, they did not consider several other indicators that have predictive value and are readily available. A study by Oh et al. (22) showed that RDW was an independent predictor of the 28-day mortality in patients with AKI receiving CRRT. Phosphate reflected disease severity and predicted mortality in AKI patients undergoing CRRT in the studies by Jung et al. (23, 24). Both RDW and phosphate were included in our study. In addition, we considered additional comorbidities and laboratory indicators.

Machine learning algorithms have also been applied to predict outcomes in AKI patients undergoing CRRT (13). Machine learning algorithms appear to provide better predictive performance than traditional models, but their hard-to-interpret nature may also lead to overestimation of model accuracy and exaggeration of actual performance (40). We chose the more robust logistic regression model in our study. Our model did not perform worse than machine learning algorithms.

The HELENICC score is an excellent model for predicting mortality in patients with sepsis-related AKI undergoing CRRT (41), but not all patients with AKI undergoing CRRT have sepsis, and we hope that our model will be useful for clinical decision making in a larger number of patients with AKI.

The greatest advantage of this study over previous studies is that the external validation was based on completely independent data, and good model performance was achieved. This finding demonstrates the good generalizability of our model.

As statistician Professor Efron stated, in the absence of genius-level insight, statistical estimation theory is intended as an instrument for peering through noisy data and discerning a smooth underlying truth (42). Our models are not solely designed to predict patient outcomes but to offer new possibilities for clinicians and patient families to participate in shared decision-making regarding patient care.

We were able to quickly assess the risk of patient death with the nomogram and web-based prognostic calculator in this study, but some challenges exist.

On the one hand, although ICU physicians readily accept data-driven advice in their interactions with smart devices and the Internet, they remain cautious regarding the advice such technology provides in clinical decision-making (43). Even when models conclude that some AKI patients will not be able to reverse their deterioration even with CRRT, ICU physicians still prefer to treat them to the fullest. Physicians are always concerned that they are doing too little, and sometimes they are willing to do more than resuscitation interventions knowing that a treatment does not fundamentally change the patient's outcome (44). Using such technologies in clinical work must provide actionable information for the right patient at the right time. For example, outcomes can be predictive information to help clinicians make clinical decisions with some basis of reference. In addition, many factors that influence clinical decisions, including clinical, social, and personal factors, are not necessarily reflected in the digital record, thus any predictive results need to be evaluated, interpreted, and fleshed out by the clinician before any action is taken. Therefore, it is still the clinician who makes the final decision. Of course, this also requires critical care physicians to have some ability to interpret and use these results (43).

On the other hand, no medical practice is immune to ethical considerations, and the application of these technologies to the management of critically ill patients is fundamentally a medical practice for patients. This also requires compliance with medical ethical requirements.

It is important to emphasize that the inappropriate use of these technologies can cause harm to patients (45). Therefore, we must be cautious and ensure that it can be reasonably and safely tested and used in critically ill patients (46).

There are potential limitations in our study.

First, missing data are unavoidable in retrospective studies. Rather than excluding all patients with missing data from the analysis, we used multiple imputation to reduce the impact of missing data. With theoretical and empirical evidence of the technique's superiority to traditional complete case analysis, multiple interpolation has become widely accepted and is increasingly used (47, 48).

Second, because our development cohort was derived from the MIMIC-IV database, variables with significant predictive value that are easily accessible, such as the mean platelet volume and some widely reported biomarkers, were not included in our study. Han et al. (25) showed that the mean platelet volume may be an inexpensive and useful predictor of the 28-day all-cause mortality in AKI patients requiring CRRT. The predictive value of biomarkers such as tissue inhibitor metalloproteinase-2 (TIMP-2), insulin-like growth factor-binding protein 7 (IGFBP7) and neutrophil gelatinase-associated lipocalin (NGAL) has also been widely reported (49, 50). Unfortunately, these variables were not available in the MIMIC-IV database. These variables may need to be considered in future model updates.

Finally, our model seems to underestimate the mortality rate of patients. However, the performance during model development, internal validation, and external validation was in the acceptable range. Importantly, our validation cohort was completely independent of the development cohort in both time and space.