We conducted a multicenter retrospective prognostic study of inpatients with AD cirrhosis at six tertiary hospitals in Chongqing, China, including the Second Affiliated Hospital of Chongqing Medical University, Yong Chuan Hospital of Chongqing Medical University, Third Affiliated Hospital of Chongqing Medical University, University-Town Hospital of Chongqing Medical University, People's Hospital of Tong Liang District, and Southeast Hospital of Chongqing. We followed the transparent reporting of a multivariable predictive model for individual prognosis or diagnosis (TRIPOD) guidelines for model development and validation (22). Clinical data were collected using electronic medical record systems. Consecutive patients with AD cirrhosis admitted to the above hospitals from September 2012 to December 2016 were enrolled as the derivation cohort, and those with the same clinical characteristics hospitalized between January 2017 and April 2020 were enrolled as the temporal validation cohort. To determine whether patients admitted in April 2020 were readmitted to the hospital, we conducted follow-up until August 2020. The end points were cirrhosis-related readmissions within 90 days from the date of hospital discharge. In cases of multiple admissions, only the first readmission was considered.

The study protocol was reviewed and approved by the ethics committee of the First Affiliated Hospital of Chongqing Medical University. Due to its retrospective nature, this study required no conformed consent.

The inclusion criteria were as follows: (1) patients aged ≥ 18 years and (2) hospital admission for AD cirrhosis.

The exclusion criteria were as follows: (1) patients with acute-on-chronic liver failure (ACLF): diagnosis of ACLF was based on the criteria from the consensus recommendation of the Asian Pacific Association for the Study of the Liver (23); (2) liver cancer or other active malignancies; (3) evidence of congestive heart failure, chronic kidney disease, or other significant chronic extrahepatic diseases; (4) hospital stay ≤ 1 day; (5) endoscopic ligation of esophageal varices or transjugular intrahepatic portosystemic shunt (TIPS) in the initial hospitalization or elective hospital admission; (6) discharge against medical advice; (7) patients lost to follow-up or death during index hospitalization; and (8) patients with >30% of data missing. Details of readmission in other hospitals or planned procedures, surgery, and therapy were compiled from medical history.

AD cirrhosis was defined as the rapid development of one or more major complications of liver disease, such as ascites, encephalopathy, gastrointestinal hemorrhage, bacterial infection, or a combination of these factors, requiring hospitalization (1, 24–27).

Ascites was recorded as the primary reason for admission if this was the sole criterion for admission, and infection was absent. Hepatic encephalopathy was characterized by altered mental status or neuropsychiatric abnormalities in the presence of liver cirrhosis after exclusion of other causes. Gastrointestinal (GI) bleeding was defined as the development of an upper and/or lower gastrointestinal hemorrhage of any etiology (27). Bacterial infection was defined in cases of spontaneous bacterial peritonitis, pneumonia, cellulitis, biliary tract infection, urinary system infection, and spontaneous bacteremia (26).

In the presence of more than one contributory factor, the main cause of admission was defined as follows: (1) in patients admitted with GI bleeding in the presence of ascites, bacterial infection, or hepatic encephalopathy, GI bleeding was considered the main cause; (2) in the absence of bleeding at admission, bacterial infection was the main cause of hospitalization; and (3) in patients with hepatic encephalopathy and ascites, the main cause was the former (11). The principal cause of hospitalization was subsequently assessed independently by two subspecialist physicians.

Medical therapies were used for all patients during hospitalization and after discharge, such as antiviral therapy, diuretics, lactulose, non-selective beta-blockers, antibiotics, symptomatic and supportive therapies. Prophylactic antibiotics were not routinely administered after discharge.

Demographic, etiological, clinical, and laboratory data were recorded within 24 h of the first hospital visit. Demographic characteristics included age and sex. The etiological characteristics, including hepatitis B virus (HBV)/hepatitis C virus (HCV) infection, autoimmunity, and alcohol consumption, were assessed from medical history. Clinical data included length of hospital stay, complications related to liver cirrhosis, comorbidities, smoking history, alcohol consumption, and family history of liver disease. Laboratory analyses included liver function test, routine blood test, creatinine, blood urea nitrogen, serum sodium, serum potassium, and international normalized ratio (INR). End-stage Liver Disease score (MELDs), Child–Turcotte–Pugh score (CTPs), chronic liver failure-consortium acute decompensation scores (CLIF-C ADs), and MELD-Na score (MELD-Nas) were calculated at admission according to previously published criteria (28–31).

Continuous variables were expressed as mean ± standard deviation (SD) or median (interquartile range), according to the distribution of normality. Categorical variables were reported as counts with percentages. Group comparisons of continuous variables were analyzed with the Mann–Whitney U test and categorical variables with χ² or Fisher's exact test. For variables with omission rates <30%, multiple imputation was used.

To avoid overfitting, we performed two steps of variable selection. First, we evaluated the association between readmission within 90 days and a set of potential predictors by using univariate analyses (Mann–Whitney U test for quantitative predictors and χ² or Fisher's exact test for binary predictors). Predictors with p < 0.05 were subsequently considered in an automated variable selection procedure within the least absolute shrinkage and selection operator (LASSO) framework to select the best predictor subset (32). The complexity of LASSO regression was controlled by a tuning parameter lambda (λ) with the rule that the penalty for each variable coefficient increases with λ value, and the relevant features with non-zero coefficients were selected that contributed to the final LASSO regression (33). The number of variables involved in the final model was considered based on the optimal λ value to balance the accuracy and simplicity of the model. Then, the retained variables were used to construct the nomogram using multivariate Cox regression. The nomogram was based on the fitted predictive model using R version 4.0.2 with the rms package (34). To streamline the power calculation estimation, we produced PowerTools, an interactive open-source web application, written in R code by using the Shiny framework (http://www.shinyapps.io/).

The discriminatory value of the models was assessed based on the concordance index (C-index). The area under the curve (AUC) of the receiver operating characteristic (ROC) curve was also used to evaluate the prognostic accuracy of the nomogram. To demonstrate the stability of the model, we applied bootstraps with 200 resamples to correct the C-index to overcome overfitting. Calibration curves were additionally drawn to evaluate the concordance between the predicted and observed probabilities. The nomogram model was validated with a temporal validation cohort using the same process of capability assessment. Decision curve analyses (DCA) were applied to compare the benefits and improved performance of different models (35).

According to the nomogram score, the patients were classified into two groups representing low and high risk. The optimal cut-off values for the total points of the nomogram were determined by maximizing the Youden index (sensitivity + specificity – 1). Kaplan–Meier curves were constructed for stratified risk groups, and log-rank tests were used to determine significant differences between the curves.

All tests were two-sided, and data were considered statistically significant at p < 0.05. Statistical analyses were performed using R software (version 4.0.2, Vienna, Austria).

A total of 8,402 patients met the inclusion criteria. Following the application of the exclusion criteria, 956 patients were finally included in the study, specifically 705 patients in the derivation cohort (from September 2012 to December 2016) and 251 patients in the temporal validation cohort (from January 2017 to April 2020). The study selection process is depicted as a flow chart (see Supplementary Figure 1 for details).

The mean (SD) age of all patients was 58.8 (12.6) years, and 68.31% were male. The etiologies of cirrhosis were chronic hepatitis B (50.4%), alcoholic (10.7%), autoimmune liver disease (9.9%), chronic hepatitis C (5.0%), and other/cryptogenic factors (19.6%). The overall readmission rates for patients at 30, 60, and 90 days were 24.6, 43.0, and 51.8%, respectively. As shown in Table 1, bacterial infection was the main reason for index admission (54.3%), followed by GI bleeding (35.0%), hepatic encephalopathy (5.8%), and ascites (4.9%). Regarding the main reason for readmission, bacterial infection was the most common cause, followed by GI bleeding, other cirrhosis-related diseases, hepatic encephalopathy, and ascites at the 30-, 60-, and 90-day time points.

Based on the baseline characteristics of the two cohorts of patients as listed in Supplementary Table 1, patients in the derivation set were older and had lower neutrophil percentages and blood urea nitrogen levels; higher total protein, total bilirubin (TB), direct bilirubin, hemoglobin, aspartate aminotransferase, and alkaline phosphatase levels; lower rates of gastrointestinal hemorrhage; higher rates of bacterial infection; higher lengths of stay at initial admission; and higher CTPs and CLIF-C ADs (p < 0.05). The 30-, 60-, and 90-day risk of readmission were higher for the temporal validation set than for the derivation set (p < 0.05). The remaining clinical and laboratory parameters at initial admission as well as MELDs and MELD-Nas were similar between the derivation and temporal validation sets.

Supplementary Table 2 provides the results of the univariate analyses for all 36 factors considered as potential predictors in our scoring system. Fifteen candidate predictor variables with p < 0.05 were used as the input data in the LASSO regression. When the lambda value was collected as 1 standard error [log (λ1se) = −2.10], four variables were selected (see Supplementary Figure 2 for details). Then, the four retained variables were used to construct the nomogram using multivariate Cox regression. As shown in Table 2, GI bleeding, serum sodium, total bilirubin (TB), and INR composed a panel of significant predictors of readmission in patients with AD cirrhosis.

A nomogram was constructed based on the four aforementioned independent prognosticators (Figure 1). The values of each risk factor were assigned a score on the point scale axis. By adding each single score and using that value in the total point scale axis, the total score could be easily calculated to assign the probability of readmission for individual patients at 30, 60, and 90 days. For the convenience of clinicians, we have provided the nomogram as a web-based calculator tool (https://cqykdx1111.shinyapps.io/dynnomapp/). Doctors can enter the indicators for each patient to automatically calculate the patient's probability of readmission within 90 days.

The model exhibited good discrimination ability. The C-index values of the nomogram for 30-, 60-, and 90-day readmission were 0.770 [95% confidence interval (CI): 0.728–0.812], 0.754 (95% CI: 0.716–0.791), and 0.731 (95% CI: 0.693–0.768), respectively. Furthermore, the predictive performance of nomogram was calculated by AUC of ROC curve. The AUC values were 0.775 (95% CI: 0.733–0.817), 0.753 (95% CI: 0.715–0.791), and 0.733 (95% CI: 0.695–0.770) for 30-, 60-, and 90-day, respectively (Figure 2A). Bootstraps with 200 resamples were performed to correct the predictive model. The adjusted C-index values of the nomogram for 30-, 60-, and 90-day readmission were 0.770, 0.750, and 0.732, respectively.

The calibration curves showed good agreement between nomogram predictions and observed probabilities for 30-, 60-, and 90-day outcomes in the derivation cohort (Figure 3A).

The validation set was estimated using the established nomogram, and the C-index values obtained were 0.703 (95% CI 0.638–0.768), 0.694 (95% CI: 0.627–0.762), and 0.707 (95% CI: 0.636–0.777), respectively. The AUC values were 0.714 (95% CI: 0.649–0.778), 0.682 (95% CI: 0.614–0.751), and 0.712 (95% CI: 0.644–0.780), respectively (Figure 2B), supporting the suitability of the nomogram for estimating 30-, 60-, and 90-day readmission.

The calibration curves revealed good agreement between the nomogram predictions and observed probabilities for 30-, 60-, and 90-day outcomes in the temporal validation cohort (Figure 3B).

Predictive power for readmission was compared for the nomogram, MELDs, CLIF-C ADs, CTPs, and MELD-Nas based on C-indexes. DCA was performed to determine the clinical utility of the nomogram by calculating the net benefits at different threshold probabilities. The C-indexes for 30-, 60-, and 90-day readmission predicted with MELDs, CLIF-C ADs, CTPs, and MELD-Nas were significantly lower than those with the nomogram, in both the derivation and temporal validation cohorts (Table 3).

Using DCA, our nomogram provided superior net benefit and displayed improved performance in prognostic evaluation over the 30-, 60-, and 90-day periods, both in the derivation (Figures 4A–C) and validation (Figures 4D–F) cohorts, compared to MELDs, CLIF-C ADs, CTPs, and MELD-Nas models.

When patients were stratified according to the optimal cut-off value by the total nomogram points (high risk: >56.8 and low risk: ≤ 56.8), each group represented a distinct prognosis. The high-risk group was more likely to have readmission than the low-risk group, with statistical significance in both the derivation cohort and temporal validation cohort (p < 0.0001, Figures 5A,B).