This single-center retrospective study included consecutive patients with ACS who underwent coronary angiography at Hamamatsu Medical Center from March 2017 to January 2021. The exclusion criteria were cases with missing data for each risk calculation tool or the absence of severe stenosis or thrombus in the epicardial coronary artery on coronary angiography.

This study adhered to the principles of the Declaration of Helsinki and was approved by the Medical Ethics Committee of Hamamatsu Medical Center (2023-3-037). As this was a retrospective study, obtaining informed consent from each patient was not required. Instead, in accordance with our institution's routine ethical regulations, we posted a notice on the study design and contact information in a public location within our institution. In this public notification, we ensured that patients had the opportunity to opt out of participating in this study.

The endpoint of this study was all-cause mortality. We evaluated the occurrence of endpoints at 1, 6, 12, 24, and 36 months after coronary angiography.

All participants were scored using three widely used risk calculation tools: ACTION, CRUSADE, and GRACE. The ACTION risk score (range: 0–103) includes age, systolic blood pressure, creatinine clearance, cardiac arrest, shock, heart failure, heart rate, ST-segment elevation myocardial infarction (STEMI), and troponin ratio (11). The CRUSADE risk score (range: 0–142) includes age, serum creatinine level, systolic blood pressure, heart failure, heart rate, weight, prior heart failure, hematocrit level, troponin ratio, prior stroke, diabetes mellitus, sex, and history of peripheral artery disease (16). The GRACE risk score (range: 1–372) includes Killip class, systolic blood pressure, heart rate, age, serum creatinine level, cardiac arrest, ST-segment deviation, and elevated cardiac enzyme level (15).

Myocardial infarction (MI) is associated with cardiac troponin release and is based on the third universal definition of MI (29). STEMI is defined as MI with ST-segment elevation ≥1 mm in at least two contiguous leads or a new or undetermined duration of left branch bundle block on electrocardiogram. Non-STEMI (NSTEMI) is defined as MI without the aforementioned electrocardiographic changes. Unstable angina is defined as myocardial ischemia at rest or with minimal exertion in the absence of acute cardiomyocyte injury or necrosis. Shock is defined as 90 mmHg systolic blood pressure or less at admission. Heart failure is defined as Killip class II, III, or IV on admission. The troponin ratio is calculated as the baseline troponin value divided by the local laboratory-specific upper limit of normal. The data required for the risk calculation tool were defined with reference to each study (11, 15, 16). Upon admission, all participants were subjected to medical history taking, physical examination, blood sampling, chest radiography, and electrical cardiography. Patients with STEMI underwent emergent coronary angiography, and those with non-ST-segment elevation ACS underwent emergent or elective coronary angiography, depending on the situation at admission. The decision regarding the therapeutic strategy was left to the attending physician's discretion, depending on the severity of the case. We collected data on the endpoints using information from the electronic medical records of our hospital and requests for written medical information from referral medical institutions but not by telephone contact with the patients directly. All extracted data were anonymized.

The continuous variables were expressed as mean ± standard deviation or median values with the associated range. The categorical variables were expressed as percentages. A comparison of baseline characteristics between the death and non-death groups was performed using the t-test and χ² test for continuous and categorical variables, respectively. The Kaplan–Meier curve represented the occurrence of events after coronary angiography. Time-dependent ROC analysis was performed to evaluate the predictive performance of risk scores. Individual disease outcomes were observed and updated in the time-dependent ROC curve analysis at each time point. This curve was created using the sensitivity (t) and 1-specificity (t) obtained from various cutoff values of the independent variables at time t. Using the measured values, a time-dependent ROC curve could be drawn at any time t. The predictive ability of independent variables can be accurately evaluated by constructing ROC curves at several time points. Therefore, time-dependent ROC curve analysis is an efficient statistical method for precisely evaluating the outcomes of independent variables (28). P < 0.05 was considered statistically significant. All statistical analyses were performed using EZR (Saitama Medical Center, Jichi Medical University, Saitama, Japan), a graphical user interface for R (R Foundation for Statistical Computing, Vienna, Austria) (30). Specifically, it is a modified version of the R commander designed to add statistical functions frequently used in biostatistics. We used the Kaplan–Meier “survival ROC” package, written in R, to assess time-dependent changes in prediction performance between different risk calculation tools (28). Furthermore, we performed additional analysis using the “timeROC” package in R to evaluate the 95% confidence intervals of the area under the time-dependent ROC curves and the statistical difference between the area under the time-dependent ROC curves of risk scores (31).

The baseline characteristics of the study population are listed in Tables 1, 2. Significant differences between most variables contributed to risk scores in the death and non-death groups. Among these variables, sex, heart rate, diagnosis of STEMI, and troponin ratio were not significantly different between the death and non-death groups.

In total, 66 patients died during the 36-month follow-up period. The median follow-up period for the overall study was 1,080 days (interquartile range: 1,022–1,080). The cumulative incidence rates of all-cause mortality at 1, 6, 12, 24, and 36 months were 6.7%, 8.7%, 9.9%, 13.3%, and 16.9%, respectively (Figure 2).

Figure 3 illustrates the ROC curves of each risk score for all-cause mortality at 1, 6, 12, 24, and 36 months after the start of follow-up, using time-dependent ROC analysis. Table 3 compares the areas under the curve (AUCs) of risk scores for all-cause mortality. The AUCs of the ACTION risk score at 1, 6, 12, 24, and 36 months were 0.942, 0.925, 0.889, 0.856, and 0.832, respectively; those of the CRUSADE risk score were 0.881, 0.883, 0.862, 0.876, and 0.862, respectively; and those of the GRACE risk score were 0.949, 0.928, 0.888, 0.875, and 0.860, respectively.

Figure 4 depicts the AUC of each risk score for all-cause mortality and cardiac death plotted monthly using time-dependent ROC analysis. The AUCs of the ACTION and GRACE risk scores for all-cause mortality were higher than those of the CRUSADE score, especially in the short-term, but decreased over time. In contrast, the AUC of the CRUSADE score remained relatively constant during the observation period.

This study evaluated the AUCs of the ACTION, CRUSADE, and GRACE risk scores for all-cause mortality using time-dependent ROC analysis for 36 months. The AUCs of each risk score for mortality in patients with ACS remained relatively high during the observation period. The AUCs of the ACTION and GRACE scores were significantly higher than that of the CRUSADE score in the short term but decreased over time. Conversely, the AUC of the CRUSADE score remained relatively constant.

The AUCs of the ACTION and GRACE risk scores were higher than those of the CRUSADE risk score, especially in the short term. However, the AUCs of the ACTION and GRACE risk scores decreased over time. Nevertheless, the AUC of the CRUSADE score remained relatively constant. Among the data used to calculate risk scores, only the CRUSADE risk score included sex, weight, diabetes mellitus, prior heart failure, prior stroke, and prior peripheral artery disease. The association between diabetes and arteriosclerotic disease is widely known; however, diabetes has also been reported to be associated with cancer, which greatly affects long-term prognosis (32–34). Furthermore, diabetes affects bone metabolism, and fractures in patients with diabetes worsen prognosis (35, 36). Other data for calculating the CRUSADE risk score also affect prognosis (37–42). Systolic pressure; cardiac enzyme levels; and the presence of heart failure, cardiac arrest, and cardiac shock, which are included in the non-CRUSADE risk scores or every risk score, are strongly associated with ACS severity. The mortality rate of ACS is highest during the acute phase (43). Therefore, the AUCs of the ACTION and GRACE risk scores, which have a larger proportion of data strongly associated with ACS severity, might be higher than that of the CRUSADE risk score in the short term and may decrease over time. In contrast, the AUCs of the CRUSADE risk score might have remained constant during the observation period because the CRUSADE risk score includes several data that affect prognosis, regardless of ACS severity.

This study used the CRUSADE risk score as the long-term risk calculation tool. However, other tools that target long-term prognosis, such as the ACEF, LASSO, and EPICOR scores, are also available (10, 12, 13). Because of the retrospective design of this study, accurate quantitative assessment of LVEF using an ultrasound cardiogram, the use of a diuretic agent, and the timing of the diuretic agent during hospitalization, which are included in these risk scores, were unclear. The left ventricular systolic function and the presence of heart failure might significantly impact long-term prognosis. Therefore, risk scores containing these parameters might demonstrate a more excellent long-term predictive performance than the CRUSADE risk score.

Only a few studies have investigated the predictive performance of risk calculation tools for >1 year in patients with ACS. This is the first study to investigate 3-year time-dependent changes in predictive performance for mortality in patients with ACS between risk calculation tools to estimate short- and long-term risk, using time-dependent ROC analysis. The AUCs of each mortality risk score in patients with ACS remained relatively high during the observation period. This study revealed excellent risk stratification performance of the ACTION and GRACE risk scores, especially in the early phase. In contrast, the AUC of the CRUSADE risk score for all-cause death was reversed from the other risk scores during the 3-year observation period; however, there was no significant difference. The difference in the AUC between the CRUSADE and other risk scores might increase by extending the observation period to 5 or 10 years. The ACTION and GRACE risk scores are risk calculation tools proposed to estimate in-hospital mortality (11, 16). Conversely, the CRUSADE risk score is a risk calculation tool proposed to estimate long-term mortality (15). This study demonstrated that each risk score was suitable for risk stratification of the original target periods and was an excellent risk calculation tool. Therefore, the results of this study may support those of previous studies.

This study has a few limitations. This was a single-center, retrospective study. Furthermore, the sample size was small for testing and comparing several risk scores. Multicenter prospective studies are required to confirm the findings of this study. More than 40% of the patients had multivessel lesions or left main trunk lesions on coronary angiography, but only 3.7% underwent coronary artery bypass grafting as a therapeutic strategy. The results may differ in a facility with a better environment for emergency surgery. The study population was limited to patients who had undergone coronary angiography for ACS. High-risk cases in which ACS was suspected but coronary angiography was not performed because of advanced age, frailty, or bleeding complications may have been included. Patients who did not undergo coronary angiography may have a poorer prognosis. Cases without visible thrombus or critical stenosis, which is almost equivalent to myocardial infarction with non-obstructive coronary disease (MINOCA), were excluded. The inclusion of MINOCA cases might lead to different results.