The study protocol was reviewed and approved by the Ethical Committee of Guangdong Second Provincial General Hospital (Approval number: GD2H-KY IRB-AF-SC.07-01.2), and the ethical guidelines of the 1975 Declaration of Helsinki were followed.

A prospective longitudinal study named Ivy Action was performed in the form of a voluntary prevention screening program for ischemic cerebrovascular disease targeting the adult population of multicenter communities (Guangzhou) in 2018–2019. The study included individuals aged 35 years or older, who had no history of stroke or had experienced a good recovery after stroke with a modified Rankin scale score of 2 or less indicating good recovery. Exclusion criteria included individuals unable to communicate with the research team, those with mobility impairments hindering study participation, individuals with heart, liver, or kidney failure, and patients with a history of malignancy. The comprehensive study protocol included gathering information on basic socio-economic status, social and residential status, smoking, housework, physical activity, sleep and dietary habits. Additionally, cardiovascular risk factors, family history and medical history were meticulously recorded.

In our machine learning models, we employed a comprehensive set of risk factors, including age, gender, biomarkers, systolic and diastolic blood pressure, health status, lifestyle factors, family history of cardiovascular disease, and medical history (Table 1). These factors were chosen for their well-documented association with carotid atherosclerosis and their clinical significance in predicting cardiovascular events. The medical exam included non-invasive tests (resting blood pressure, anthropometric measurements), ECG, anxiety and depression Scale, venous blood tests performed in a central laboratory with conventional enzymatic methods, echocardiography and carotid duplex scans (10). Waist and hip circumference were measured with light clothing. Cardiac History encompasses any medical history of coronary artery disease, myocardial infarction, angina, atrial fibrillation, or valvular heart disease. A waist-to-hip ratio over 0.90 for men and >0.85 for women was considered central obesity.

It is essential to note that the primary aim of this study was focused on the prevention and management of stroke. All examination procedures were provided free of charge, and participants had the option to voluntarily undergo testing, ensuring that the study adhered to ethical standards and obtained informed consent from the participants.

An examination was conducted with the patient supine with the head slightly extended. Carotid intima media thickness (IMT) was evaluated with a 7.5 MHz linear array transducer using a LOGIO E ultrasound (GE healthcare, USA). Measurements were taken carefully, analyzing the maximum IMT at two sites and all interfaces of the near and far walls of the common carotid artery (CCA), located 1–1.5 cm below its bifurcation. To obtain anechoic images, adjustments the image gain, depth, and focus were meticulously made for each participant. Plaques were defined as a focal thickening extending into the lumen by at least 0.5 mm or as IMTs greater than 1.5 mm, measured between the near- and far-walls of any carotid segment in the internal carotid artery, common carotid arteries, and carotid bulb. Carotid artery (CA) was defined as having a maximal intima-media thickness (IMT) of ≥1.0 mm and/or the presence of atherosclerotic plaques. To minimize intra-operator variation, a single experienced sonographer conducted all ultrasound examinations.

The analytical approach employed in this study leveraged R code to perform several critical steps. To address missing data, the “missRanger” package was utilized for imputing missing values. Subsequently, a diverse set of machine learning models, encompassing support vector machines (SVM), XGBoost, decision trees, random forests, and logistic regression, were employed. Hyperparameter tuning was conducted through random search to identify the optimal model configuration. To ensure robust model performance while mitigating overfitting, a 5-fold cross-validation strategy was implemented. Given the presence of imbalanced target classes, data balance was achieved through oversampling. Model performance evaluation primarily relied on the area under the curve (AUC) metric, facilitating the selection of the best-performing machine learning method as the final model.

In the final model evaluation, three key approaches were employed: (1) ROC and PRC (Precision-Recall Curve) plots provided in-depth insights into model performance. (2) Model calibration was conducted using the calibration function from the “calibrate” package. This process involved generating calibration plots for apparent and bias-corrected probabilities to comprehensively assess model performance. (3) Decision Curve Analysis (DCA) was carried out to evaluate the clinical utility of the model. Decision curves and the plot_decision_curve function were employed to visualize the net benefit of the model across various threshold probabilities.

Additionally, a forest plot was created using the forest_model function from the “forestmodel” package, offering deeper insights into the relationship between predictor variables and the outcome variable. Sample size calculation was executed using the ShowRegTable function, contributing to an understanding of dataset characteristics and ensuring an adequate sample size for the analysis.

In summary, this methodological approach comprised data imputation, machine learning modeling, hyperparameter tuning, cross-validation, data balancing, and comprehensive model performance evaluation, culminating in the selection of the best-performing model. The final model evaluation involved ROC and PRC plots, model calibration, Decision Curve Analysis, and a forest plot. Sample size calculations were conducted to support the robustness of the analysis.

Grouping and Reference Value Selection of Risk Factors: Firstly, risk factors are grouped based on their clinical significance or common usage. In each group, appropriate numerical values are selected as reference values (Wij). Typically, the midpoint of the group is chosen as the reference value. Secondly, Handling Categorical Variables: For categorical variables such as gender, a category is chosen as the reference, and its reference value is set to 0, while other categories are naturally assigned numerical values, typically 1. Basic Risk Reference Value: Each risk factor needs to have a suitable group selected as the basic risk reference value (WiREF). When constructing a scoring tool later, the score for this group will be set as 0. Scores for other groups will be assigned positive or negative values based on their relationship with WiREF. Scoring Calculation: Using the regression coefficients estimated by a multiple logistic regression model (βi) and the reference values for each risk factor group (Wij), the distance between each risk factor's group and the basic risk reference value (WiREF) is calculated (D). The calculation formula is D = (Wij−WiREF) * βi. Thirdly, Risk Probability Calculation: Using the equation of a multiple logistic regression model, the probability of risk prediction (p^) for each score is calculated. This probability value represents the likelihood of an individual experiencing a specific event under certain risk factors. The formula is:p^=11+exp⁡(−∑i=0pβiXi),where ∑i=0pβiXi represents a linear combination, which is approximately a constant term added to the product of the scores for various risk factors. Analyses were conducted using the R Statistical language (version 4.3.1; R Core Team, 2023).

Initially, 1,560 participants were recruited; however, 45 individuals with incomplete carotid ultrasound examinations were excluded from analysis. Consequently, the final analytical sample comprised 1,515 subjects. Carotid ultrasound revealed CAS in 869 participants, constituting a prevalence of 57.4% in the study cohort. Individuals in the CAS group demonstrated more advanced age compared to the non-CAS group (mean 63 years vs. 54 years). Additionally, the majority of CAS individuals were female (519 subjects, accounting for prevalence of 60%). Detailed demographic information for the entire study population is presented in Table 1.

The preprocessing phase involved imputing missing values to ensure a complete dataset, as depicted in Figure 1. Subsequently, five machine learning models were explored, including logistic regression, support vector machines (SVM), XGBoost, decision trees and random forests. Hyperparameter optimization was carried out through random search across defined parameter spaces for each algorithm. To mitigate class imbalance, the “oversample” method from the classbalancing package was utilized, equalizing the training set's target categories. For imbalanced classification, the area under the receiver operating characteristic curve (AUC) served as a robust metric of overall performance. As shown in Figure 2, the ROC curves described each learner's discrimination capacity across thresholds, with the AUC score quantifying overall predictive power.

Simultaneously, the AUC values for each learner on the training set were computed and visually represented in the figure. The outcomes of different machine learning algorithms on the training and test sets were summarized in Table 2. Remarkably high accuracy levels (79%–100%) were achieved by all classifiers during the training phase. However, when applied to the test set, the overall accuracy dropped to 75%. Notably, an accuracy of 82% was achieved by the logistic classifier, outperforming its counterparts. This discrepancy underscores potential challenges in the model's generalization to unseen data, with the logistic regression model exhibiting greater robustness during the transition from training to testing phases. Therefore, the logistic regression model (Figure 3) was selected as the final model, incorporating six variables: age, systolic pressure, hypertension, total cholesterol, HDL cholesterol, and sex. Based on these six variables, the model was named the AP2C2S model. Specifically, age showed a significant positive correlation with CAS (OR = 1.14, 95% CI: 1.13–1.16, P < 0.001), while systolic pressure also demonstrated a positive correlation (OR = 1.02, 95% CI: 1.01–1.02, P < 0.001). Conversely, high-density lipoprotein (HDL) exhibited a significant negative correlation with CAS (OR = 0.30, 95% CI: 0.17–0.50, P < 0.001), indicating its role as a protective factor against CAS. Additionally, total cholesterol (TC) showed a positive correlation with CAS (OR = 1.58, 95% CI: 1.35–1.86, P < 0.001). Gender, when treated as a categorical variable, was significantly associated with CAS, with males showing a higher risk (OR = 2.09, 95% CI: 1.57–2.80, P < 0.001). Moreover, individuals with a history of hypertension exhibited a significant positive correlation with CAS (OR = 1.53, 95% CI: 1.11–2.11, P = 0.009). These findings underscore the significance of these variables in predicting CAS risk and justify their inclusion in the AP2C2S model.

Expanding upon our analysis, we extracted the regression model and generated ROC and PRC curves (Figures 4A,B) to further assess the model. We performed consistency checks using the calibrate function and depicted the calibration curve (Figure 4C). Additionally, employing decision curve analysis allowed us to assess patient benefits at different thresholds, providing insights into the model's potential value in real clinical decision-making (Figure 4D). This series of analyses and evaluations contributes to a thorough understanding of the model's performance and applicability.

Firstly, we categorized the variables of the final model as per the methodology section, assigning values to each variable based on the description provided (refer to Table 3). Subsequently, utilizing the equation of the multiple logistic regression model, we calculated the risk prediction probability for each corresponding score. Through this iterative process, we established a comprehensive table illustrating the correspondence between total scores and risk prediction probabilities, as depicted in Table 4.

The seamless integration of this grouping and calculation procedure facilitates a systematic evaluation of each variable's contribution to the final model. It ensures a thorough and precise prediction of risk probability for patients. This methodical approach underscores the scientific rigor and practical utility of our risk assessment model in a clinical context.

To illustrate the application of the AP2C2S risk scoring tool concretely, we provide an example based on Tables 3, 4. Let's consider a 55-year-old male patient with a systolic blood pressure of 135 mmHg, total cholesterol (TC) of 5.5 mmol/L, high-density lipoprotein (HDL) of 1.2 mmol/L, and a history of hypertension. Referring to Table 3, we can assign points for each risk factor: Age (55–64 years): 8 points; Gender (male): 3 points; SBP (130–139 mmHg): 1 point; HDL (1.14–1.29 mmol/L): 0 points; TC (reference range): 3 points; Hypertension (yes): 1 point. Adding up these points, the patient's baseline total score is 16 points. Next, referring to Table 4, we find the corresponding CAS risk prediction probability based on the total score. In this example, a score of 16 corresponds to a risk prediction probability of 15.07%.