The SEER*Stat software (www.seer.cancer.gov, version 8.43)was used to extract data on MBM patients from the SEER database. The International Classification of Oncology Diseases and Version 3 (ICD-O-3) criteria were used to determine the primary cancer site, and the diagnosis was finally histologically confirmed. CSED was defined as patients dying due to MBM within three months. ACED was defined as patients dying due to all causes within three months. For the SEER database component, ethics approval was not required as SEER data is publicly available and de-identified.

The inclusion criteria were as follows: (1) primary cancer only, (2) patients histologically confirmed with MBM from 2010 to 2020 in the SEER database. The exclusion criteria were as follows: (1) diagnosis confirmed by autopsy or death certificate only, (2) age less than 18 years, (3) with unknown clinicopathological information, (4) Unknown metastatic site. For metastatic site identification, we utilized the Collaborative Stage (CS) metastatic site codes in the SEER database to identify patients with brain metastasis and to determine the presence of other distant metastases, including liver, lung, and bone. It’s important to note that the SEER database records these metastases as binary variables (Yes/No) without providing further anatomical subdivision. This limitation prevented more granular analyses of metastatic patterns in our study.

Regarding missing data handling, we implemented a conservative approach to maintain data integrity. For categorical variables where “Unknown” was explicitly coded as a category in the SEER database (such as ulceration status, marital status, etc.), we retained these patients and analyzed “Unknown” as a separate category rather than excluding them. This approach is consistent with previous SEER-based studies and preserves the sample size while acknowledging data limitations transparently (18). This strategy allowed us to develop more robust machine learning models that can provide predictions even when certain data points might be missing in clinical practice.

To confirm the reliability of the data, the included patients from the SEER database were randomly divided into training and testing cohorts in a ratio of 7:3. The validation data cohort was extracted from the Affiliated Tumor Hospital of Xinjiang Medical University from 2010 to 2023, and follow-up lasted until October 31, 2023( Figure 1 ). Approval for this study was obtained from the Institutional Review Board of the Affiliated Tumor Hospital of Xinjiang Medical University.

Independent risk factors were identified through a systematic variable selection process using univariate and multivariate logistic regression analysis. Initially, all candidate variables were evaluated using univariate logistic regression. Features with p values <0.05 in the univariate analysis were then entered into a backward stepwise multivariate logistic regression model. The backward elimination process started with all significant univariate variables included in the full model and sequentially removed the variable with the highest p-value (least significant) at each step, provided that p>0.10. This process continued iteratively until all remaining variables in the model achieved statistical significance (p<0.05). The final multivariate model retained only variables that remained statistically significant after controlling for other factors through this backward elimination process. These independently significant risk factors identified through backward stepwise regression were then used as input features to construct the machine learning models. Seven ML models were developed, including logistic regression (LR), K-nearest neighbor (KNN), support vector machine (SVM), decision tree (DT), random forest (RF), XGBoost, and LightGBM. The training cohort was used for building ML models and cross-validation, while testing and validation cohorts were used to evaluate the predictive performance of the models.

Categorical variables are presented as numbers (%), and comparisons were made using the Pearson Chi-square test. Univariate and multivariate logistic regression analyses were conducted to determine risk factors. All statistical analysis, model development, and validation are implemented in R software (version 4.3.0). A two-sided p-value <0.05 was considered statistically significant.

Cross-validation methodology: All models were evaluated using stratified 10-fold cross-validation to ensure robust performance estimation while maintaining outcome distribution balance across folds. For the logistic regression model, which requires no hyperparameter tuning, cross-validation was used solely for unbiased performance evaluation. For the remaining six models (KNN, SVM, DT, RF, XGBoost, and LightGBM), a nested cross-validation approach was implemented where hyperparameter optimization occurred independently within each fold to prevent data leakage and ensure valid generalizability estimates.

Hyperparameter optimization: Each model’s hyperparameters were tuned using random grid search within the cross-validation framework. KNN optimized the number of neighbors (3-11); SVM tuned cost (-5 to 5) and RBF sigma (-4 to -1); DT optimized tree depth (3-7), minimum node size (5-10), and cost complexity (-6 to -1); RF tuned the number of variables per split (2-10), number of trees (200-500), and minimum node size (20-50); XGBoost optimized six parameters including learning rate, tree depth, and regularization terms; LightGBM similarly tuned six key parameters. For models requiring data preprocessing (KNN, SVM), normalization and dummy coding were performed within each fold using the tidymodels recipe framework to prevent data leakage.

Model validation strategy: Model performance was evaluated through a three-tier validation approach: (1) Internal validation using 10-fold cross-validation on the training cohort to assess model stability and select optimal hyperparameters; (2) Temporal validation using an internal test set (30% holdout from the same institution/database) that was never used during model development or cross-validation; and (3) External validation using an independent cohort from a different institution/database/time period to evaluate model generalizability across different populations and clinical settings. This comprehensive validation strategy ensures robust assessment of model performance and clinical applicability across diverse patient populations.

ROC analysis, calibration curves, and DCA curves were used to evaluate the models’ accuracy.

A total of 1547 MBM patients from the SEER database were finally included in this study. Among them, 531 (34.3%) cases suffered from CSED, and 554 (35.8%) cases suffered from ACED. The details of the patients are displayed in Table 1 . For ACED, most patients were > 60 years old (50%), male (74.2%), white (98%), did not receive surgery (89%), received radiation (75.3%), and did not receive or had unknown chemotherapy status (83.2%). As for CSED, most of the patients > 60 years old (49.5%), male (74%), white (97.9%), without surgery (88.7%), with radiation (75.9%), without or unknown whether perform chemotherapy (82.9%). The demography of patients in the training cohort (n=1080), test cohort (n=467), and validation (n=42) cohort is shown in Table 2 .

According to the univariate logistic regression analyses, age, marital status, primary site, radiation, chemotherapy, ulceration, surgery, bone metastasis, liver metastasis, lung metastasis, and months from diagnosis to therapy may be risk factors for the ACED of MBM patients. Age, radiation, chemotherapy, ulceration, surgery, bone metastasis, liver metastasis, lung metastasis, and months from diagnosis to therapy may associated with the CSED of MBM patients.

Multivariate logistic regression analyses revealed that age, marital status, radiation, chemotherapy, ulceration, surgery, bone metastasis, liver metastasis, and months from diagnosis to therapy were significant risk factors for the ACED of MBM patients. Age, radiation, chemotherapy, ulceration, surgery, bone metastasis, liver metastasis, and months from diagnosis to therapy were significant risk factors for the CSED of MBM patients. ( Table 3 )

The 10-fold cross-validation analysis revealed that the XGBoost model performed best in predicting ACED in MBM patients ( Figure 2A ). In the testing cohort, Xgboost model showed the best with area under curve (AUC)=0.776, accuracy =0.713, sensitivity = 0.682, and specificity = 0.730 ( Figures 3A, B ; Table 4 ). In the external validation cohort, Xgboost model was also shown excellent prediction performance, with AUC=0.717, accuracy =0.659, sensitivity = 0.765, and specificity = 0.593 ( Figures 4A, B ; Table 4 ). Calibration curve analysis revealed that Xgboost model has a more accurate prediction performance ( Figures 3C , 4C ). DCA curves revealed that Xgboost model clinical application value ( Figures 3D , 4D ). Besides that, a nomogram based on the Xgboost model showed that age contributes most to the ACED of MBM patients, followed by chemotherapy, ulceration, surgery, liver metastasis, bone metastasis, radiation, months from diagnosis to therapy, and marital status ( Figure 5A ).

The 10x cross-validation analysis revealed that the LR model performs best in predicting the CSED of MBM patients ( Figure 2B ). In the testing cohort, the LR model showed the best performance with AUC=0.694, accuracy =0.614, sensitivity = 0.737, and specificity = 0.555 ( Figures 6A, B ; Table 4 ). In the external validation cohort, the LR model also showed excellent prediction performance, with AUC=0.730, accuracy =0.659, sensitivity = 0.750, and specificity = 0.609 ( Figures 7A, B ; Table 4 ). Calibration curve analysis revealed that the LR model has a more accurate prediction performance ( Figures 6C , 7C ). DCA curves revealed the LR model clinical application value ( Figures 6D , 7D ). Besides that, a nomogram based on the LR model revealed that ulceration contributes most to the CSED of MBM patients, followed by chemotherapy, age, surgery, liver metastasis, bone metastasis, radiation, and months from diagnosis to therapy ( Figure 5B ).

To further interpret the machine learning models, we visualized the feature importance using the Shapley Additive Explanations (SHAP) analysis for all seven models predicting both ACED and CSED outcomes ( Figures 8A–G ), ( Figures 9A–G ). In the SHAP plots, variables are arranged in descending order of importance based on their mean absolute SHAP values.

Despite algorithmic differences and different outcome definitions (ACED vs CSED), there was remarkable consistency in feature importance rankings across all seven models (Logistic Regression, XGBoost, KNN, LightGBM, Random Forest, Decision Tree, and SVM). Chemotherapy, age, liver metastasis, and months from diagnosis to therapy consistently emerged as the top four most important features across both ACED and CSED prediction models, highlighting their critical role in predicting early death in melanoma brain metastasis patients regardless of death cause classification.

The consistent ranking of these four variables across different algorithms and outcome definitions suggests their robust prognostic value. The SHAP values provide insights into both the magnitude and direction of each feature’s contribution to the prediction, with higher absolute SHAP values indicating greater influence on the model’s output. This consistency between ACED and CSED models reinforces the reliability of these predictive factors in clinical practice.