We retrospectively included 614 patients with colorectal cancer who underwent pelvic multiparameter MRI from January 2015 to January 2022 in the gastrointestinal surgery department of Gezhouba Central Hospital of Sinopharm. Comparison against abdominopelvic CT, SPECT/CT, or histopathologic tissue sampling was used to establish the baseline ground truth for presence or absence of bone metastases at the time of enrollment. The inclusion criteria of patients are as follows: (i)Patients suspected of colorectal cancer or undergoing pelvic multi-parametric diffusion weighted imaging(mp-DWI) scan due to reexamination after colorectal cancer treatment; (ii)Patients with complete pelvic DWI images; (iii)Patients without primary pelvic bone disease (primary osteosarcoma, bone cyst, blood system disease, fracture, etc.). Exclusion criteria: (i)Patients with a history of pelvic bone structure surgery; (ii)Patients with a history of other malignancies; (iii)Patients with unsatisfied image quality, such as motion artifacts and chemical shift artifacts; (iv)Patients with incomplete scanning scope and not including most pelvic bone structures. This retrospective study was approved by the Ethics Committee of Gezhouba Central Hospital of Sinopharm, and the research scheme was implemented according to the artificial intelligence(AI) model training specifications of the unit. All patients’ personal information is encrypted to prevent leakage, and complies with the Declaration of Helsinki. All patients in this study were informed of the study protocol and approved the study by written consent. The process of incorporating patients and building prediction models was shown in Figure 1 .

We used GE Discovery MR750 W3.0 T machine to perform pelvic MRI plain scan and enhanced scan. The patient was instructed to lie on his back with his head advanced, and the scanning range was from the umbilical foramen level to the pubic symphysis. The sequences included conventional transverse T1WI, (with or without) conventional transverse T2WI, (transverse, sagittal, coronal) fat compression T2WI, (with or without sagittal) transverse DWI (b value=1000s/mm, b value=2000 s/mm), transverse and sagittal fat compression LAVA enhanced sequences. Sagittal fat compression T2WI scanning parameters: TR 4 800.0 ms, TE 110.5 ms, matrix 320 × 320, layer thickness and layer spacing are 5 mm and 1 mm respectively, FOV range: 28 cm × 28 cm.

Next, we standardized the format of DWI, that is, converted the high b value DWI image in DICOM format to Nifty format, and then the radiology resident (with film reading experience of 3 years or more) used ITK-SNAP3.6.0 software(http://www.itksnap.org/pmwiki/pmwiki.php?n=Main.Publications ) to manually delineate and label the DWI image along the edges of various pelvic bone structures. In addition, a radiologist (with film reading experience ≥ 15 years) modified and confirmed the label, and the confirmed image label was used as the gold standard of pelvic bone structure segmentation model.

We preprocessed the image and extract the texture features, and used the software GE (Shanghai) AK (Artifical Intelligence Kit, V3.2.0R version) application platform to preprocess the image: linear method is used for resampling, with X, Y, Z spacing of 1.000; Gaussian 0.50 is used for noise removal; MR bias field correction is adopted to eliminate the stray intensity change caused by the non-uniformity of magnetic field and coil; Intensity standardization adopts gray discretization, and the expected minimum value and maximum value are 0.015 and 0.255,respectively. After image preprocessing (resampling, offset field correction, intensity standardization), a total of 48 GLCM texture features are finally collected by AK software, including 8 types of parameters, namely: cluster facilitation, cluster shade, correlation, GLCM energy, GLCM entropy, Haralick correlation, inertia, inverse difference motion; 6 angles are calculated, including full angle, full angle SD, 0°, 45°, 90° and 135°.

We preprocessed the DWI images of 614 patients, namely: size=64 × 224 × 224 (z, y, x), automatic window width and level. Patients were randomly divided into training set (70%) and verification set (30%) according to 7:3. In order to eliminate the imbalance of the classified training set data, we balance the positive/negative samples by reducing the sampling, and use Min Max to normalize the feature matrix. At the same time, we use Pierce correlation coefficient to reduce the dimension of the data, and the eigenvectors of the transformed eigenmatrix have independent features.

There are four machine learning model building methods used in this study, including artificial neural network model (ANNM), random forest model(RFM), decision tree model(DTM) and support vector machine model (SVMM) (18–21). As linear regression models are often used to build clinical prediction models, this study builds generalized linear regression model (GLRM) based on Softmax regression (22, 23), namely: ϕ k = 1 − ∑ i − 1 k − 1 ∅ i RFM, DTM, ANNM and SVM are the most commonly used algorithms in machine learning (18). In this study, a pelvic bone metastasis prediction model was built based on four supervised learning algorithms.

Before building the prediction model, we use recursive feature elimination algorithm to select features and sort them, and select the first 8 features as the best feature subset; At the same time, for the GLRM, the minimum absolute shrinkage and selection operator classifier are selected to establish a classification model for predicting pelvic bone metastasis based on DWI images (24). The effectiveness evaluation of each model mainly includes decision curve analysis(DCA) (25), area under the receiver operating characteristic (AUROC) curve and clinical influence curve(CIC) (26).

The distribution of “measurement” and “counting” data in accordance with normal distribution in this study is expressed by means (interquartile interval) and percentage (%). For the independent two sample nonparametric test, the Mann Whitney rank sum test is used for the inter-group comparison that does not meet the normal distribution (27); The t-test or chi square test is used for the inter group comparison of samples from normal or nearly normal populations. In addition, the visual analysis of all charts in this study was completed with R studio software (download website: https://www.r-project.org/); Two tailed P values less than 0.05 were considered statistically significant.

According to Caret software package algorithm, 614 patients included in this study were randomly divided into training set (N=429,70%) and internal verification set (N=185,30%) according to 7:3. The clinical characteristics and image data sources of patients in the data set were summarized in Table 1 and Supplementary Table 1 . The average age of patients used for pelvic bone structure segmentation model training was 58 (50, 68) years. Among all the patients used to establish the pelvic bone metastasis classification histological model, 53 patients had bone metastasis [average age 55 (47, 65) years], and 561 patients had no bone metastasis [average age 58 (50, 68) years]. In addition, in the split model sample group, there was no statistically significant difference in clinical characteristics (age, pathology, osseous alteration, CEA and tumor location) between the training set and the internal validation set (P>0.05); The three types of GLCM parameters, namely correlation (full angle, 0°, 45°, 90°), inertia (full angle, full angle SD, 0°, 45°, 90°), inverse difference (full angle, 0°, 45°, 90°), cluster prominence and cluster shadow, had statistical differences (P<0.05), while energy, entropy and Haralick correlation had no statistical differences (P>0.05).

Then, we compared the classification features of bone metastasis lesions based on GLCM between groups, and analyzed the correlation between patients’ baseline data and candidate variables of bone metastasis. The results showed that Haralick_90, Haralick_all, IV_0, IG_90, IG_0, Haralick_30, CSV, Entropy and Haralick_45 were highly positively correlated with pelvic bone metastasis in colorectal cancer patients ( Figure 2A ). In addition, in order to build radiomics model for colorectal cancer patients to conduct classification and evaluation with and without pelvic bone metastasis, we conducted feature extraction from the labeled and segmented images and labels based on the manually labeled and automatically segmented pelvic bone structures, respectively. The extracted features were used to establish the radiomics model, and the processing steps included data equalization, data normalization, feature dimension reduction, and feature selection. As shown in Figure 2B , Haralick_90, IV_0, IG_90, Haralick_30, CSV, Entropy and Haralick_45 were the intersection candidate predictors of RFM, SVM, ANNM and DTM.

According to the results of multiple logistic regression analysis, a nomograph ( Figure 3A ) was developed. The visual quantitative mapping tool of the prediction model was based on the scaling of each regression coefficient to 0 to 100 points in the multiple logistic regression. β The influence of the variable with the highest coefficient (absolute value) is assigned 100 points. Add the scores of all independent variables to get a total, and then convert it into the probability of predicting pelvic bone metastasis. Generally, the C index and AUC value exceeding 0.6 implied a reasonable estimate. This study showed that the C index of GLRM was 0.72, and resampling also showed that the model had an ideal robustness ( Figures 3B, C ).

As shown in Figure 4 and Supplementary Table 2 , the RFM based on the “bagging” algorithm sorted the GLCM parameters, where Haralick_90, Haralick_all, IV_0, IG_90, IG_0, Haralick_30, CSV, Entropy and Haralick_45 were suitable for further model building of the RFM algorithm; The prediction efficiency of the model showed that the model still had a robust and efficient prediction efficiency (AUC: 0.926,95% CI: 0.873-0.979), even though it passed the ten fold cross validation. Consistent with the parameter variables of the RFM model, DTM ( Supplementary Figure 1 ) adopted CSV, Entropy and Haralick_30 as the decision factors in the “branches” of the model, and its prediction efficiency in the training set was worse than that of RFM (AUC: 0.889,95% CI: 0.836-0.942); However, ANNM included eight parameters, namely, Entropy, IG_90, IV_0, IV_45, IV_90, Haralick_al, Haralick_30, Haralick_45, Haralick_90 and CSV. The prediction efficiency obtained was AUC: 0.912, 95% CI: 0.859-0.965, which was worse than RFM, but better than DTM, SVM and GLRM.

DCA is a relatively new model evaluation method compared with ROC curve (28). In this study, we adopted two model effectiveness evaluation methods. We verified the linear regression model GLRM and four machine learning models (RFM, ANNM, SVM and DTM) with an internal test set. From the perspective of prediction accuracy, we can see that RFM has the largest “net benefit” in DCA(threshold probability=0.81), followed by ANNM, DTM and SVM ( Figure 5 and Supplementary Table 3 ). GLRM was the least predictive machine learning model, with threshold probability=0.54.

At the same time, the ROC curve showed that the diagnostic efficacy of bone metastasis of RFM in training set and verification set was [AUC: 0.926,95% CI: 0.873-0.979] and [AUC: 0.919,95% CI: 0.868-0.970] respectively, while the diagnostic efficacy of bone metastasis of ANNM in training set and verification set was [AUC: 0.912,95% CI: 0.859-0.965] and [AUC: 0.894,95% CI: 0.843-0.945] respectively, which was slightly lower than that of RFM; As shown in Table 2 and Supplementary Figure 2 , in general, the prediction efficiency of the prediction model for pelvic metastasis of colorectal cancer constructed by machine learning algorithm was better than that of the traditional generalized linear model. The results were consistent both in the training set and internal test set.

Based on the evaluation of five prediction models for pelvic metastasis of colorectal cancer, we found that RFM was the best in terms of prediction efficiency. In order to further evaluate the differentiation efficiency of RFM, we used CIC to evaluate the “classification accuracy” of RFM in training set and internal verification set. As shown in Supplementary Figure 3 , the blue curve (number high risk with outcome) indicated the number of true positives under each threshold probability, and the red curve (numberhigh risk) indicated the number of people classified as positive (high risk) by the prediction model under each threshold probability. It was credible that RFM can accurately distinguish patients with bone metastasis from those without bone metastasis, whether in training set or internal verification set, which further confirms that RFM can not only increase the interpretability of bone metastasis risk grading model for colorectal cancer patients, but also improve the grading accuracy. Therefore, RFM was suitable for stratified diagnosis and treatment.