This multi-institutional retrospective study included the eligible participants from five institutions. From November 2016 to July 2023, 1,747 participants who underwent spinal MRI and DR within 2 weeks of back pain were enrolled in this study. We excluded 3,858 participants, including those in whom with more than 1 week between DR and MRI examinations (n = 1,745) and those without complete raw data and qualified DR/MRI images (n = 2,113). Notably, participants with old VCFs, those with suspected malignant VCFs, and those who underwent surgical treatments, such as vertebroplasty, fixation, and fusion surgery, were not excluded to improve the robustness of our model. The study flowchart and detailed inclusion and exclusion are shown in Figure 1 and the Supplementary Appendix (p. 3).

This study was approved by the Ethics Committees of the five hospitals and was conducted in accordance with the ethical principles of the Declaration of Helsinki. We used only preexisting medical data; therefore, the requirement for informed consent was waived for this retrospective study. The work was compliant with the STROCSS criteria (Mathew et al., 2021).

The participants underwent routine anteroposterior and lateral lumbar spine radiography using DR instruments of six devices. All MR scans were acquired using a 1.5-T or 3.0-T MR system with a multichannel phased-array spine coil, and FS T2WI was performed using a turbo spin-echo sequence. The image acquisition parameters and details of the DR instruments of each institution are presented in the Supplementary Appendix (p. 3). The DR and MR images were stored and retrieved using a picture archiving and communication system.

The whole spinal DR image, without the segmentation and delineation of region of interest (ROI), were labelled based on spinal MRI. FS T2WI is regarded as the standard reference for identifying fresh VCFs owing to its excellent ability to visualize and characterize high-signal areas and bone marrow edema (Genant and Jergas, 2003; Zhang et al., 2023). The sagittal MRI and lateral DR images were analyzed by at least one radiologist and one spinal surgeon. Any disagreements were resolved through consensus discussions.

A total of 1,312 participants from three institutions were divided into a training cohort (n = 1,050; male: 416, 39.62%) and a validation cohort (n = 262; male: 118, 45.04%) based on the patient identification number (first 80% in the training cohort, the remaining 20% in the validation cohort) to train and validate the DRN model. A total of 221 participants (male: 100, 45.25%) and 214 participants (male: 87, 40.65%) from the other two institutions, were divided into YHDH and BMUH cohorts based on their respective institutions, and were categorized as external test cohorts to evaluate the external generalizability of the DRN model.

The DRN model was developed based on a previously reported 101-layer residual network (He et al., 2016), and trained on classified spinal DR images of patients in the training cohort. The architecture of the DRN included an input layer, a first layer (a 7 × 7 convolutional layer and an output channel of 64), and second to fifth layers (four residual modules). Each residual module comprised several residual blocks, a global average pooling layer (performing global average pooling on the output of the last residual module), and a fully connected layer (connecting the output of the global average pooling layer to a fully connected layer for the classification task).

The input images from the training cohort were resized to 224 × 224 pixels. We performed image augmentation and amplification of the original images using a horizontal flip to reduce overfitting. This normalization process allows the image data to converge smoothly during training, thereby improving the model’s stability and training effectiveness. All loss values were summed and used to measure the model’s performance. The DRN model was trained based on ImageNet pre-trained parameters. The Adam optimizer was utilized in this model, and the parameters to be optimized and the learning rate were passed to it. The Adam optimizer is an adaptive learning rate optimization algorithm that automatically adjusts the learning rate based on the gradients of the parameters, which results in more efficient updates of the model’s weights. The learning rate was set to 0.0001. The batch size was 6, and the number of training epochs was 200. All the development processes were performed using the PyTorch framework. Detailed information on the development processes, parameters, software, and packages is presented in the Supplementary Appendix (p. 4).

We assessed the diagnostic performance of the DRN model in both the validation and external test cohorts using the best model training weights and the same thresholds. In the present study, the DRN model was compared with previous competitive models, including 50-layer residual network (ResNet-50) (He et al., 2016), Shufflenet-v2 (Zhang et al., 2018), EfficientnetV2-S (Tan and Le, 2021), EfficientnetV2-M (Tan and Le, 2021), and EfficientnetV2-L (Tan and Le, 2021) to verify the effectiveness of the DRN model. The diagnostic efficacy of the models for identifying fresh VCFs were analyzed. The true positive (TP), true negative (TN), false positive (FP), false negative (FN), accuracy, balanced accuracy, sensitivity, and specificity of the different models were calculated. The TP event is defined as correct identified the VCFs from X-ray images. Receiver operating characteristic (ROC) curves were used to assess the sensitivity and specificity of the models, and the area under the ROC curves (AUC) was used to describe the discriminative power of the models. In addition, the macro- and micro-averaged ROC curves, precision recall (PR) ROC, and PR AUC were included in the evaluation metrics.

Both fresh and old VCFs can lead to changes in vertebral morphology. Therefore, in clinical practice, even if there are changes in vertebral morphology observed on a participant’s spinal DR, clinicians still find it challenging to accurately identify fresh VCFs solely through visual inspection. In order to assess the impact of the simultaneous presence of old VCFs on the diagnostic results of the DRN model, we grouped participants of the external test cohorts based on whether they had concomitant old VCFs, and the subgroup analysis was conducted to further evaluate the diagnostic efficiency of the model for fresh VCFs.

Visualization experiments based on gradient-weighted class activation mapping (Grad-CAM) have been previously conducted to locate and identify the activation area over the input images to explain the feature learning and classification results of the DL models (Panwar et al., 2020; Liu C. et al., 2021). In addition, Grad-CAM technology uses gradient information flowing into the last convolution layer to assess the weight of each neuron in the final decision of the fresh VCFs (Mu et al., 2021).

All statistical analyses were performed using SPSS Statistics software version 22.0. (SPSS Inc., Chicago, IL, United States of America). and Python version 3.7. We used the bootstrap sampling to calculate the confidence interval directly (n_bootstrap = 1,000), and take the desired percentile of the bootstrap estimates as the bounds of the confidence interval. Normally distributed continuous variables were represented as means ± standard deviations and categorical variables as percentages (%). The Student’s t-test and Mann–Whitney U test were used to compare continuous clinical variables, where appropriate. The chi-squared test or Fisher’s exact test was used for categorical variables. Statistical significance was set at p-value <0.05.

A totally of 1,747 participants (572 fresh VCFs and 1,175 non-fresh VCFs; 721 male and 1,026 female) with a mean age of 62.08 ± 13.92 years, were enrolled in this study (Supplementary Appendix p. 6). The training cohort from the three institutions included 1,050 participants (355 fresh VCFs and 695 non-fresh VCFs; 416 male and 634 female), with a mean age of 62.78 ± 13.50 years. In addition, there were 228 participants diagnosed with lumbar fresh VCFs in the training cohort. The validation cohort from the same three institutions included 262 participants (88 fresh VCFs and 174 non-fresh VCFs; 118 male and 144 female), with a mean age of 61.97 ± 13.05 years, and there were 32 participants diagnosed with lumbar fresh VCFs in the validation cohort. The external test cohorts were obtained from YHDH and BMUH. The YHDH cohort included 221 participants (59 fresh VCFs and 162 non-fresh VCFs; 100 male and 121 female), with a mean age of 56.8 ± 16.26 years. The BMUH cohort included 214 participants (70 fresh VCFs and 144 non-fresh VCFs; 87 male and 127 female), with a mean age of 64.28 ± 13.12 years. Additionally, 117 (11.14%) participants in the training cohort, 31 (11.83%) in the validation cohort, 56 (25.34%) in the YHDH cohort, and 47 (21.96%) in the BMUH cohort were diagnosed with old VCFs (or combined old VCFs). Detailed demographics of each cohort are shown in Table 1 and the Supplementary Appendix (p. 7, 8).

As shown in Figure 2, we visualized the feature attention maps of different DL models using the Grad-CAM technique, which intuitively illustrated the effectiveness of our DRN model. The DRN model generated correct activation on the fresh VCFs and accurate peak responses on the area of the target vertebral body parts and demonstrated better feature representation learning and classification performance. The attention heatmap is designed to refine attention on the most distinctive region; however, ResNet-50, Shufflenet-v2 and EfficientnetV2-S generated wider, even erroneous, activation areas than the target vertebral body.

In addition, Grad-CAM demonstrates different feature attention maps for different predicted results of DRN model. The generated focused peak responses for images with FP predictions were similar to those of the TP images; however, the activated area corresponded to the erroneous vertebral body or the area outside of the vertebral body. The DRN model generated a low intensity activation area or no peak responses for TN and FN images (Figure 3).

The AUC was 0.99, 0.89, and 0.88 in the validation, YHDH, and BMUH cohorts, respectively, for the DRN model for detecting and discriminating fresh VCFs (Figure 4). Moreover, the validation cohort had an accuracy of 96.62%, sensitivity of 93.18%, and specificity of 98.35%. The accuracies were 81.45% and 72.90%, sensitivities were 84.75% and 91.43%, and specificities were 80.25% and 63.89% in the YHDH and BMUH cohorts, respectively (Table 2). The detailed performance of the DRN in different cohorts is shown in the Supplementary Appendix (p. 8, 10). In addition, the results of the DRN with different parameters are shown in the Supplementary Appendix (p. 11, 12).

In the present study, the DRN model was compared with previous DL models regarding the ability to discriminate fresh VCFs to evaluate its effectiveness. The AUC of ResNet-50, Shufflenet-v2, EfficientnetV2-S, EfficientnetV2-M, and EfficientnetV2-L were 0.89, 0.89, 0.15, 0.29, and 0.22, respectively, in the YHDH cohort and 0.84, 0.84, 0.52, 0.46, and 0.49, respectively, in the BMUH cohort (Figure 5). The accuracy, sensitivity, and specificity of the validation and external test cohorts for each model are listed in Table 2. The ROC curves in the validation cohort and the overall external test cohort for the models are shown in the Supplementary Appendix (p. 13, 14).

The influence of old VCFs on the performance of the DRN model in identifying fresh VCFs was assessed in the subgroups. Detailed demographics of each subgroup of the YHDH and BMUH cohorts are shown in Table 3. The AUC was 0.90 (95% confidence interval [CI] 0.84–0.95) and 0.84 (95% CI 0.72–0.93) in the non-old VCFs and old VCFs groups, respectively, in the YHDH cohort (p = 0.067). The AUC was 0.89 (95% CI 0.84–0.94) and 0.85 (95% CI 0.72–0.95) in the non-old VCFs and old VCFs groups, respectively, in the BMUH cohort (p = 0.051) (Table 4). The confusion matrix for each subgroup is shown in Figure 6, and attention heatmaps of DRN in old VCFs group of external test cohorts is shown in Figure 7.

To the best of our knowledge, this study is the first to develop and test a fully automated DL diagnosis model for fresh VCFs based on spinal DR images and achieve visualization. The segmentation and delineation of ROI in DL or ML models require considerable time and effort. Therefore, achieving fully automated positioning and detection of fresh VCFs and reducing the complexity of segmentation is necessary. In previous studies on DL models, the diagnosis and detection of fresh VCFs from spinal DR images required segmentation of each vertebral body from the whole thoracic and lumbar vertebrae (Chen et al., 2022; Xu et al., 2023).

In present study, we achieved fully automated positioning and detected fresh VCFs with no ROI requirement using our DRN model. The local perception property and local feature extraction capability of four convolution layers endows the DRN model with an advantage in handling spinal DR image, enabling the DRN model to capture intricate details and structural information of fresh VCFs, thereby improving the recognition and automatic localization capabilities for the fresh VCFs. Intermediate activation layers were visualized to assess the accuracy of the positioning function in the DL network to explore the detection process and DL features. Our model enables better feature representation learning and a clear activation heatmap compared to other methods, as shown in the results of the Grad-CAM and final output, which allows better automatic positioning performance for the identification of fresh VCFs. In addition, we attempted to identify fresh VCFs in the C-arm DR images based on our models; however, we did not achieve accurate detection. Therefore, multicenter analyses with a large number of C-arm images of the spine are required to explore more stable and accurate DL models to determine the surgical segments and prevent surgical accidents.

In addition, many DL models are dedicated to segmenting bones and vertebrae from radiological images (Huang et al., 2020; Wennmann et al., 2022a). Klinder et al. (Klinder et al., 2009) reported an automated model-based vertebral detection, identification, and segmentation method; this model can achieve an identification success of more than 70% for a single vertebra. Wennmann et al. (Wennmann et al., 2022b) explicitly provided segmentations of vertebrae based on U-Nets, which trained 106 patients from eight centers. These findings may strengthen our future studies on DL models, particularly those focusing on automatic segmentation and the identification of spinal diseases. For example, by utilizing the hierarchical structure of the DRN model as the encoder and integrating the U-shaped architecture of segmentation models like U-Net, we may can achieve a sophisticated approach aimed at enhancing the model’s versatility and performance. This integrated approach not only enhances the model’s versatility, but also opens new possibilities for image segmentation problems in various application scenarios.

Previous studies have reported that bone signal intensities and radiomic features can markedly deviate between different scanners and protocols of MRI, computed tomography, and radiography (Wennmann et al., 2022c; Wennmann et al., 2022d). In addition, models that are developed and tested on a single cohort may be prone to overfitting, resulting in the final model performing well only for images in that cohort and showing low accuracy in other datasets (Ueda et al., 2023).

In previous studies, only a single dataset was used for the development of the DL model, and the images used for development and testing were obtained from the same dataset (Chen et al., 2022; Xu et al., 2023). The training and test cohorts of our study were collected from different institutions and using different scanners to prevent overfitting and enhance robustness and external generalizability. In this study, the participants in the training cohort were recruited from three hospitals, which improved the robustness of the DRN model. In addition, two external test cohorts were used to assess the external generalizability of the DRN model, and a high AUC (0.89 and 0.88 in the YHDH and BMUH cohorts, respectively) was achieved. Although variations in DR instruments, image acquisition parameters, age, gender composition, and other factors may contribute to the model’s slightly superior results in the YHDH cohort, these results still strongly support the stability and excellent performance of our DRN model on external test cohorts.

It is difficult to identify fresh VCFs, particularly differentiating them from old VCFs, based on DR images in clinical practice. Both fresh and old VCFs can result in similar morphological changes in the vertebral body, such as loss of height, wedging, or deformity. These changes can be visually similar, making it difficult to differentiate between the two types VCFs based solely on the limited information of DR images. Therefore, multiple normal vertebral columns and old VCFs were included in this study to improve the robustness of the model and evaluate whether the DRN model’s detection of fresh VCFs would be affected. The results of the subgroup analysis showed that the AUC of the DRN model decreased in the old VCFs group compared to the non-old VCFs group (both in two external test cohorts); however, the p-value between the two subgroups was greater than 0.05 (YHDH, p = 0.067; BMUH, p = 0.051). This indicated that there was no significant difference in the diagnostic performance between the two subgroups, i.e., the diagnostic process of the DRN was not affected by the presence of old VCFs. In addition, as shown in Figure 7, there were no peak responses generated in the area of old VCFs which indicates that the DRN model has learned to distinguish fresh VCFs from normal vertebrae and old VCFs based on specific features.

This study has some limitations. In general, anteroposterior and lateral radiographs are jointly captured for spinal DR interpretation. However, only lateral views are collected and used to identify fresh VCFs in the present study, and the lack of anteroposterior radiographs may have resulted in the loss of some imaging features in the DR images. In addition, the influence of malignant VCFs on the performance of the DRN model and the ability of the DRN model to distinguish between benign and malignant VCFs were not further evaluated. However, noninvasive differentiation of benign versus malignant VCFs is a potentially important clinical application, particularly for patients with VCFs without typical morphological features.