We collected 102 patients with pathologically confirmed primary LC who were confirmed to have synchronous or metachronous bone metastases and 100 patients who were confirmed to not have bone metastases by CT-guided biopsy pathology from June 2012 to May 2022. After reviewing the clinical and imaging data, we excluded patients who did not undergo CT examination (n = 53); who underwent surgery, chemotherapy, and radiation therapy for bone metastases before CT examination (n = 9); and who had poor CT image quality (n = 14). Finally, the CT images from 126 patients were included for the DCNN model development to detect bone metastases from LC, including a positive sample dataset of patients with biopsy-proven LC and bone metastases (n = 57), and a negative sample dataset of patients with biopsy-proven LC without bone metastases (n = 69). The process of patient enrollment is shown in Figure 1 .

We randomly split the whole dataset (n = 126) into three cohorts: training (76 cases, to train the DCNN model), internal validation (12 cases, to fine-tune the hyper-parameters of the DCNN model), and external testing (38 cases, to evaluate the model and radiologists’ performance). Furthermore, the patients included in the validation and testing datasets were excluded from the training dataset.

The clinical and imaging information of all patients was obtained through the medical record system and follow-up. To protect patients’ privacy, all identifying information, such as name, sex, age, and ID, on CT was anonymized and omitted through image processing when data were first acquired. After image preprocessing in the Digital Imaging and Communications in Medicine (DICOM) format, complete thin-layer CT images were stored (see Supplementary Methods for CT protocols). The manual annotations of bone metastases were performed using LabelMe (36) with an image segmentation software (Mimics; Materialize, Belgium).

Two board-certified radiologists (8 and 9 years of experience in CT diagnosis) evaluated all CT examinations and, section by section, manually annotated five locations (spine, pelvis, limb, sternum, and clavicle) on the CT images of patients with LCBM. Two expert radiologists (14 and 20 years of experience in CT diagnosis, respectively) checked and manually delineated the volume of interest of the bone metastatic lesion in a voxel-wise manner on CT images using the diagnosis reports from two board-certified radiologists for establishing the reference standard of bone metastases. Furthermore, both of them repeated the annotations and modifications at least 3 weeks later and used them as ground truth (GT) labels for diagnosis and evaluation.

We developed a cascaded three-dimensional (3D) U-Net with 3D spatial SE modules and 3D GAU modules based on 3D U-Net (37), which is a robust state-of-the-art DCNN-based medical image segmentation method (see Supplementary Methods for training protocol). The Cascaded 3D U-Net (38) contains two 3D U-net architectures, wherein the first one is trained on down-sampled images and the second one is trained on full-resolution images. Training on down-sampled images first can enlarge the size of patches concerning the image and also enable the 3D U-Net network to learn more contextual information. Training on full-resolution images next refines the segmentation results predicted from the former 3D U-Net.

The 3D Spatial SE (39) module and 3D GAU (40) module are used more fully as the spatial attention module and the channel attention module to exploit the multiscale and multilevel features, respectively; they guide DCNN to efficiently focus on the targets rather than the background. The flowchart of our DCNN model for segmenting bone metastases is shown in Figure 2 . The input of the end-to-end DCNN model is the 3D CT volume, and the output is the segmentation result of the bone metastases margin and the possibility of LCBM.

To scale the proposed deep learning system with human readers, five radiologists (no overlap with the radiologists who labeled and checked the annotations) with 3–8 years of experience in CT diagnosis were required to participate in an independent human-only observer study. These radiologists were randomly shown the testing dataset to independently segment bone metastases and record the localization (corresponding CT layers); they were blinded to the bone metastases results and patient information.

To simulate the real clinical scenario, a radiologists-DCNN model collaboration study was conducted on three junior radiologists with 1–3 years of CT diagnostic experience, besides the independent observer study. These radiologists independently assessed the images to reach the first conclusion (whether bone metastases are present). Readouts of the DCNN model, including lesion labeling and probability of LCBM, were sent to the radiologists for reevaluating the images. The second assessment of radiologists served as the final output. In addition, we scheduled the 8 radiologists to perform the test in different locations to ensure their relative independence.

Our method followed a segmentation methodology to perform a detection task; therefore, both segmentation and detection metrics were important for evaluating the DCNN model performance. The metastases segmentation performance of the network was assessed using the metrics of dice coefficient and Intersection-over-Union (IoU) (41). The dice value indicates the overlapped voxels between the predicted results and GT. Its mathematical definition is as follows:

where denotes the number of labeled voxels, and P and G represent the predicted and GT values, respectively. The larger the value of the dice, the higher the degree of overlap between the segmentation prediction and GTs.

The evaluation of detection performance was based on case-based analysis; cases with at least one positive lesion were considered true positive (TP). We compared the model predictions with radiologists with TPs, false negatives (FNs), and false positives (FPs). According to the TP rate (sensitivity) versus the FP rate (1-specificity), we calculated the areas under the receiver operating characteristic (ROC) curve (AUC) and the 95% confidence interval (CI) for the five radiologists (averaged), DCNN model alone, three junior radiologists alone (averaged), and DCNN-assisted three junior radiologists. P< 0.05 was considered statistically significant. All statistical analyses were performed using SPSS 22.0 (IBM Corp., Armonk, NY, USA). The interpretation time for each scan was recorded automatically by the viewer and the detection time of the DCNN model was obtained from the computer terminal.

A threshold of 0.5 (IoU > 0.5) was defined as the detection hit criterion; then, the segmentation metrics were computed with the GT labels generated in the image annotation procedure. At a threshold of 0.5, the detection sensitivity of our DCNN model was 0.898 with 5.23 average FPs for the validation dataset and 0.894 with 5.24 average FPs for the testing dataset. Besides, our DCNN model achieved an acceptable segmentation performance (dice = 0.859 and 0.856 on the validation and testing datasets, respectively). The overall results of the DCNN model for the validation and testing datasets are shown in Table 2 . An illustration of the predicted segmentation by DCNN is shown in Figure 3 , where all cases were predicted with highly similar segmentation to GT.

We compared our model with several state-of-the-art deep neural networks in the validation and testing datasets to validate the effectiveness of the proposed DCNN model ( Figure 4 and Table 2 ). As shown in the results, the Cascaded 3D U-Net outperformed the 3D U-NET and 3D FCN (42) by large margins, which verified the effectiveness of network design in the proposed DCNN model. Moreover, the results of the ablation experiments demonstrated the best performance of our 3D U-Net with a 3D GAU module and 3D spatial SE module.

We performed observer studies and compared them with five radiologists using all images in the testing dataset to characterize the diagnostic value of the DCNN model. The DCNN model achieved high performance, outperforming any radiologists for LCBM detection with respect to both the primary metric AUROC (0.875 vs. 0.819 for the best radiologist) and sensitivity (0.894 vs. 0.892) tested in the observer-independent study. As for segmentation performance, our DCNN model outperformed all other networks and five radiologists ( Table 2 ).

We further validated the radiologists-DCNN model collaboration performance in the testing dataset. As demonstrated, the three junior radiologists (with the assistance of the DCNN model) showed substantial improvement in identifying LCBM ( Figure 4 and Table 3 ).

The DCNN model assisted junior radiologists in diagnosing LCBM with a higher mean AUROC (0.874 [95% CI: 0.807–0.874] vs. 0.609 [95% CI: 0.591–0.634], P< 0.001), mean accuracy (0.879 vs. 0.617, P< 0.001), and mean sensitivity (0.902 vs. 0.680, P = 0.009) compared with those achieved alone. Moreover, the mean interpretation time per case of the junior radiologists was significantly reduced from 370.3 s to 142.3 s (228 s decrease, P = 0.045) when assisted by the DCNN model.