This single-center study protocol was approved by the Affiliated Tumor Hospital of the Zhengzhou University Medical Ethics Committee (Ethics Approval Number: 2021-KY-0022). The requirement of obtaining informed consent from participating patients was waived because of the retrospective nature of this study. Database were retrospectively collected from 736 patients, with one or more GGNs detected on a thoracic CT from January 2017 to March 2021.

The inclusion criteria were (1) patient reported as having one or more GGNs detected on a thoracic CT, (2) GGNs were stable or increased in size after follow-up at two or more years, (3) defined ground truth owing to accompaniment of clinical symptoms or severe patient anxiety, and (4) patient had no history of or currently known extra-thoracic malignancies. The exclusion criteria were (1) patients without pathological diagnoses or patients with no follow-up, (2) rejection of chest CT by DL owing to incompatible image parameters (i.e., CT slice thickness >5 mm or poor image quality), and 3) patients with lung or other site infections. The process of selecting the study population is illustrated in Figure 1. The enrolled patients were randomly divided into a training set (205 patients, 223 observations) and validation set (301 patients, 328 observations). The CT images from all cases were anonymized, and the clinical data or pathological diagnosis findings were collected.

The CT examinations of all patients were performed using a multi-slice CT scanner (iCT-256, Siemens or LightSpeed-16, GE) with a tube voltage of 120 kVp and tube current of 100–300 mA. The pixel spacing of the CT images ranged from 0.625 to 0.867 mm, depending on the patient size, and the reconstruction slice thickness was 1 mm. Each CT image was reconstructed in an image matrix of 512 × 512 pixels. Unenhanced spiral acquisitions were obtained with a breath-hold from the thoracic inlet to the lung bases with images. These images were reconstructed using a standard algorithm. A non-ionic contrast agent was used for the multi-phase enhanced scanning process in 76 patients, and a high-pressure bolus was injected through the elbow vein at a rate of 1.8–2.5 mL/s. The dosage of the contrast agent was 1.5–2.0 mL/kg, and the flow was 2–3 mL/s. Each GGN on the CT, along with its multiplanar reconstruction, was independently interpreted by three thoracic radiologists with 7, 10, and 15 years of experience, respectively, they were not privy of the pathological results. Considering cases of disagreement on the cLung-RADS 1.1 categories among the three radiologists, the images were re-reviewed together, and a consensus categorization was achieved.

The framework of commercial DL was based on multi-stage three-dimensional deep convolutional neural network (3D-DCNN) algorithms (12). The DL algorithm of the lung nodule diagnosis model comprises two stages (Figure 2): a nodule detection stage and nodule classification stages. The first stage extracts high-quality nodule proposals based on the 3D ResNet model and faster convolutional neural network detector, whereas the second stage employs a false positive reduction network (referred to as FPRNet-101) for precise lung nodule classification. Combining both stages, the lung nodule diagnosis model achieved state-of-the-art performance and was endowed with human domain knowledge, resulting in more precise, powerful, and understandable diagnoses. The malignancy scores of GGNs using the commercial DL approach were classified as low (<50%), medium (50–70%), and high (70–100%). Moreover, nodules with medium or high malignant risk scores were defined as positive nodules.

The cLung-RADS 1.1 was redesigned according to the GGN-vessel relationships (GVR) which were categorized into four different types according to imaging features. We identified the type I of GVR and size <30 mm as Lung-RADS 2, type I of GVR and size≥30 mm or type II of GVR as Lung-RADS 2, and any sizes with GGN of type III as Lung-RADS 4a and with type IV as Lung-RADS 4b, Category 3 or 4 nodules with additional features or imaging findings that increased the suspicion of malignancy were defined as Lung-RADS 4x, detailed in Table 2 and the reference (13), and therefore a novel DL-based cLung-RADS 1.1 model was developed to incorporate the additional information provided by the DL risk scores. A management strategy was developed to leverage the increased diagnostic accuracy achieved by the artificial intelligence (AI) algorithm. Pulmonary GGNs that were initially classified as category 3, 4A, or 4B of cLung-RADS, however, they were considered as sufficiently middle- or high-risk by the DL algorithm (i.e., with a DL risk score above the operating point defined to match the average sensitivity of the three radiologists applying cLung-RADS 1.1), were upgraded separately to categories 4A, 4B, and 4X, respectively. This was done to either reduce the follow-up period of low-dose CT (LDCT) scanning or to indicate the optimal operating time. Lung GGNs initially classified as category 3, 4A, or 4 B and deemed to be low -risk by the DL algorithm (that is, with a DL risk score below the chosen operating point) were maintained as category 3, 4A, or 4B, respectively. Pulmonary GGNs for which the cLung-RADS 1.1 classifications were considered concordant (category 2 or 4X with any DL risk score) were managed based on the initial cLung-RADS 1.1 classification. There was no change in management based on the DL-informed management strategy, shown in Table 3. A risk stratification of more than or equal to category 4A based on the DL-based cLung-RADS 1.1 for GGNs was defined as the neoplasm point in clinical scenarios.

Statistical analyses were performed using the SPSS software (version 24; IBM Corporation). Data were reported as mean ± standard deviation (SD). An independent t-test was used to compare the quantitative data. Counting data were described in terms of frequency and percentage, and comparisons between groups were conducted using the chi-square test. When the expected value was <1 or the pre-test probability was approximately the same as the test level, Fisher's exact test was used instead. The validity and predictive values of cLung-RADS 1.1, DL, and DL-based cLung-RADS 1.1 were calculated for the recall rate, precision, accuracy (ACC=TP+TNTP+FP+TN+FN), per-class F1 score (F1=2×Precision×RecallPrecision+Recall), weighted average F1 score (F1weighted=(1+β2)×Precision×Recall(β2×Precision) +Recall), and Matthews correlation coefficient (MCC=TP×TN-FP×FN(TP+FP)(TP+FN)(TN+FP)(TN+FN)), respectively. Here, TP denotes true positive, FP denotes false positive, TN denotes true negative, FN denotes false negative, precision denotes the precision value (Precision=TPTP+FP), and recall denotes the recall value (Recall=TPTP+FN). To attenuate the influence of false negatives, we set β=0.5 to calculate F1_weighted. The overall performance was evaluated using the area under the receiver operating characteristic curve (AUC) analysis. The statistical significance was set to p < 0.05.

In this study, a dataset of 506 patients with 551 observations was established. Table 1 outlines the baseline patient data. There were six subjects with three nodules, 33 subjects with two nodules, and 467 subjects with only one nodule. Only 66 patients were presented with clinical symptoms. There were 24 (4.4%) non-neoplastic and 527 (95.6%) neoplastic (including atypical adenomatous hyperplasia, adenocarcinoma in situ, minimally invasive adenocarcinoma, and invasive adenocarcinoma) lesions. Twenty-four participants had a family history of carcinoma, and 30 participants had chronic obstructive pulmonary disease. There were 47 observations with G-V-R type I, 58 lesions with G-V-R type II, 64 with G-V-R type III, and 381 with G-V-R type IV. The CT images are shown in Figure 3. The age distribution of the patients was 56.5 ± 9.5 (mean ± SD) years and 56.3 ± 9.5 years for the training and validation set, respectively. A total of 327 (64.62%) women were included. The follow-up period between the training and validation sets was 43.6 ± 11.3 (mean ± SD) and 43.5 ± 12.1 months, respectively, and 229 patients maintained good compliance with medical advice.

Considering the training set, the DL model yielded a higher accuracy (91.03 vs. 84.30%), recall (99 vs. 97.9%), and F1_weighted (96.64 vs. 95.45%); nevertheless, it had a lower F1 score (89.40 vs. 91.31%) and MCC value (15.64 vs. 20.06%), compared to the cLung-RADS 1.1. The validation set was used to evaluate the performance of both cLung-RADS 1.1 and DL. The 90.55% validation accuracy value and 99% validation recall rate of the DL exceeded those of the cLung-RADS 1.1 (80.49%). Moreover, the DL model achieved a 94.99% F1 score, 95.27% F1_weighted, and 95.45% precision rate, whereas cLung-RADS 1.1 achieved an 88.85% F1 score, a 93.54% F1_weighted, and a 96.96% precision rate. However, the validation MCC value of cLung-RADS 1.1 was higher than that of the DL (19.43 vs. 2.73%). The AUC value of cLung-RADS 1.1 was higher than that of the DL in the training set (0.712 vs. 0.606, respectively) and validation set (0.676 vs. 0.561, respectively) set, as shown in Table 4.

Considering the complementary performance in the risk management of GGNs, a novel DL-based cLung-RADS 1.1 model was developed to incorporate the additional information provided by the DL risk scores (Table 2). Regarding the training set, compared with both cLung-RADS 1.1 and DL, the DL-based cLung-RADS 1.1 model achieved the highest accuracy value (92.38%), F1 score (95.96%), F1_weighted (97.49%), and MCC value (32.43%). Considering the validation set, the DL-based cLung-RADS 1.1 model achieved excellent performance with a 97.33% precision, 91.77% accuracy value, 95.58% F1 score, and 37.15% MCC, although it had a lower recall rate than that of DL (93.89 vs. 94.53%). The DL-based cLung-RADS 1.1 model yielded the highest AUC value of 0.753 (0.526–0.980) and 0.734 (0.585–0.884) in the training and validation sets, respectively (Table 4).