This study was approved by the Institutional Review Board and the requirement for informed consent was waived as the data were analyzed retrospectively and anonymously.

A total of 141 solid solitary pulmonary nodules (72 adenocarcinomas and 69 benign nodules) were consecutively included in this study from April 2019 and May 2020, according to the following inclusion criteria: 1) detection of solid solitary pulmonary nodule without calcification for typical benign lesion; 2) LDCT obtained from lung cancer screening; 3) standard-dose CT obtained within 24 h after LDCT to evaluate hilar and mediastinal lymph nodes; 4) pathologically confirmed. The exclusion criteria were as follows: 1) history of cancer in previous 5 years; 2) images of poor quality with respiratory and movement artifacts; 3) nodules with undefined border resulting in poor segmentation.

We divided the nodules into two independent cohorts according to a ratio of 7:3 and the date of inclusion. Fifty adenocarcinomas and 48 benign nodules enrolled between April 2019 and November 2019 constituted the primary cohort, and 22 adenocarcinomas and 21 benign nodules enrolled between November 2019 and May 2020 constituted the validation cohort. The radiologist (HQ, with 7 years of experience in thoracic radiology) who was blinded to the final diagnosis performed categorization on nodules according to Lung-RADS (34).

All LDCT and standard-dose CT scans were performed on a 256-slice multi-detector CT scanner (Brilliance iCT, Philips Healthcare, Amsterdam, Netherlands), using the following acquisition parameters: tube voltage of 100 kV and tube current of 20 or 30 mAs for LDCT, tube voltage of 120 kV and tube current of 100 to 250 mAs for standard-dose CT, standard resolution mode, detector collimation of 128 × 0.625 mm, helical pitch of 0.915, and gantry rotation time of 0.4 s. All the raw datasets were then reconstructed using the hybrid iterative reconstruction method (iDose4, level 6, Philips Healthcare, Amsterdam, Netherlands) with standard reconstruction filter for body, slice thickness of 0.625 mm, slice increment of 0.625 mm, field of view of 350 mm × 350 mm, and matrix of 1,024 × 1,024. The estimated effective dose of LDCT scan for all subjects was 0.68 ± 0.11 mSv (range from 0.40 to 0.93).

All target nodules were automatically detected and segmented using uAI platform (United Imaging Healthcare, Shanghai, China), an artificial intelligence software basing on deep learning method (35, 36). No manual adjustments of the segmentation results were performed to avoid inter-observer and intra-observer variability. The representative segmentation results were shown in Figure 1 . A total of 104 original radiomic features, including first order, shape, and texture features, were extracted from the target nodules using an open-source Python package (PyRadiomics, version 3.0, https://pyradiomics.readthedocs.io) (37). Further details of radiomic features are provided in the Supplementary Material .

The concordance correlation coefficient (CCC) was used to quantify reproducibility of extracted radiomic features between LDCT and standard-dose CT in the combined primary and validation cohorts (38). A radiomic feature with CCC ≥ 0.90 was then defined as a reproducible feature, as previously described (39–41). The percentage of features in each category with a CCC of ≥0.85, ≥0.90, and ≥0.95 was calculated, respectively.

Features were standardized using z-score normalization and then selected to build radiomic signature in the primary cohort of LDCT and standard-dose CT dataset respectively. Firstly, the Mann-Whitney U test was employed to select the features that were statistically different between groups (unadjusted P < 0.05), as the quantitative radiomic features did not have a normal distribution. Secondly, Spearman correlation analysis and minimum redundancy-maximum relevance (mRMR) (42) were sequentially conducted to exclude redundant radiomic features. Highly correlated features (Spearman correlation coefficient >0.9) were excluded and the top ranked 10 features were reserved. Thirdly, the least absolute shrinkage and selection operator (LASSO) method was used to select the most predictive features from the primary cohort (43). Finally, multivariate logistic regression with backward stepwise selection was applied to construct radiomic score (Rad-score), in which the stopping rule was the likelihood ratio test with Akaike’s information criterion (44). Rad-score of each patient in was calculated via a linear combination of the selected features and weighted by the respective coefficients.

Independent factors for differentiating adenocarcinomas from benign nodules among Rad-score and clinical variables were identified by inputting significant variables found using univariate logistic regression analysis. Multivariable logistic regression analysis was applied to build radiomic nomogram for LDCT and standard-dose CT respectively, which was a visualized and individual tool that integrated independent factors to predict the probability of adenocarcinoma in the primary cohort.

The area under the curve (AUC) of receiver operating characteristic (ROC) curve was determined to evaluate the discrimination performance of the radiomic signature and radiomic nomogram of LDCT and standard-dose CT in both primary and validation cohorts. The sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV) were also calculated. To evaluate the calibration performance of radiomic nomogram, calibration curves were plotted. The Hosmer-Lemeshow (H-L) test was performed to assess the goodness-of-fit of radiomic nomogram.

Decision curve analyses were conducted to estimate the clinical utility of the radiomic models and Lung-RADS by calculating the net benefits at a range of threshold probabilities in the combined primary and validation cohorts (45).

Statistical analysis was performed by R software (version 4.0.0, http://www.r-project.org), SPSS software (version 19.0, https://www.ibm.com), and MedCalc (v. 18.21, https://www.medcalc.org). The chi-squared test was used to compare the differences in gender, and group comparisons of age was performed using independent sample t-test. P < 0.05 was considered statistically significant.

The performance of Lung-RADS was also evaluated. The ROC of Lung-RADS was performed in both primary and validation cohorts according to that nodules of category 4A, 4B, and 4X were labeled as malignancy while category 2 and 3 as benign nodules. The corresponding sensitivity, specificity, accuracy, PPV, and NPV were calculated. Then comparisons of AUCs among radiomic models and Lung-RADS were performed using the Delong test in the primary and validation cohorts (46). In these pairwise models with significant difference in AUC, further comparisons of sensitivity and specificity were performed using the McNemar test in the combined cohort (47). Analyses were corrected for multiple comparisons using the false discovery rate (FDR) method (48).

The baseline clinical-pathologic characteristics, including gender, age, Lung-RADS category, and histologic subtype of benign nodules in the primary and validation cohorts are listed in Table 1 . There was no difference in gender between the adenocarcinoma group and the benign group in the primary or validation cohorts. Significant differences were found in age between the two groups in primary and validation cohorts (P = 0.003 and P < 0.0001, respectively).

The reproducibility of radiomic features between LDCT and standard-dose CT regarding different feature categories is presented in the Supplementary Material . Shape and neighboring gray tone difference matrix (NGTDM) features were most reproducible (100%), followed by first order features (n = 16/18, 89%). Besides, 57% gray-level cooccurrence matrix (GLCM, n = 12/21), 50% gray level size zone matrix (GLSZM, n = 8/16), and 43% gray level dependence matrix (GLDM, n = 6/14) features were reproducible. Gray-level run length matrix (GLRLM) features were least reproducible (n = 5/16, 31%).

The process of feature selection is presented in the Supplementary Material . Finally, three features in LDCT (GLCM_DifferenceVariance, GLRLM_RunEntropy, and NGTDM_Strength) and two features in standard-dose CT (GLSZM_ZoneEntropy and NGTDM_Strength) were selected in the primary cohort. The representative maps of these texture feature were shown in Figure 1 . The calculation formulas of Rad-score basing on these features with nonzero coefficients are presented in the Supplementary Material . Distributions of the Rad-score in the adenocarcinoma and benign groups in the primary and validation cohorts are shown in the Supplementary Material .

According to univariate logistic regression analysis, age, Rad_score of LDCT, and Rad_score of standard-dose CT were significant independent differentiators of adenocarcinomas and benign nodules ( Table 2 ), and they were integrated to develop the radiomic nomograms for predicting the probability of adenocarcinoma of LDCT and standard-dose CT respectively ( Figures 2A, B ). The calculation formulas of radiomic nomogram are presented in the Supplementary Material .

The ROC curves of radiomic models and Lung-RADS are shown in Figure 3 . The AUC, sensitivity, specificity, accuracy, PPV, and NPV of each model are shown in Table 3 .

The calibration curves of the radiomics nomogram of LDCT and standard-dose CT for the probability of adenocarcinoma demonstrated good agreement between prediction and observation in the primary and validation cohorts ( Figures 2C, D ). The H-L test yielded non-significant results in the both primary and validation cohorts of LDCT (P = 0.650 and 0.998) and standard-dose CT (P = 0.151 and 0.988), which suggested no departure from a perfect fit.

According to the DeLong test, the AUCs of the radiomic models were higher than that of Lung-RADS in the primary and validation cohorts (all correct P < 0.05), while there were no significant differences among the radiomic models ( Table 4 ). The McNemar test results further showed the radiomic models had higher specificity than Lung-RADS in the combined cohort (all correct P < 0.05), while there were no significant differences of sensitivity between Lung-RADS and radiomic models ( Table 5 ).

The results of decision curve analyses for the radiomic models and Lung-RADS are presented in Figure 4 . The decision curves showed that the model of radiomic signature of low-dose CT, radiomic nomogram of low-dose CT, radiomic signature of standard-dose CT, and radiomic nomogram of standard-dose CT added more net benefit than Lung-RADS in differentiating adenocarcinomas from benign nodules within the range of the threshold probability of 0.02 to 0.84, 0.02 to 0.85, 0.02 to 0.74, and 0.02 to 0.79, respectively.