The study was approved by the Institutional Ethics Review Committee, which waived the requirement for informed consent. It involved only anonymous imaging datasets, and no individual patient data or human tissue samples were collected.

A total of 336 patients who underwent parotid gland mastectomy in Nanchong Central Hospital from January 2015 to December 2022 were retrospectively analyzed. The clinicopathological and ultrasound data of the patients were retrospectively analyzed. The inclusion criteria were as follows: 1) patients underwent surgery and pathological results, 2) underwent routine ultrasound examination within 1 week before surgery, and 3) had complete imaging data. The exclusion criteria were as follows: 1) patients receiving neoadjuvant therapy (chemotherapy, immunotherapy, or radiotherapy) and 2) were uncooperative during an ultrasound examination or had an image quality that was poor and not suitable for analysis.

The ultrasound (US) instruments employed to acquire the images used in this study included Mindray Resona 7 (Shenzhen Mindray Bio-Medical Electronics Co., Ltd., Shenzhen, China), Esaote MyLab (Esaote, Genoa, Italy), and linear array probe frequency (8–12 MHz). Considering that the corresponding diagnostic results of transverse and longitudinal assessments may overlap and that the longitudinal section is less disturbed, our study selected the longitudinal section of the tumor.

All images were anonymized and measured using the 3D Slicer software (version 4.10.2, https://www.slicer.org). The measurement was completed by two physicians with more than 5 years of experience in ultrasound diagnosis who were blinded to the pathological results.

The feature extraction was implemented using the open-source python package “Pyradiomics V3.0.1” (http://www.radiomics.io/pyradiomics.html). Then, a total of 474 radiomics features were extracted.

We standardized the gray value of all ultrasound images to ensure that the gray value range was consistent and did not need resampling because ultrasound is a two-dimensional (2D) image and does not contain three-dimensional (3D) spatial information. After we ensured that the mask matched the image, we used the same radiomics feature extractor for feature extraction to ensure that the extracted radiomics feature extraction was consistent and reliable.

Similar methods were used in other studies by our team (9). All clinical features (15) and radiomics features (474)were selected using the Mann–Whitney U test after z-score normalization, and then seven feature selection methods (least absolute shrinkage and selection operator, analysis of variance, mutual information, recursive feature elimination, forward selection, random forest, and logistic regression) were used for further selection. The adjustment parameter (λ) in the LASSO model was selected using fivefold cross-validation, and the grid search method was used to adjust the parameters to obtain the LASSO coefficient spectrum of radiomics and clinical features. Subsequently, 11 modeling algorithms were used to model the features selected by each feature screening method. A total of 77 models were established, and heatmaps showing the area under the curve of each model were obtained.

The fivefold cross-validation method was used to test the best model performance for differentiating benign from MT and differentiating PA from WT; the area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy of the two models were obtained.

The receiver operating characteristic (ROC) curve and decision curve analysis (DCA) were used to evaluate the optimal model for differentiating benign tumors from MT, as well as PA from WT.

Two expert radiologists (with 13 and 20 years of experience in ultrasound diagnosis of parotid gland diseases) studied the images. Neither of the radiologists knew the patients’ clinical features and pathological results. First, 263 cases of benign and 73 cases of malignant lesions were differentiated among all cases, and the benign and malignant cases were determined by ultrasound images. Furthermore, 118 cases of WT and 145 cases of PA in benign lesions were differentiated and diagnosed. If there was any disagreement, both parties reached a consensus through discussion. The results were compared with those of the best model. In case of differences, a consensus was reached through mutual discussion. The results were compared with those of the best model.

Statistical analysis was performed using SPSS 25.0 (IBM, Armonk, NY, USA). First, the Kolmogorov–Smirnov test was used to evaluate whether the count data were normally distributed. The independent samples t-test was used when the normal distribution was met, and the Mann–Whitney U test was used when the normal distribution was not met. Continuous variables were described as mean ± SD, and enumeration data were compared among groups using χ² test or Fisher’s exact probability. ROC curve was used to evaluate the accuracy, sensitivity, specificity, and AUC value of the two best models. p < 0.05 was considered to be statistically significant.

In this study, there were 336 patients with parotid tumors: 263 benign and 73 malignant. In the benign cases, there were 118 cases of WT and 145 cases of PA. The clinical features and ultrasonographic features of benign and malignant parotid tumors are shown in Table 1 . The clinical characteristics and ultrasound manifestations of benign parotid gland tumors—pleomorphic adenoma and Warthin’s tumor—are shown in Table 2 .

A total of 474 radiomics features were extracted as follows: a) Shape 2D-based features (n = 9), b) first-order statistical features (n = 18), c) gray-level co-occurrence matrix (GLCM)-based features (n = 24), d) gray-level dependence matrix (GLDM)-based features (n = 14), e) gray-level run-length matrix (GLRLM)-based features (n = 16), f) gray-level size zone matrix (GLSZM)-based features (n = 16), g) neighboring gray-tone difference matrix (NGTDM)-based features (n = 5), and (h) transform-filtered features (including wavelet) (n = 372).

The optimal λ value of the log (λ) function of MSE was 0.019 by LASSO regression cross-validation, and the LASSO coefficient spectrum was obtained, as shown in Figure 1 .

Based on 77 models integrating radiomics and clinical features, the heatmaps of the area under the curve (as shown in Figure 2 ) showed that the LASSO-BNB model was superior to other models in differentiating benign from malignant parotid lesions (AUC 0.910). Voting ensemble learning modeling algorithm with recursive feature elimination feature selection method (RFE-Voting) showed the best performance in the differential diagnosis of benign parotid PA and WT (AUC 0.962). The LASSO-BNB and RFE-Voting models were modeled using 12 and 13 features, respectively. The detailed features are shown in Supplementary Tables 1 , 2 .

The AUC of LASSO-BNB was 0.910 (CI 0.907–0.914) as shown in Figure 3A , sensitivity was 92.5%, specificity was 66.7%, PPV was 90.4%, NPV was 71.4%, and accuracy was 86.8%. The AUC of RFE-Voting was 0.962 (CI 0.959–0.963) as shown in Figure 4A , sensitivity was 84.0%, specificity was 82.1%, PPV was 80.8%, NPV was 85.2%, and accuracy was 83.0%, as shown in Table 3 .

The ROC curve based on the best model for distinguishing benign and malignant parotid glands (LASSO-BNB model) is shown in Figure 3A . The ROC curve based on the best model for distinguishing benign parotid glands (RFE-Voting model) is shown in Figure 4A . The decision curve for evaluating the model is shown in Figure 3B and Figure 4B .

The accuracy (ACC) of experienced radiologists for distinguishing benign and malignant parotid glands was 61.9%, and for the diagnosis of PA and WT, it was 65.4%.