This retrospective study at our institution was approved by the institutional ethics review board, which waived the requirement for patient approval or signed informed consent to review medical images or information. A total of 231 patients underwent pancreatic surgery operation or EUS-guided fine-needle aspiration (EUS-FNA) at our institution (The First Affiliated Hospital of Guangxi Medical University) between January 2013 and October 2023, comprising 127 with pancreatic cancer and 104 with PNET, were retrospectively enrolled. The inclusion and exclusion criteria are outlined as follows.

The inclusion criteria for patients were as follows: (1) underwent preoperative EUS scan of the pancreas meticulously; (2) had pancreatic cancers or PNET confirmed by postoperative pathology or EUS-FNA pathology; and (3) had complete and clear EUS images available before the patient’s preoperative or pathological biopsies. (4) Patients who received no chemotherapy or radiotherapy before EUS. The exclusion criteria for patients were as follows:(1) inability to display the whole lesion; (2) significant motion artifacts or noticeable noise; (3) presence of other types of tumors. The enrolled patients were randomized into a training cohort and a test cohort at a ratio of 7:3. The flowchart for enrolling the study population is shown in Figure 1 .

The study employed the standard dynamic EUS scan procedure using the EU-ME2 (Olympus, Japan) and SU-9000 (FUJIFILM, Japan) devices. A highly experienced EUS specialist with a record of more than 5000 EUS procedures meticulously scanned the entire pancreatic region and obtained high-resolution images of the pancreatic masses. At the same time, to reduce the image bias caused by different devices as much as possible, these images were consistently standardized with a window width of 250 Hounsfield units (HU) and a window level of 125 HU. The imaging records were gathered by retrieving data from our institutional Picture Archive and Communication System (PACS).

This study retrospectively analyzed multiple clinical parameters, such as age, gender, and pathological diagnosis. To enhance the study’s credibility, all EUS images were meticulously examined and evaluated by two proficient UES experts, each with 6–7 years of experience in pancreatic EUS. In the case of disagreement, a consensus would be reached by consultation. Crucially, these experts were blinded to the histopathological and clinical data about the analyzed cases.

This study examined various parameters and features of the pancreatic masses, including its location, echo characteristics, uniformity of the echo, maximum diameter, shape, margin characteristics, and the presence of calcifications or cystic degeneration, via EUS. A detailed explanation of the ultrasonic features can be found in the Supplementary Material . In instances where multiple lesions were present in the pancreas, the analysis primarily concentrated on the largest lesion with confirmed pathology.

The images were stored in Digital Imaging and Communications in Medicine (DICOM) format. Two EUS specialists, each with 6 and 7 years of experience, manually delineated the region of interest (ROI) using a gray-scale EUS image of the most extensive long-axis cross-section layer for pancreatic masses. This segmentation was performed utilizing ITK-SNAP software (version 3.8.1), accessed at http://www.itksnap.org. The specialists were unaware of the patients’ histopathological diagnosis. The lesions were captured carefully along the margins on conventional EUS images, excluding adjacent normal tissue, vessels, bile ducts, and pancreatic ducts. A comprehensive diagram can be found in Figure 2 .

Standardization techniques were employed to preprocess the images and data, thereby guaranteeing the replicability of the results. Both intra-observer and inter-observer replicability were assessed using the intra-class correlation coefficient (ICC). A group of 102 patients, comprising 81 individuals with pancreatic cancer and 21 with PNET, was randomly chosen, and following a two-week interval, the same EUS specialists performed ROI segmentation once more. An ICC value exceeding 0.9 indicated a remarkable level of agreement.

The classification of handcrafted features can be divided into three distinct categories: geometric, intensity, and textural. Geometric features pertain to the three-dimensional morphological attributes of tumors. The intensity features encompass the statistical distribution of voxel intensities within the tumor in the first order. Conversely, textural features describe patterns and higher-order spatial distributions of intensities. This article utilized a range of techniques, including Gray Level Co-occurrence Matrix (GLCM), Gray Level Run Length Matrix (GLRLM), Gray Level Size Zone Matrix (GLSZM), and Neighborhood Gray-level Difference Matrix (NGTDM), to extract texture features. The algorithms employed to extract radiomic features were based on the image biomarker standardization initiative (IBSI) (16).

To ascertain the dependability of the ultrasomics features, we conducted a Mann−Whitney U test to compare the PNET and pancreatic cancer cohorts, followed by feature screening. Subsequently, only ultrasomics features exhibiting p<0.05 were retained for subsequent analysis.

Correlation coefficient screening was conducted utilizing Spearman’s rank correlation coefficient to assess the interrelationship between each ultrasomics feature, aiming to ensure the robustness of the features ( Supplementary Figure 1 ). Any feature with a correlation coefficient greater than 0.9 between any two features was retained with one of them. A greedy recursive deletion method was employed for feature filtration to enhance feature representation, systematically removing the feature with the highest redundancy within the existing set. As a result, a total of 8 ultrasomics features were ultimately preserved.

Finally, the 10-fold cross-validation method was used to identify ultrasomics features with nonzero coefficients using the least absolute shrinkage and selection operator (LASSO) regression model. All ultrasomics feature selection procedures were conducted within the training cohort and subsequently applied to the test cohort. Ultrasomic features with non-zero coefficients were selected for inclusion in the regression model and combined to form an ultrasomics signature. Subsequently, each patient was assigned an ultrasomics score by applying a linear combination of the selected ultrasomics features and their corresponding coefficients. LASSO regression analysis was conducted utilizing the Python scikit-learn package.

The ultimate retained ultrasomics features were utilized in the development of ultrasomics models. To identify a classifier model with optimal tumor data recognition, our study employed eight prominent machine learning algorithms training models, including logistic regression (LR), light gradient boosting machine (LightGBM), multilayer perceptron (MLP), random forest (RF), extra trees, k nearest neighbors (KNN), support vector machine (SVM), and extreme gradient boosting (XGBoost). The definitive ultrasomics models were obtained through 5-fold cross-validation. The diagnostic efficacy of these different machine learning models was assessed through the evaluation of metrics such as the area under the receiver operating characteristic curve (AUC), sensitivity, accuracy, specificity, positive predictive value (PPV), and negative predictive value (NPV). Ultimately, the most optimal ultrasomics model was chosen and defined as the ultrasomics signature model.

Furthermore, we conducted univariate logistic regression analysis on each clinical predictor variable, encompassing both clinical and ultrasonic characteristics. To determine statistically significant clinical-ultrasonic features and establish the clinical-ultrasonic signature model, we performed a multivariate logistic regression. This allowed us to calculate the odds ratio (OR) and 95% confidence interval (CI) for each variable.

The identical ultrasomics signature model was performed to construct the clinical-ultrasonic signature model through the same machine learning algorithm. To ensure equitable comparison, a fixed 5-fold cross-validation and test cohort were utilized. The performance of this clinical-ultrasonic signature model was evaluated using various metrics such as the area under the curve (AUC), accuracy, sensitivity, specificity, PPV, and NPV. Additionally, the net benefit of the clinical-ultrasonic signature model in identifying pancreatic cancer and PNET was quantified through the use of decision curve analysis (DCA), which involved the application of diverse threshold probabilities in both the training and test cohorts.

Finally, an ultrasomics nomogram was developed in the test cohort to intuitively and efficiently evaluate the incremental predictive value of the ultrasomics signature for the clinical-ultrasonic signature. Based on logistic regression analysis, the ultrasomics nomogram was constructed by integrating the ultrasomics signature with the clinical-ultrasonic signature. The consistency between the prediction of the nomogram and the actual observation was compared by calculating the calibration curve, which compared the prediction of the nomogram with the actual observation.

The diagnostic effectiveness of the ultrasomics nomogram was evaluated in both the training and test cohorts by constructing receiver operating characteristic (ROC) curves. Subsequently, the Delong test was conducted to compare the performance of the ultrasomics nomogram, clinical-ultrasonic signature model, and ultrasomics signature model in terms of the AUC.

The calibration efficiency of the ultrasomics nomogram was assessed by constructing calibration curves. At the same time, the Hosmer-Lemeshow (H-L) analytical fit was employed to evaluate the calibration capability of the ultrasomics nomogram. Additionally, mapping DCA was utilized to evaluate the clinical utility of these predictive models.

We compared the clinical and ultrasonic features of the participants utilizing an independent sample t-test, Mann-Whitney U test, or X ² test, where appropriate. The threshold was set at a two-tailed p-value < 0.05, indicating statistical significance. Table 1 shows the baseline clinical and ultrasonic features of the participants in the training and test cohorts respectively.

A total of 231 patients (102 women, 129 men) with pancreatic tumors, comprising 127 patients with pancreatic cancer and 104 patients with PNET, were included in this study, including 161 patients in the training cohort and 70 patients in the testing cohort. The findings illustrated that there was no significant difference in the gender ratio between the patients in the training and test cohorts. All the details of the clinical and ultrasonic characteristics are delineated in Table 1 . The outcomes of this study revealed significant differences in age, maximum diameter, shape, margin characteristics, echo characteristics, and the presence of cystic degeneration between pancreatic cancer patients and PNET patients in the training cohort. However, no significant differences were observed in terms of echo characteristics or uniformity of the echo in the test cohort. The results of univariate and multivariable logistic regression analyses indicated that age, maximum diameter, and shape independently predicted the presence of PNET.

The findings indicated that elderly individuals (OR 0.989; 95% CI 0.985 to 0.992), those with an enlarged maximum diameter (OR 0.990; 95% CI 0.987 to 0.993), and those with an irregular shape (OR 1.242; 95% CI 1.133 to 1.361) were more likely to be diagnosed with pancreatic cancer ( Table 2 ; Figure 3 ).

A comprehensive set of 7 categories and 107 manually derived ultrasomics features were obtained, comprising 18 first-order features, and 14 shape features, while the remaining features were texture features. Exhaustive information regarding these handcrafted features can be found in the Supplementary Materials .

An in-house feature analysis program, PyRadiomics, was utilized to extract all handcrafted features. Supplementary Figure 2 presents the complete set of ultrasomics features along with their corresponding p-value results. Following the downscaling of ultrasomics features and LASSO logistic regression, a total of 6 ultrasomics features with nonzero coefficients were retained. The coefficients and mean standard errors (MSEs) of LASSO regression, resulting from the 10-fold validation, are depicted in Figure 4 .

The coefficients of these retained 6 ultrasomics features are depicted in Figure 5 , while the Ultrasomics score is shown as follows:

The ROC curers and AUCs of the eight ultrasomics machine-learning models obtained with the eight mainstream machine-learning algorithms in the training and test cohorts are presented in Figure 6 . Additionally, more details are shown in Table 3 . Concerning the training cohort, the ExtraTrees model demonstrated superior performance, achieving an AUC of 1.000 and an accuracy of 1.000. Conversely, in the test cohort, the RF model emerged as the most effective ultrasomics model, exhibiting an AUC of 0.649, an accuracy of 0.600, a sensitivity of 0.567, a specificity of 0.775, a PPV of 0.550, and an NPV of 0.620.

The ExtraTrees and XGBoost models presented a tendency toward overfitting in the training and test cohort. Contrarily, the RF model seemed to own better consistency between the training and test cohorts, with a great net benefit as presented in the DCA curve ( Supplementary Figure 3 ). To establish the stability and sustainability of the ultrasomics machine-learning model, the RF model was determined to be the most suitable for subsequent analysis and defined as the ultrasomics signature model. Consequently, RF was selected as the foundational algorithm for developing the clinical-ultrasonic signature.

Both the clinical-ultrasonic signature and ultrasomics signature appear to exhibit overfitting tendencies in the training cohort, as indicated in Table 3 . Nevertheless, these two models demonstrated a good fit in the test cohort. To address this issue, an ultrasomics nomogram model was employed to combine the ultrasomics signature and clinical-ultrasonic signature, resulting in the most optimal performance achieved by constructing a more precise and consistent prediction model. The ROC curves of these three models, namely the clinical-ultrasonic signature model, ultrasomics signature model, and ultrasomics nomogram, are exhibited in Figure 7 for both the training and test cohorts. To assess the effectiveness of these models, the Delong test was conducted. Fortunately, the AUC of the ultrasomics nomogram demonstrated significant superiority over the ultrasomics signature model and clinical-ultrasonic signature model in the testing cohort ( Table 4 ), indicating that this ultrasomics nomogram ( Figure 8 ) exhibited the highest level of diagnostic efficacy.

The calibration curves of the ultrasomics nomogram performed respectable consistency between the predicted and observed PNET in both the training and test cohorts. The results of the H-L test illustrated that the ultrasomics nomogram had greater prediction accuracy than the other models ( Table 4 ). The calibration curves for the training and test cohorts are displayed in Figure 9 .

Additionally, DCA curves were conducted to assess the performance of each model, and the results are depicted in Figure 10 . The ultrasomics nomogram demonstrated a significant net benefit for patients receiving intervention based on its prediction probability compared to the hypothetical scenarios where no prediction model was accessible, such as the treat-all or treat-none schemes. Furthermore, the ultrasomics nomogram exhibited elevated values compared to those of the other signatures in both the training and test cohorts. Consequently, this ultrasomics nomogram has been shown to potentially enhance the clinical efficacy of predicting PNET, before surgery and EUS-FNA procedures.