Pancreatic neuroendocrine tumor (PNET) is a relatively rare pancreatic disorder thought to arise in hormone secretory cells of the islets of Langerhans (1) and ever known as islet cell tumor (2). It consists of about 3%–5% of all the pancreatic neoplasm but predominates human neuroendocrine tumors (3). Additionally, the incidence and prevalence of PNETs are steadily increasing in recent decades (4, 5).

PNETs are characterized by tumor heterogeneity (6), and of which the clinical behavior are relatively indolent but vary dramatically (7). Tumor grade is the crucial determinant of the biological aggressiveness of PNETs. Additionally, it is suggested to be associated with lymph node involvement (7), tumor recurrence (8), and overall prognosis (9). According to the 2010 World Health Organization (WHO) classification criteria (10), tumor grade is defined numerically by the proliferative indicator Ki-67, in which low-grade (grade 1 (G1)) tumors have a Ki-67 index from 0% to 2%, intermediate-grade (G2) tumors have a Ki-67 index from 3% to 20%, and high-grade (G3) tumors have a Ki-67 index greater than 20%. Surgery is thought to be the cornerstone of treatment of PNETs in each stage and the only potential way to cure local PNETs (5, 11). However, different surgical strategies could be applied for PNETs of grades 1 and 2/3. The last but not the least, for advanced PNETs, there are also other treatment options, e.g., somatostatin analog (SSA), targeted therapy, or chemotherapy, based on tumor grades. In short, the WHO tumor grading is the best current tool to predict prognosis, guide therapy selection, and aid surgical decision-making by stratification of PNETs.

Of note, tumor grade is always obtained according to postoperative pathology specimen. Although the preoperative endoscopic ultrasound (EUS)-guided fine-needle aspiration cytology (FNA) is proved to be efficient in diagnosing PNETs, the accuracy in differentiating tumor grade remains challenging, possibly due to limited tissue availability or missing the most mitotically active areas (hot pot) of the tumor. The research of Heidsma et al. showed that tumor grade differentiation could be accurately determined by FNA in only 20%–50% of patients (6, 12). Additionally, EUS-guided fine-needle biopsy (FNB) with thicker tissue biopsy needle was reported to have better performance in tumor grade differentiation, as more tumor tissues could be obtained (13). However, both of them were invasive procedures which largely depended on the operators’ experience (13). Therefore, the effective method of preoperatively predicting the pathologic grade of PNETs is still imperatively needed to help establish individualized therapeutic strategies and aid surgical decision-making.

Several previous studies tried to identify the tumor grade of PNETs by computed tomography (CT), magnetic resonance imaging (MRI), and PET/CT (14–17). Although they provided a noninvasive way to preoperatively predict the aggressiveness of PNETs, the accuracy was limited, as the prediction of the frequently occurring heterogeneous tumor was mainly established based on visual observation rather than quantitative information. Recently, “radiomics” brings a new hope for this problem. It is a method that automatically extract a large number of quantitative features from medical images using data-characterization algorithms, and subsequently identify the most significant radiomic signatures through machine learning methods (18, 19). Therefore, we can realize cancer detection, prediction of clinical outcome, and treatment evaluation as reported previously (20, 21). Additionally, radiomics was reported to be successfully applied in differentiating pathologic grading in patients with clear cell renal cell carcinoma (22), colorectal adenocarcinoma (23), etc. Nevertheless, to the best of our knowledge, a noninvasive optimal combined model to incorporate imaging features with clinical characteristics (such as tumor size and tumor margin status) to predict the pathologic grade of PNETs is extremely limited.

Thus, this work attempted to establish a multimodal artificial intelligence (AI) model that integrates a radiomic signature based on plain CT images with clinical features for noninvasive and preoperative prediction of the pathologic grades of PNETs.

The detailed distribution of clinical characteristics in the G1 group (grade 1) and G2/3 group (grade 2/3) is summarized in Table 1 . The tumor size of PNET in grade 2/3 group was significantly larger than that in grade 1 group (p = 0.028). T stage (T3–T4), clinical TNM stage (IIB and above), Dilated MPD/BD, and poorly defined tumor margin were more frequently detected in patients with grade 2/3 PNETs than those with grade 1 (p = 0.004, p = 0.001, p = 0.02, and p = 0.013, respectively). The consistent results occurred both in the training and validation datasets.

As illustrated in Figure 1 , this study aims to build a radiomic> signature and evaluate the ability of the signature to predict PNET grades. Table 2 shows the features used to build the radiomic signature, that is, the result of feature selection on the training set. We also evaluated the performance of combining the radiomic signature and 4 clinical variables to predict PNET grades. The clinical variables are x ₉ to x ₁₂ in Table 2 .

The linear combination of x ₁ to x ₈ in Table 2 expresses the radiomic signature y. The linear combination is shown in Equations (1) and (2).

where y is the score of group 1 (or 2 or 3) for grade 1, −y is the corresponding score for grade 2/3, x → is an observation comprising the m predictors, s is the kernel scale, β is the bias term, and the vector β contains the coefficients that define an orthogonal vector to the hyperplane,, and μ_i and σ_i are the corresponding weighted mean and weighted standard deviation for the ith predictor (used for standardization). When predicting the result, we inputted [−y, y] into the function softmax to obtain the probabilities that the observer belongs to the positive class (grade 2/3) and the negative class (grade 1).

Next, we trained prediction models based on the radiomic signature and the clinical variables to approximately calculate the value of each unknown variable in Equations (1) and (2). We then validated the performance of these models on the independent validation set. The training results and the validation results are shown in Table 3 . Of note, compared with A (arterial) phase and V (venous) phase, P (plain) phase obtained the best prediction performance for each group in the validation set. What is more, for each phase, we calculated the receiver operating characteristics curves (ROCs) and compared the ROCs of validation using the DeLong’s test method. Figure 2 illustrates the ROC results. It demonstrates that the models based on the radiomic signature combined with clinical data (models based on groups 2 and 3) obtain better prediction results than the models based on the radiomic signature alone (models based on group 1). Additionally, although the indicator values based on group 3 show the highest performance, the model based on the radiomic signature combined with 4 clinical data in group 3 showed no significantly better prediction results in plain phase, compared with that combined with 2 clinical data in group 2 (p < 0.629). As can be seen from Table 3 , the experiments based on plain phase obtained the best prediction performance than other phases. Thus, we also calculated the indicator values of accuracy, sensitivity, and specificity in the experiments of plain phase. Table 4 and Supplementary Figure S2 show the prediction results (on the validation set) as the threshold varied.

Above all, the model based on the radiomic signature of plain phase combined with 2 clinical data (T stage and Dilated MPD/BD) in group 2 obtained the best prediction performance. Although the prediction results of the model based on the radiomic signature combined with 4 clinical data in group 3 seemed a little better, there was no significant differences between the groups 3 and 2 models (p < 0.629). Consider the balance between the convenience and predictive power of the model, we established a novel nomogram to preoperatively predict histologic grade in PNETs based on the radiomic signature of plain phase combined with 2 clinical data (T stage and Dilated MPD/BD) in group 2 ( Figure 3 ).

Figure 3 shows that the nomogram achieves a goodness of fit of 0.868 to the ideal model. Correspondingly, the score for the radiomic signature based on plain phase is

The score for the radiomic signature and clinical data in group 2 based on plain-phase is

The score for the radiomic signature and clinical data in group 3 based on plain phase is

PNETs are relatively rare neoplasms, the incidence of which is about 4–5 individuals per 100,000 annually (27). Nevertheless, PNETs have been increasingly detected and diagnosed in recent decades and currently represent the second most common pancreatic tumor followed by pancreatic adenocarcinoma (28). Most of PNETs carry MEN1, ATRX, or DAXX gene mutations, while approximately 15% activate mammalian target of rapamycin (mTOR) signaling (29, 30). PNETs are heterogenous neoplasms, of which the prognosis varies widely. The current most important prognostic stratification factor is WHO tumor grade classification, which might optimize tailored therapeutic strategies. So far, tumor grade is obtained by postoperative pathology. The preoperative fine-needle aspiration (FNA) is invasive, and the accuracy in differentiating tumor grade remains challenging. In the present study, we establish a combined nomogram that integrates a radiomic signature based on plain CT images with clinical features for noninvasive and preoperative prediction of pathologic grades of PNETs with high accuracy.

Firstly, we build a model based on an eight-feature radiomic signature to stratify PNET patients into G1 and G2/3 groups with an AUC of 0.911 (95% CI, 0.908–0.914) and 0.837 (95% CI, 0.827–0.847) in the training and validation cohorts, respectively. Moreover, we identified some objective clinical features (including T stage and dilated main pancreatic duct/bile duct status) related to tumor grade. Interestingly, the predictive performance was further improved by combining the radiomic signature with the clinical features mentioned above as a combined nomogram, achieving an AUC of 0.919 (95% CI, 0.916–0.922) and 0.875 (95% CI, 0.867–0.883) in the training and validation cohorts, respectively.

Recent developments in radiomics attract much interests in tumor detection, subtype classification, therapeutic response assessment, prediction of clinical outcome and tumor monitoring, etc. Most of them were attempt to stratify the biological behavior and optimize tailored therapeutic strategies for these heterogenous tumors such as PNETs. Traditional radiographic assessment [including CT (15) and MRI (17)] which commonly relies on visual evaluation, was previously reported to predict the biological aggressiveness of PNETs. Moreover, (18)F-FDG-PET/CT and (68)Ga-DOTANOC-PET/CT were reported to be useful in predicting tumor grade (14). However, the results vary a lot and the accuracy remains challenging, as the prediction was mainly established based on visual observation rather than quantitative information.

Radiomics and artificial intelligence (AI) automatically extract high-throughput quantitative image data. Just as limited studies reported previously, it could be more useful for differentiating pathologic grading in patients with PNETs than routine CT image features alone (31, 32). Whereas, combining feature engineering and machine learning is a widely used scheme in radiomics-aided diagnosis (32). Deep learning features are highly versatile, their ability to solve specific problems is relatively weak (33). In contrast, building an interpretable AI model based on feature engineering is relatively easy. The output of the model is expected to be understood by physicians in clinical applications. Nomograms based on linear models intuitively illustrate what drives the recognition in machine learning. We build a nomogram based on the group 2 model in plain phase, as shown in Figure 3 . Wan’s research (34) investigated the performance of the combination of conventional handcrafted and learning-based features in disease recognition. For a specific research question, they emphasised that developing specific feature selection and model optimization approaches was necessary to achieve high accuracy and robustness. Consistent with this, the present paper proposed our optimized approaches according to the PNET grading issue (as illustrated in Figure 1 ).

As depicted in Table 1 , our data showed that tumor size in grade 2/3 group was significantly larger than that in grade 1 group (p = 0.028). Dilated MPD/BD and poorly defined tumor margin were more frequently detected in patients with grade 2/3 PNETs than those with grade 1 (p = 0.02 and p = 0.013, respectively). Consistent results occurred in both training and validation sets in the present study. Moreover, research by Kim and colleague (15) identified the three indentical tumor CT features above as predictors of higher tumor grade of PNETs. Of note, the assessment of these features was relatively objective and the data can be automatically acquired in bulk. On the contrary, the data of tumor T stage, TNM stage, and diagnosis of insulinoma were obtained partly by doctors’ experience, although these features were suggested to be significantly different between grade 2/3 PNETs and grade 1 group in our study ( Table 1 ). Therefore, to improve the predictive performance, we establish a combined nomogram model that integrates radiomic signature with the former three relatively objective clinical features (including tumor size, tumor margin, and dilated main pancreatic duct/bile duct) ( Table 2 ). To our knowledge, our present comprehensive nomogram is the first study that integrates radiomic signature based on plain CT images with objective clinical features for noninvasive and preoperative prediction of pathologic grades for each PNET patients with high accuracy (both in training and independent validation set). Wenjie Liang and colleagues (35) reported a nomogram combining radiomic signature based on contrast-enhanced CT and clinical stage. Plain CT has lower cost and more convenience than contrast-enhanced CT. Also, accurate preoperative TNM staging of the tumor is difficult, as the preoperative assessment of “N” and “M” status remains challenging. Interestingly, Zhang’s research (36) depicted impressive results based on enhanced CT radiomic features with 3D modeling.

As is known to us, a quite different therapeutic strategy could be applied for PNETs of grades 1 and 2/3. For clinical practice, the present combined nomogram may facilitate personalized treatment decisions for each patient with this heterogeneous tumor. It is noninvasive and could identify PNETs of grades 1 and 2/3 with high accuracy preoperatively. According to ENET guidelines in terms of PNET, NF-PNETs of less than 2 cm with grade 1 were optimized candidates for a “wait and see” policy. Moreover, parenchyma-sparing procedure such as enucleation could be an alternative for PNET with grade 1, while radical resection with formal lymphadenectomy was recommended for PNET with grade 2/3. In addition, the therapeutic strategies for the advanced PNETs of graded 1 and 2/3 varied dramatically (palliative surgery, somatostatin analog, targeted therapy, or chemotherapy). Our present combined model may facilitate tailored surgical decisions. Additionally, given the spatial and temporal heterogeneity of the specific tumor, the noninvasive model can be used repeatedly for tumor monitoring (especially for the patient initially recommended to wait and see) and to dynamically optimize therapeutic regimen for patients with advanced PNETs.

A major limitation of the present study was the relatively insufficient sample size. In addition, given that the G3 group was small (approximately 10% of PNETs in our series and as previously reported), our present nomogram model was established to stratify PNET patients into G1 and G2/3 groups. To better optimize personalized therapeutic strategies, a nomogram to separate G2 and G3 groups is further needed to be established based on lager samples. We are trying to collect more cases from multicenters and explore more appropriate methods to conduct further studies. Thirdly, we used single-layer CT image in this study, while 3D modeling may more comprehensively reflect the overall characteristics of the tumor, it is worth exploring whether it can obtain a more powerful predictive capability. On the other hand, manual tumor segmentation for 3D modeling was time consuming, and it was not applied for small tumors without thin-slice CT scans.

The developed combined nomogram that integrates radiomic signature based on plain CT images with clinical features (including T stage and dilated main pancreatic duct/bile duct status) can effectively predict the pathologic grades of PNETs preoperatively with powerful predictive capability. The noninvasive predictive model could assist clinicians to optimize tailored therapeutic strategies and facilitate surgical decision-making for each patient with PNETs in practice. It intuitively illustrates what drives the recognition in the prediction, which is potentially valuable in actual clinical applications and precision medicine in the future.

The original contributions presented in the study are included in the article/ Supplementary Material . Further inquiries can be directed to the corresponding authors.

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements.

XW, J-JQ, DC, X-BL, and N-WK: study concept and design, data analysis and interpretation, and drafting of the manuscript. XW, J-JQ, C-LT, Q-QT, S-JR, and Y-HC: substantial contribution to data acquisition and interpretation and critical revision of the manuscript. XW and J-JQ: substantial contribution to data acquisition and data analysis. All authors: final approval of the manuscript and agreement with all the aspects of the work. All authors contributed to the article and approved the submitted version.

This work was supported in part by research grants from the National Natural Science Foundation of China (82002579), the China Postdoctoral Science Foundation Funded Project (2019M663519), the Science and Technology Support Project of Sichuan Province (2020YFS0262), and the Post-Doctor Research Project, West China Hospital, Sichuan University (2019HXBH044).

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.