Gastrointestinal stromal tumors (GISTs) are mesenchymal neoplasms that mostly originate from the gastrointestinal tract with variable malignant potential, which ranges from small lesions with a benign behavior to aggressive sarcomas (1) and account for 1% to 2% of gastrointestinal neoplasms (2). The prevalence of GISTs is about 130 cases per million population (1, 3, 4). Evaluation of malignancy risk of GISTs is mainly based on tumor size, location, and mitotic count through postoperative specimens. These factors are combined in the National Institutes of Health (NIH) risk category criteria (5), which stratify GISTs into four risk categories: very low, low, intermediate, and high-risk tumors. An accurate preoperative categorization of risk classification can provide valuable information for evaluating the adequacy of surgical resection and the need for adjuvant treatment (6, 7).

Contrast-enhanced CT is widely recognized as the main imaging method for the diagnosis, characterization, and evaluation of curative effect in GIST patients (8, 9). In recent years, multiple researches have evaluated the predictive CT imaging features of the risk stratification of GISTs (10–13). However, these subjective assessments are likely affected by the individual experience and heterogeneous definition of imaging features (14–17). Radiomics, which transforms medical images into mineable high-dimensional data, allows to quantify lesion heterogeneity, which cannot be evaluated by the naked eye (18, 19). Several studies have shown that radiomics based on CT scan was of certain value for the prediction of malignancy in GISTs (20–24).

Nevertheless, the radiomics approach depends heavily on handcrafted feature engineering, which is vulnerable to human biases and may result in a high superfluity of information (25). Deep learning, as one of the powerful algorithms of representation learning, has recently been widely applied in the field of diagnostic imaging and prediction owing to their advantages of being fast, accurate, and reproducible (26, 27). Theoretically, the risk stratification of GISTs by deep learning may yield a great diagnostic approach. However, to the best of our knowledge, this is the first-ever study that investigates whether deep learning could be used as a tool to predict risk stratification in GISTs.

Moreover, most of the existing studies assessing the risk stratification of GISTs are based on single-center data, which introduce bias to a model and limit its applicability. In this multicenter study, we further investigate if a quantitative CT-based deep learning approach can objectively predict the risk stratification of GISTs, by developing and validating a deep learning-based model on a large collection of patient data from two different institutions.

The findings of our study show that the DLM could accurately predict the risk classification of GISTs with 0.90 AUROCs in the testing cohort. The performance in the external validation cohort was somewhat weaker but nevertheless very encouraging (AUROCs = 0.81). The performance of our proposed DLM is better than that of the radiomics model in both the testing and external validation cohorts, indicating that the DLM could mine more image features useful for assessing the risk classification in patients with GISTs. Our work represents an improved approach to the assessment of risk stratification based on the CT images from patients GISTs obtained before surgery and significantly improves on current prediction methods that rely on postoperative specimens.

To the best of our knowledge, this is the largest cohort study using deep learning for GIST risk stratifications and the only one distinguishing high-risk GISTs from intermediate-risk to low-risk GISTs. With few exceptions, reported model performance metrics in previous studies were focused on distinguishing low-malignant-potential GISTs (very low risk and low risk) from high-malignant-potential GISTs (intermediate risk and high risk), thus limiting their clinical impact for identifying high-risk GISTs (23). The European Society for Medical Oncology guidelines recommend adjuvant therapy for patients with a significant risk of relapse, with “room for shared decision-making when the risk is intermediate” (30). Joensuu et al. (1) reported that with modified NIH criteria, only high-risk patients might be considered for adjuvant treatment. Therefore, it is important to improve risk assessment in the high-risk GISTs to make more informed treatment decisions. Zhou et al. (10) indicated that the AUROCs of the multinomial logistic regression model for three risk degrees of GISTs (high-risk, intermediate-risk, and low-risk GISTs), established with three subjective CT features, was 0.806 (95% CI: 0.727, 0.885). In our study, the DLM demonstrated the AUROC value of 0.89 (95% CI: 0.83, 0.95) in the testing cohort and 0.85 (95% CI: 0.81, 0.89) in the external validation cohort for differentiating high-risk GISTs from intermediate-risk to low-risk GISTs, showing better performance than the subjective model.

Nevertheless, there exists a performance drop from the primary cohort (training and testing cohorts) to the external validation cohort, especially in the intermediate-malignant class as accuracy from 87% to 75%. Two main factors may account for the decreased performance: 1) for a three-class classification model, we adopted a one-vs.-rest method to evaluated the performance, which means that when the intermediate-malignant GISTs are masked as positive, the remaining two groups are regarded as negative. The sample size of intermediate-malignant GISTs is much less than that of the non-intermediate-malignant data, at around 1:5 in all datasets. This may hinder its performance, as the machine learning algorithms tend to be bias towards the majority class while exhibiting poor performance for the rest of the class. 2) There is a remarkable difference of CT scanner distribution between the acquisition of the primary and external validation cohorts, where future work could include a large variety of images from different CT scanners to further improve its generalizability. Deep learning (31) is a branch of artificial intelligence in which computers are not explicitly programmed but instead perform tasks by analyzing relationships between existing data points. More recently, deep learning algorithm-based image analysis has been applied to establish a direct link between diagnostic images and disease prediction (27, 32, 33). For example, Zhou et al. (34) recently demonstrated that a DLM based on ultrasound (US) images could provide an early diagnostic strategy for lymph node metastasis in patients with breast cancer. Choi et al. (35) showed that the deep learning system performed better than radiologists in the staging of liver fibrosis with CT images. In our study, we showed that a DLM with ResNet-based method was able to predict the risk classification of GISTs. Furthermore, the uninterpretable neural network system with applications in medical imaging is usually dubbed “black box” medicine (36). It is generally difficult to explain the internal relationship between input data and the predictive labels. The method of visualization with CAM can solve this problem by showing the predictive parts of the image. In our study, the output of CAM attention roughly covering the tumor indicates that the model could exactly locate on the tumor and could make a reliable and interpretable decision in the predictive ability.

In early studies, radiomics features were used for risk stratification of GISTs (22, 37–41). Therefore, in addition to deep learning, we also performed diagnosis using radiomics model for comparison. In the current study, 161 radiomics features were selected to build a radiomics model for predicting the risk classification of GISTs, which achieved acceptable performance in the testing (AUROCs = 0.84, 95% CI: 0.76–0.92) and external validation (AUROCs = 0.78, 95% CI: 0.74–0.81) cohorts. Our accuracy was comparable with that of Zhang et al. (21), who reported that the generated radiomics model demonstrated favorable performance for the risk stratifications of GISTs with an AUROC value of 0.809 (95% CI: 0.777–0.841) in the validation cohort. However, the handcrafted radiomics features can only reflect simple features of relatively low order and may lack the specificity to assess the risk classification (42). Notably, the proposed DLM (AUROCs; testing, 0.90; external validation, 0.81) in our study outperformed the radiomics model for risk classification of GISTs.

Our study has several limitations. First, this is a retrospective study, and the data are not balanced for risk stratification. The performance of the DLM may have been better if we had trained the DLM with an ideal training set including a large amount of CT data that were balanced across the different risk stratifications. Second, the DLM is not a fully automated model, as it requires manual tumor segmentation on the CT images. Third, although we performed the clinical validation of the DLM by using relatively large datasets, the generalizability of this assessment tool needs to be evaluated further. Translating technical success to meaningful clinical impact is the next major challenge. Thorough evaluation and further improvement would be required to evaluate the clinical benefits of the DLM in predicting the risk stratification of patients with GISTs.

In conclusion, we developed a DLM for predicting risk stratification on CT images in patients with GISTs. With further validation in a larger population and model calibration, our DLM has great potential to serve as an important decision support tool in clinical applications.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

The studies involving human participants were reviewed and approved by the institutional review board of both Shandong Provincial Hospital and The Affiliated Hospital of Qingdao University. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

BK, XSY, and XW contributed to the conception and design of the study. XSY, HW, SQ, XS, and XXY organized the database. BK, CS, and XW assessed the image feature. QZ, YW, and FS performed the statistical analysis. BK wrote the first draft of the manuscript. XSY, SY, and XW wrote sections of the manuscript. All authors contributed to the article and approved the submitted version.

The present study was supported by a grant from the Taishan Scholars Project (XW), National Natural Science Foundation of China Grant (81871354 and 81571672), and Academic Promotion Programme of Shandong First Medical University (2019QL023).

Authors QZ, YW, and FS were employed by United Imaging Intelligence.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.