This retrospective study was performed at three endoscopy centers, located in Shanghai General Hospital – South (center 1), Shanghai Jiao Tong University Affiliated Sixth People’s Hospital (center 2) and Shanghai General Hospital – North (center 3). It should be noted that the center 1 and the center 3 are independently operated in different locations, with fixed staff and endoscopy equipment purchased in different years. This study was conducted in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of the Shanghai General Hospital (2020KY236). ME-NBI images from October 2014 to June 2021 were retrospectively collected. Every patient underwent conventional upper gastrointestinal WLE, and then the suspected superficial lesions were carefully examined under ME-NBI. The ME-NBI images were captured by standard endoscopes (GIF-H260Z or GIF-H290Z, Olympus Co., Tokyo, Japan) with an EVIS LUCERA ELITE endoscopic system (CV-290, Olympus Co.) and an endoscopic cold light source (CLV-290SL, Olympus Co.). A black soft hood was attached to the tip of the endoscope to obtain stable images at maximum magnification.

Suspected superficial lesions were confirmed by pathological data obtained from the targeted biopsy samples, endoscopic submucosal dissection (ESD) samples or surgical samples. All pathological data from the three centers were assessed by senior gastrointestinal pathologists according to the Vienna classification (18). Category 3 was defined as LGIN lesions, and Categories 4 and 5 were defined as EGC lesions. LGIN and EGC lesions were defined as GNLs.

To distinguish neoplastic lesions from their background mucosa, ME-NBI images of chronic gastritis and intestinal metaplasia were obtained as background training under the same imaging conditions. Images of esophageal lesions, duodenal lesions, submucosal tumours, those with a lack of pathological data, and images of poor quality (including bleeding, mucous adherence, presence of foreign bodies and out of focus) were excluded from the analysis.

In the ME-NBI images, the following two criteria should be met for labelling superficial gastric lesions as LGIN: (1) there are DLs that can be identified under ME-NBI; (2) the presence of EANA or types IV–VI pit pattern of the Sakaki classification or regular white opaque substance (WOS) or dense-type crypt opening (8–11). The following criteria should be met for labelling superficial gastric lesions as EGC: (1) an irregular MV pattern with DL and (or) (2) an irregular MS pattern with DL (6).

All images were JPEG files with various sizes such as 1020×764 pixels, 716×512 pixels and 764×572 pixels. Baseline data such as the date of diagnosis of GNLs, lesion locations, macroscopic types and maximum diameter, were collected. Then, the redundant parts of the images, such as the patient information and acquisition time, were cropped before labelling. Two senior endoscopists separately labelled the images as “GNLs” or “non-GNLs” according to the pathological data and marked the location of GNLs using rectangular box annotation by LabelImg (version 1.8.6, https://github.com/heartexlabs/labelImg), then saved as XML files. If they were in agreement, the image was considered to be acceptable. If they had any disagreement, then another senior endoscopist made the judgement. An expert endoscopist used polygon annotation to label the extent of the GNLs by Labelme (version 4.2.10, https://github.com/wkentaro/labelme), saved as a JSON file. Before training, the images were resized to 416×416 pixels, and the number of ME-NBI images in the training set was expanded by rotating and flipping. Figure 1 shows a pipeline diagram of this study.

All of the images were divided into the following datasets:

A flowchart of setting the datasets is shown in Figure 2 .

The CAD system consists of CNN1, which is responsible for diagnosis, and CNN2, which is responsible for the lesion segmentation and localisation. CNN1 is based on YOLO V3 (19). EfficientNet B2 was used as the backbone to replace the original DarkNet53 to obtain better feature extraction results ( Figure 3A ). The EfficientNet B2 model was pretrained on the Pascal VOC 2007 dataset and used as the initial weights. CNN2 is based on UNet, and a pretrained VGG-16 on ImageNet was used to replace the backbone of UNet ( Figure 3B ) (20, 21). Feature pyramid networks (FPN) (22) and an attention module (23) were additionally added to improve the feature extraction performance.

To improve the accuracy and the generalisation and to avoid overfitting, the shallow-level convolutional layers of the pretrained DarkNet 53 and VGG-16 were frozen, and the weights of the high-level convolutional layers were updated during the training. The initial image size was 416×416 pixels, 8 images were put into training at a time, and the learning rate was automatically adjusted for the rate of decline. The model was evaluated every 10 iterations to calculate the average accuracy of the validation set and save the weights. The total number of training iterations was 500 in CNN1 and 100 in CNN2. Training details for CNN1 and CNN2 are provided in the Supplementary Material .

In the CNN1 model, the Youden index was used to calculate the threshold for binary classification. If the output confidence value of a neoplastic image is higher than the specified threshold, the image is considered a true positive (TP), and if it is less than the specified threshold, it is considered a false negative (FN). A non-neoplastic image is considered a false positive (FP) if its confidence value is higher than the specified threshold or a true negative (TN) if it is less than the specified threshold.

The diagnostic performance of CNN1 was evaluated by accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), receiver operating characteristic (ROC) curve and area under the curve (AUC).

In the CNN2 model, intersection over union (IOU), precision, recall and the Dice coefficient were used to evaluate the segmentation performance. IOU means the ratio of the intersection and union of the AI segmentation region to the lesion labelled by an expert endoscopist (ground truth). If the IOU between the CNN2 segmentation and the ground truth is higher than a specified threshold, it is considered a TP; otherwise, it is an FP. If IOU = 0, it is considered an FN.

Precision, recall, and the Dice coefficients (24) were calculated as follows:

To test for differences in the diagnostic outcomes between the CAD system and the endoscopists, endoscopists of different experience levels were selected to judge the images in the test set. Three endoscopists were considered senior endoscopists with more than five years of endoscopy experience. Moreover, they have received education and training in identifying EGC under ME-NBI and had participated in EGC screening. Three endoscopists were regarded as junior endoscopists with less than five years of experience in endoscopy and no specific EGC training.

All statistical analyses were conducted using SPSS 22.0 (IBM, Armonk, NY, USA). The 95% confidence intervals (95% CI) were evaluated using the Wilson method. Continuous variables were compared by one-way ANOVA, and categorical variables were compared by the chi-square test or Fisher’s exact test. A chi-square test was conducted to compare the accuracy, sensitivity, specificity, PPV and NPV between CNN1 and the endoscopists. A p value less than 0.05 was considered as statistically significant.

This study included 498 patients diagnosed with GNLs, including LGIN (n=296) and EGC (n=202), who were separated into a training set (n=333), internal test set (n=59) and external test set (n=106). The characteristics of the patients and lesions from the three sets are shown in Table 1 . Patients in all three datasets showed no significant differences in terms of mean age, gender composition, lesion location or macroscopic type, except for a higher proportion of patients with EGC in the external test set. For EGC lesions, there were no significant differences in the mean diameter, depth of infiltration and degree of differentiation among patients from the three sets.

The diagnostic performance of CNN1 was tested in the internal and external test sets. The maximum Youden index is achieved when the classification threshold is set to 0.4920. In the internal test set, CNN1 correctly diagnosed 363 of 400 ME-NBI images with an AUC of 0.928 (95% CI, 0.899-0.956) and an accuracy of 90.8% (95%CI, 87.9%-93.6%). In the external test set, CNN1 diagnosed 1372 of 1600 ME-NBI images with an AUC of 0.918 (95% CI, 0.903-0.932) and an accuracy of 85.8% (95%CI, 84.0%-87.5%). The diagnostic performance of CNN1 is shown in Table 2 , and the ROC curves of the internal and external test sets are shown in Figures 4A, B .

To compare the diagnostic performance between CNN1 and the endoscopists, six endoscopists (three senior endoscopists and three junior endoscopists) who had not participated in the previous work on this study were selected to analyse the 100 images in the comparison test set. The diagnostic performance of GNLs by CNN1 and the endoscopists is shown in Table 3 . The data showed that CNN1 outperformed the senior and junior endoscopists in average diagnostic performance.

Next, the performance of the endoscopists in diagnosing GNLs with CNN1 assistance was further evaluated. The endoscopists rediagnosed the same images with CNN1 assistance at least two weeks after their previous independent diagnosis. The results showed an average increase in diagnostic accuracy of 11.8%, with an average improvement of 10.3% for senior endoscopists and an average of 13.3% for junior endoscopists ( Figure 5A ). Because of differences in their diagnostic style, the improvement in sensitivity after CNN1 assistance varied considerably among the endoscopists, with an average improvement in sensitivity of 18.7%, including 9.3% for senior endoscopists and 28.0% for junior endoscopists ( Figure 5B ). The diagnostic performance of each endoscopist for GNLs is shown in Table 4 .

Specifically, the average accuracy of diagnosing EGC increased 3.5% among the senior endoscopists, which may be because the participating senior endoscopists were already very good at diagnosing EGC under ME-NBI, and it increased 9.7% among the junior endoscopists ( Figure 5C , Table 5 ). The use of CNN1 significantly improved the specificity of EGC diagnosis by senior endoscopists (68.1% to 80.6%) and the sensitivity of EGC diagnosis by junior endoscopists (75.0% to 91.7%) ( Table 5 ). For LGIN lesions, the average accuracy was significantly increased for both senior and junior endoscopists (both at 16.7%) ( Figure 5D , Table 6 ). This suggests that with CNN1 assistance, the endoscopist’s ability to diagnose LGIN is greatly improved and will reduce the rate of missed LGIN diagnoses.

The results show that CNN2 has a better segmentation effect on GNLs. The average IOU between CNN2 and the expert label was 0.584. To assist in the endoscopic resection of GNLs, the IOU threshold was set at 0.5. The precision, recall and Dice coefficient were 0.776, 0.983 and 0.867, respectively. The performance of CNN2 and other segmentation modules was tested to compare the segmentation effect. CNN2 exhibits better segmentation than PSPNET and DEEPLAB in average IOU, precision, recall and Dice coefficient ( Figure 6 ). The segmentation performance of different GNLs is shown in Figure 7 .