From January to July 2023, 6357 ROSE cytological images from 721 patients who underwent transbronchial biopsy were collected at the Tangdu Hospital, Air Force Medical University. Among the 721 patients who underwent bronchoscopic biopsy, the lesions of 390 patients were identified through direct visualization under bronchoscopy, while the lesions of 323 patients were detected using radial-endobronchial ultrasound with guide sheath (r-EBUS-GS). In 8 cases, ultrasound did not detect the lesions, and the blind biopsy approach was employed. 5176 ROSE cytological images from 576 patients, which were collected between January and May 2023, were used to develop the ROSE AI system and test the system internally, while 1181 images from 145 patients collected from June to July 2023 were used to test the ROSE AI system externally. The former was called the AI-develop group, while the latter was called the AI-test group. The flow chart of the study design is shown in Figure 1 .

The study was approved by the regional ethics committee of Tangdu Hospital, Air Force Medical University (TDLL-202311-03).

Patient preparation: All patients received local anesthesia using 2% lidocaine spray before undergoing FB. A preoperative assessment was conducted to rule out contraindications. The FB used were Olympus BF-260, BF-P260, and BF-P290. Different bronchoscopes were used according to their availability. Transbronchial biopsy: First, a flexible bronchoscope provided visualization and access to the bronchus. Subsequently, a gentle but precise tissue sampling was performed by opening and closing the biopsy forcesp, capturing a small segment of the lesion. Five to six biopsy specimens were taken from each lesion site. ROSE smear: ROSE slide preparation was conducted by a proficient and professionally trained cytotechnologist based on methods reported in the literature (18) for each specimen. Each biopsy specimen was spread in a concentric circle with a diameter of 1 centimeter on a sterile cytology slide. The slides were then air-dried and stained using a Diff-Quik stain kit (immersion in A solution for 30 seconds, rinsing with phosphate-buffered saline (PBS), immersion in B solution for 20 seconds, rinsing with PBS), and finally observed under an Olympus BX43 microscope. The remaining tissue after preparation of the ROSE slides was placed in tissue fixative fluid and sent for pathological examination.

All available ROSE slides of the included patients were photographed with a camera (Olympus DP74) mounted on the microscope (Olympus BX43) at 200X magnification. All the images were labeled by an experienced cytopathologist as cancer or non-cancer to indicate whether there exist malignant cells in the images under the guidance of the final pathological diagnoses. To ensure the accuracy of the annotation results, the label for one image was accepted only when the annotation by ROSE cytopathologist was consistent with the final pathological diagnosis. Otherwise, the image was removed from the dataset.

After data annotation, all the images in the AI-develop group were mixed, and randomly assigned to training dataset, validation dataset, or internal testing datasets with a probability of 0.8, 0.1, and 0.1 respectively. All the images in the AI-test group were considered as the external testing dataset. The details for the datasets are shown in Table 1 .

Both the internal and external testing dataset were also used to compare the performance of the ROSE AI system with cytopathologists.

Design ROSE AI model. The basic function of the ROSE AI system is to verify whether a ROSE cytological image contains malignant cells or not. We trained an AI classification model to identify whether there existed malignant cells in a given ROSE cytological image. Several classification models using deep convolutional neural network (DCNN) have been trained on the training dataset, and the ResNet101 (19) (implemented with Pytorch 1.13.1), which achieved the best performance on the validation dataset was selected finally.

Train ROSE AI model. The ROSE AI model was trained on two NVIDIA 3090Ti GPUs with the training dataset. All images in the training dataset were resized into 1024x1024 pixels before feeding into the classification model. The data augmentation strategies, which include random flipping, random rotation, and random center cropping, were used to increase the amount of training data to boost the training performance of the system. We used the cross entropy loss function to provide feedback on the model. Label smoothing, a regularization technique, was used to prevent overfitting and improve the generalization performance. The AI ROSE model was trained for a total cycle of 200 epochs (Adam optimizer with a learning rate of 0.0001 and batch size of 8). After each epoch, the trained model was tested on the validation dataset. The model which achieved the highest accuracy on the validation dataset was selected as the final AI model.

The performance of identifying lung malignant cells of the ROSE AI system was firstly accessed on the internal testing dataset and the external testing dataset. The classification result of the ROSE AI system for each image was validated with the annotation. And then the accuracy, sensitivity, specificity, PPV, and NPV were analyzed. An experienced cytopathologist was asked to identify the existence of malignant cells in the same dataset, without details regarding the clinical characteristics, endoscopic features, and pathological results presented. The performance of the cytopathologist was compared with our ROSE AI system.

Then the ability of the ROSE AI system to diagnose lung cancer based on cytological images was also evaluated. The ROSE AI system would diagnose a patient with cancer when at least one cytological image was identified containing malignant cells by the ROSE AI system. The experienced cytopathologist provided a comprehensive diagnosis based on all ROSE cytological images of the same patient. Both the judgment of the ROSE AI system and the cytopathologist for each patient were validated with the annotation, and the accuracy, sensitivity, specificity, PPV, and NPV were analyzed.

The accuracy, sensitivity, specificity, PPV, and NPV were computed to measure the performance of the ROSE AI system and cytopathologist at the per-image level and per-patient level using Python (version 3.9.13). The consistency of the ROSE AI system and the cytopathologist was evaluated with the Kappa test using SPSS (version 26.0).

From January 2023 to July 2023, 721 patients (shown in Table 1 ) diagnosed with lung lesions based on chest computed tomography (CT) underwent FB with ROSE. A total of 751 lesions were biopsied from 721 patients. Among these, 582 (77.50%) lesions were located peripherally, while 169 (22.50%) lesions were located centrally. 6357 ROSE images were taken for all the patients, with 8.82 ± 3.20 images for each patient. The mean age of the patients was 60.40 ± 11.74, and 69.49% of patients (501) were male. Among all the patients, 61.72% of patients (445) were diagnosed with cancer, and 38.28% of patients (276) were diagnosed with non-cancerous conditions. The patient characteristics in each dataset were analyzed (shown in Table 1 ).

Recognizing the significance of assessing the ROSE AI system’s robustness across diverse settings and patient populations, we conducted a detailed analysis focusing on patients in the AI-test group. Among the 145 patients examined, 137 underwent biopsy for a single lesion, while 7 patients underwent biopsies for two lesions, and 1 patient underwent biopsies for three lesions. Of the total 154 lesions examined, 114 (74.03%) were located peripherally, while 40 (25.97%) were located centrally. Furthermore, among the 145 patients, 79 (54.48%) had lesions identified through direct visualization under bronchoscopy, while 63 (43.45%) had lesions detected using radial-endobronchial ultrasound with a guide sheath (r-EBUS-GS). In 3 (2.07%) cases, ultrasounds failed to detect the lesions, necessitating the use of a blind biopsy approach.

The final pathological diagnoses of patients were presented in Table 2 . In the AI-develop group, 229 patients were diagnosed as non-cancerous, 108 patients with squamous cell carcinoma, 109 patients with adenocarcinoma, and 59 patients with small cell lung cancer. Additionally, specimens from 55 patients in the AI-develop group were histopathologically diagnosed as malignant, but a specific subtype was not identified due to the lack of follow-up immunohistochemistry or unavailable follow-up information. These cases were categorized as “no specific subtype.” Furthermore, 16 patients in the AI-develop group were categorized as “others”, including adenoid cystic carcinoma (1 patient), B-cell lymphoma (1 patient), adenocarcinoma combined with small cell lung cancer (1 patient), breast cancer metastasis (1 patient), mesothelioma (3 patients), sarcoma (1 patient), adenosquamous carcinoma (2 patients), large cell carcinoma (1 patient), squamous cell carcinoma combined with neuroendocrine carcinoma (1 patient), cervical cancer lung metastasis (1 patient), renal malignant tumor metastasis (1 patient), thyroid cancer metastasis (1 patient), and classical Hodgkin lymphoma (1 patient). The final pathological diagnoses for AI-test group patients were also presented in Table 2 , indicating 47 non-cancerous, 31 squamous cell carcinoma, 33 adenocarcinoma, 19 small cell lung cancer, and 15 no specific subtype.

The trained AI ROSE classification model was evaluated on the internal testing dataset and the external testing dataset. For each ROSE image, the ROSE AI classification model determined whether it contained malignant cells or not. The performance of ROSE AI classification was measured with the accuracy, sensitivity, specificity, PPV, and NPV. The specificity-sensitivity curves on both internal and external testing datasets are shown in Figure 2 .

A classification confidence threshold of 0.5 was selected to compute the final accuracy, sensitivity, specificity, PPV, and NPV on both internal and external testing datasets, the results are shown in Table 3 . The accuracy, sensitivity, specificity, PPV, and NPV of the ROSE AI classification in classifying cancer images were 92.97%, 94.40%, 90.37%, 94.67%, and 89.89% respectively on the internal testing dataset, and 90.26%, 90.76%, 88.89%, 95.74%, and 77.78% respectively on the external testing dataset. Meanwhile, the performance of an experienced cytopathologist in classifying cancer images was 97.15% (accuracy), 100.00% (sensitivity), 91.98% (specificity), 95.76% (PPV), and 100.00% (NPV) on the internal testing dataset, and 94.83% (accuracy), 94.69% (sensitivity), 95.24% (specificity), 98.20% (PPV), and 86.71% (NPV) on the external testing dataset. The consistency between the ROSE AI system and the experienced cytopathologist was good ( κ = 0.798 on the internal testing dataset, κ = 0.673 on the external testing dataset). Collectively, these results indicated that the ROSE AI classification model can identify whether there were malignant cells in ROSE cytological images with high accuracy and substantial robustness.

We next evaluated the performance of the ROSE AI system in identifying lung cancer in ROSE cytological images at the per-patient level. A patient was diagnosed with lung cancer by the ROSE AI system when at least one ROSE cytological image of the patient was classified as a cancer image (with a confidence threshold of 0.55). Representative predictions of the ROSE AI system are shown in Table 4 . The results showed that the ROSE AI system identified lung cancer with an accuracy of 89.61%, sensitivity of 90.57%, specificity of 88.00%, PPV of 92.75%, and NPV of 84.62% on the internal testing dataset, and accuracy of 87.59%, sensitivity of 94.90%, specificity of 72.34%, PPV of 87.74%, and NPV of 87.18% on the external testing dataset.

We also asked an experienced cytopathologist to diagnose the patients based on the same ROSE cytological images dataset. The experienced cytopathologist provided a comprehensive diagnosis based on all ROSE cytological images of one patient. The accuracy, sensitivity, specificity, PPV, and NPV of the experienced cytopathologist in diagnosing lung cancer were 97.03%, 100.00%, 92.00%, 95.50%, and 100.00% respectively on the internal testing dataset, and 95.17%, 96.94%, 91.49%, 95.96%, and 93.48% respectively on the external testing dataset. The consistency between the ROSE AI system and experienced cytopathologist was also analyzed with the Kappa test. The Kappa values on internal and external testing datasets were 0.738 and 0.618, indicating good consistency between the two diagnoses.

Despite the overall good consistency observed between the ROSE AI system and the experienced cytopathologist, there were still significant instances where interpretations diverged between the two. To further scrutinize the factors contributing to disparities between the ROSE AI system and cytopathologist’s interpretations, we conducted an analysis specifically focusing on cases where inconsistencies between the ROSE AI system and cytopathologists emerged. The detailed findings are presented in Table 5 . Examining Table 5 revealed that in the internal testing dataset, the predominant instances of discrepancies between the ROSE AI system and pathologists were associated with diagnoses such as “non-cancer” (15.51%), “no specific subtype” (8.89%), “small cell lung cancer” (8.77%), and others (7.69%). Similarly, in the external testing dataset, the most notable instances of discrepancies encompassed “no specific subtype” (22.73%), “non-cancer” (15.87%), and “small cell lung cancer” (13.59%).

To further evaluate the efficacy of the ROSE-AI system, we conducted a detailed analysis of its performance in identifying various types of tumors as malignant, with a particular focus on non-primary lung tumors and less prevalent malignant lung tumors. The results are comprehensively outlined in Table 6 . Patients with final pathological diagnoses of squamous cell carcinoma, adenocarcinoma, and small cell lung cancer were classified under the “common” group. Meanwhile, those with less prevalent malignant lung tumors and non-primary lung tumors were grouped under “others”, and patients pathologically diagnosed as malignant but without a specific subtype were categorized under “no specific subtype.” Since all patients in these three groups were diagnosed with malignant tumors, sensitivity equaled accuracy, and specificity, PPV, and NPV were 0%, 100%, and 0%, respectively. Consequently, we solely presented accuracy as the performance metric in Table 6 . In the internal testing dataset, the accuracy for the “common,” “others,” and “no specific subtype” groups was 91.81%, 88.89%, and 84.38%, respectively. For the external testing dataset, no patients were diagnosed with the “others” category, and as such, no accuracy is reported for this group. The accuracy for the remaining two groups in the external testing dataset was 96.39% for the “common” group and 86.67% for the “no specific subtype” group. The Kappa test was subsequently administered to the “common” group in both the internal and external testing datasets, yielding Kappa values of 0.930 and 0.932, respectively. These results suggest almost perfect consistency between the diagnoses provided by the ROSE AI system and those of experienced cytopathologist for common lung malignant tumors.