The local institutional review board approved this retrospective analysis, and the patient had signed the informed consent form. Also, patient records and images (including the source or raw imaging data) were anonymized before image analysis. Anonymized data are available on reasonable request to the corresponding author, and the data collected in the repository will be made accessible to qualified researchers worldwide, based on the recommendations of a scientific committee that will evaluate proposed research projects. The confidentiality of patients' information will be rigorously protected.

We recruited patients with AIS between April 2020 and April 2021 from the eStroke China national thrombolytic and thrombectomy imaging platform. Thirteen subcenters are registered on the platform and upload CTP or PWI images examined from patients with AIS before treatment to the eStroke platform. In addition, clinical information, including age, sex, national institute of health stroke scale (NIHSS), and exact stroke onset time are recorded. In order to align the examination performance among subcenters, we adjusted imaging protocols based on different device types, which are summarized in Table 1. To avoid the bias of the stroke onset time of patients with AIS, the data were collected by neurologists with more than 5 years of clinical experience, and they were recorded fully on the eStroke platform. Patients were recruited into this study based on the following criteria: (1) AIS due to anterior circulation artery (ACA) occlusion; (2) the recorded exact stroke onset time; (3) the recorded time of initial pretreatment imaging; (4) examined CTP or PWI before treatment; and (5) complete clinical information. All patients were anonymously recruited, and they were informed of and agreed to the study. The dataset will be released on the website https://github.com/bianyueyan/CNN-EVT.

According to previous works, the stroke onset time is correlated with CBF/ADC, Tmax, and changes in the benign hypoperfusion, ischemic, and infarct core regions. These regions can be estimated by different thresholds in CBF and Tmax (9, 15). Therefore, three factors including CBF/ADC, Tmax and the region of diseased hemispheres, are correlated with the identification of the stroke onset time. In order to enable to be compatible with both CT and MR examinations, we chose CBF, Tmax and the region of diseased hemispheres as input images. In this study, we constructed three types of classifiers, namely, support vector machine (SVM), random forest (RF), and CNN, to identify patients with AIS within a 6-h window for the treatment of EVT. These classifiers were trained by three channels of images. The first channel was CBF images, the second was Tmax images, and the third was ROI mask, which was one of the regions of CBF < 30%, Tmax > 4 s, Tmax > 6 s, Tmax > 8 s, and Tmax > 10 s.

In order to compare the performance among the different classifiers (SVM, RF, and CNN), each classifier was trained by three channels of images, consisting of CBF, Tmax, and ROI masks of Tmax > 6 s. Meanwhile, for observing the differences from the ROI masks (CBF < 30%, Tmax > 4 s, Tmax > 6 s, Tmax > 8 s, and Tmax > 10 s), the CNN classifier was trained by CBF, Tmax, and each ROI mask. Through the above process, the classifier with the best performance was selected. Finally, we trained the best classifier using CBF, Tmax, and ROI mask, respectively, from CTP and PWI to compare their agreement.

The CTP and PWI of patients with AIS were examined before the treatment, and then intra-phase rigid registration was performed to correct motion artifacts. After this, the images were smoothed using a Gaussian filter with a kernel with a width of 2.5 mm. In order to reduce disturbance of skull and cerebrospinal fluid (CSF), the images were segmented using BET2 (16) and the thresholding method, respectively, and then the ROI was selected while the rest of the image was excluded. Perfusion parameter maps, including CBF, CBV, MTT, and Tmax, were constructed by block-circulant singular value decomposition (bSVD) provided by the eStroke platform. Perfusion parameter maps were resampled to the spacing of 1 mm in the x, y, and z directions to reduce the impact of image resolutions. The resampled images were chosen as the analytical basis of feature extraction, training, and testing datasets.

According to previous studies, ROI masks segmented by different thresholds based on CBF and Tmax express the progression of AIS, which are strongly related to the stroke onset time (3–8). In order to compare their performances in estimating the stroke onset time, we segmented the ROI masks by CBF < 30%, Tmax > 4 s, Tmax > 6 s, Tmax > 8 s, and Tmax > 10 s.

Features for training machine learning methods, including SVM and RF, were generated based on CBF, Tmax images, and ROI masks, which mainly included first-order descriptive statistics, features of shape, gray level co-occurrence matrix (GLCM) features, gray level dependance matrix (GLDM) features, and gray level size zone matrix (GLSZM) features. All of the features are shown in Table 2. They were extracted with the Radiomics module in the 3D Slicer software, version 4.11 (NA-MIC, NAC, BIRN, NCIGT, and the slicer community, USA). After extracting the initial features, the principal component analysis (PCA) approach was applied to reduce dimensionality and decrease the dependance on the number of training data.

We compared the performance of three types of classifiers, namely, SVM, RF, and CNN in identifying a 6-h window for the treatment of EVT. Briefly, SVM is a supervised machine learning algorithm, mainly used to process classification and regression tasks. The objective of SVM is to find a hyperplane in a N-dimensional space that is defined by the number of features in order to classify the dataset (17). RF is an ensemble learning method that can operate a variety of tasks, including regression and classification. It commonly constructs a multitude of decision trees during the training time. In a classification task, RF creates many decision trees on data samples, each of which votes based upon the results of the prediction. Finally, the output of RF means the class selected by the most trees (18). A CNN is a feed-forward neural network, which is used to handle computer vision tasks such as image classification, object detection, and image recognition (19).

In this study, a CNN was constructed based on VGGNet with 2 convolutional blocks (20), which consisted of a structure of eleven layers: an input layer, three convolutional layers, two batch normalization layers, two rectified linear unit (ReLU) layers, a max pooling layer, a fully connected layer, and a soft-max layer, which are shown in Figure 1. According to the previous works, the stroke onset time of patients with AIS was correlated with the severity and range of CBF reduction and Tmax delay. Thus, the input layer in our network was designed as a three-channel layer, which included CBF, Tmax and ROI mask respectively. The CBF and Tmax channels of the input layer can provide the detail features about the severity of CBF reduction and Tmax delay, and the ROI mask channel can present a weight map to express the range of CBF reduction and Tmax delay. The input layer was separated into blocks with the size of 64 × 64 × 64. The convolutional layer contained 16 filters with a receptive field of 5 × 5 × 5 voxels in a one-voxel stride sliding. The batch normalization layer and ReLU layer which followed the convolutional layer, batch-normalized and rectified the feature map. The max pooling layer reduced the number of rectified features, and they were flattened into a single linear vector by the fully connected layer. Finally, the classification was processed in the soft-max layer. Binary cross-entropy loss was used as loss function. Comparing VGGNet with 2 convolutional blocks, the input layer in our network included three channels, and each channel was 3D images. Apart from that, we removed a max-pooling layer in the first convolutional block in order to decrease the loss of the detail features. All classifiers were trained by fivefold cross-validation to avoid overfitting bias.

We computed the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC), which can compare the ability of all classifiers to identify patients with AIS within a 6-h window. To determine the significance of differences among classifiers in the task of identification, we used the DeLong test to compare the AUCs of the classifiers (21). We also computed patient-wise accuracy, sensitivity, specificity, and precision for each classifier. SPSS version 22.0 (IBM, USA) and GraphPad Prism version 6.0 (GraphPad, USA) powered all of the statistical computations, with significance set at p < 0.05.

We recruited 2,500 patients from the eStroke platform; 426 were excluded due to loss of original data, and 922 were excluded because of poor image quality, such as motion artifacts during scanning. Additionally, 775 with an onset time exceeding 24 h were excluded. Finally, a total of 377 patients (263 men and 114 women; mean age = 66.0 ± 11.9 years) were included in this study. The stroke onset time was 6.7 ± 5.7 h (range = 0–24 h). All patients had ACA occlusion. The patients' baseline and NIHSS are listed in Table 3.

Training and testing datasets were selected randomly, which were grouped by the onset time of stroke. Table 4 shows the patient characteristics in the training and testing datasets. All p-values for each patient characteristic between the training and testing datasets were estimated. We observed that all p-values were higher than 0.05, which means that there were no significant differences in each patient characteristic between the training and testing datasets.

Figure 2 shows the ROC curves of the classifiers (SVM, RF, and CNN) for identifying patients with AIS within a 6-h treatment window for EVT. All of the AUCs of the classifiers were higher than 0.76, which was the highest AUC for identifying patients with AIS within a 4.5 h window for rt-PA thrombolysis treatment in a previous study (8). The AUC of RF was the lowest at 0.775 (0.732–0.818), while the AUC of the CNN was the highest at 0.935 (0.893–0.975). The AUC of the CNN was significantly higher than that of the SVM (p = 0.001) and RF (p = 0.001). The AUCs of the classifiers compared with the previous work are depicted in Table 5.

The CNN classifier was trained by CBF, Tmax, and each ROI mask (respectively, CBF < 30%, Tmax > 4 s, Tmax > 6 s, Tmax > 8 s, and Tmax > 10 s), each ROC curve of which is shown in Figure 3. The AUC of Tmax > 6 s was the maximum value (AUC = 0.935), which was significantly higher than that of Tmax > 8 s and Tmax > 10 s (p = 0.017 and p = 0.002, respectively). Although the AUC of Tmax > 6 s was higher than that of Tmax > 4 s, there was no significant difference between them (p = 0.285). Comparing the ROI masks segmented by Tmax, the AUC of CBF < 30% was only 0.796 (0.723–0.867).

We separated the training dataset into two groups (CTP and PWI), and the CNN classifier was trained by CBF, Tmax, and ROI mask of Tmax > 6 s in each group. Figure 4 shows the ROC curves of two groups. The AUCs of the two groups were higher than 0.9, and there was no significant difference between them (p = 0.557).

Figure 5 shows four examples for identifying patients with AIS within a 6-h window for the treatment of EVT using our method. The classifier was CNN-trained by CBF, Tmax, and ROI masks of Tmax > 6 s. The results of the classifier identification were matched with the ground truth, which was the accurate stroke onset time of patients. DWI and FLAIR are listed in Figure 5 for comparison with a previous study (8), which detected patients with AIS within a 4.5 h window for rt-PA thrombolysis treatment using the machine learning method and the imaging biomarker of DWI–FLAIR mismatch.