Front. Neurol. Frontiers in Neurology Front. Neurol. 1664-2295 Frontiers Media S.A. 10.3389/fneur.2021.734345 Neurology Mini Review Machine Learning in Action: Stroke Diagnosis and Outcome Prediction Mainali Shraddha 1 * Darsie Marin E. 2 3 † Smetana Keaton S. 4 † 1Department of Neurology, Virginia Commonwealth University, Richmond, VA, United States 2Department of Emergency Medicine, University of Wisconsin Hospitals and Clinics, Madison, WI, United States 3Department of Neurological Surgery, University of Wisconsin Hospitals and Clinics, Madison, WI, United States 4Department of Pharmacy, The Ohio State University Wexner Medical Center, Columbus, OH, United States

Edited by: Jiang Li, Geisinger Medical Center, United States

Reviewed by: Feifei Ma, Vall d'Hebron Research Institute (VHIR), Spain; Harshawardhan Deshpande, National Institute on Drug Abuse (NIDA), United States

 *Correspondence: Shraddha Mainali shraddha.mainali@vcuhealth.org

This article was submitted to Stroke, a section of the journal Frontiers in Neurology

†These authors have contributed equally to this work

 06 12 2021 2021 12 734345 01 07 2021 28 10 2021 Copyright © 2021 Mainali, Darsie and Smetana. 2021 Mainali, Darsie and Smetana

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

The application of machine learning has rapidly evolved in medicine over the past decade. In stroke, commercially available machine learning algorithms have already been incorporated into clinical application for rapid diagnosis. The creation and advancement of deep learning techniques have greatly improved clinical utilization of machine learning tools and new algorithms continue to emerge with improved accuracy in stroke diagnosis and outcome prediction. Although imaging-based feature recognition and segmentation have significantly facilitated rapid stroke diagnosis and triaging, stroke prognostication is dependent on a multitude of patient specific as well as clinical factors and hence accurate outcome prediction remains challenging. Despite its vital role in stroke diagnosis and prognostication, it is important to recognize that machine learning output is only as good as the input data and the appropriateness of algorithm applied to any specific data set. Additionally, many studies on machine learning tend to be limited by small sample size and hence concerted efforts to collate data could improve evaluation of future machine learning tools in stroke. In the present state, machine learning technology serves as a helpful and efficient tool for rapid clinical decision making while oversight from clinical experts is still required to address specific aspects not accounted for in an automated algorithm. This article provides an overview of machine learning technology and a tabulated review of pertinent machine learning studies related to stroke diagnosis and outcome prediction.

 machine learning artificial intelligence deep learning stroke diagnosis stroke prognosis stroke outcome prediction machine learning in medical imaging machine learning in medicine Introduction

The term machine learning (ML) was coined by Arthur Samuel in 1959 (1). He investigated two machine learning procedures using the game of checkers and concluded that computers can be programmed quickly to play a better game of checkers than the person who wrote the program. Simply put, machine learning can be defined as a subfield of artificial intelligence (AI) that uses computerized algorithms to automatically improve performance through iterative learning process or experience (i.e., data acquisition) (2). Of late, the field of ML has vastly evolved with the development of various computerized algorithms for pattern recognition and data assimilation to improve predictions, decisions, perceptions, and actions across various fields and serves as an extension to the traditional statistical approaches. In our day-to-day life, a relatable example of ML is the application of spam filters to the 319 billion emails sent and received daily worldwide, of which, nearly 50% can be classified as spam (3). Use of ML technology has made this process efficient and manageable. The ML technology utilizes various methods for automated data analysis including linear and logistic regression models as well as other methods such as the support vector machines (SVM), random forests (RF), classification trees and discriminant analysis that allow combination of features (data points) in a non-linear manner with flexible decision boundaries. The advent of neural networks and deep learning (DL) technology has transformed the field of ML with automatic and efficient feature identification and processing within a covert analytic network, without the need for a priori feature selection. Notably, performance of DL is known to improve with access to larger datasets, whereas classic ML methods tend to plateau at relatively lower performance levels. Hence, in this era of big data where clinicians are constantly inundated with plethora of clinical information, use of DL technology has significnalty enhanced our ability to assimilate the vast amount of clinical data to make expeditious clinical decision.

Stroke is a leading cause of death, disability, and cognitive impairment in the United States (4). According to the 2013 policy statement from the American Heart Association, an estimated 4% of US adults will suffer from a stroke by 2030, accounting for total annual stroke-related medical cost of $240.67 billion by 2030 (5). For ischemic stroke, acute management is highly dependent on prompt diagnosis. According to the current ischemic stroke guidelines, patients are eligible for intravenous thrombolysis up to 4.5 h from symptom onset and endovascular thrombectomy without advanced imaging within 6 h of symptom onset (6–8). For patients presenting between 6 and 24 h of symptom onset (or last known well time), advanced imaging is recommended to assess salvageable penumbra for decisions regarding endovascular therapy (9–11). Similarly for hemorrhagic stroke, timely diagnosis utilizing imaging technology to evaluate the type and etiology of hemorrhage is important in guiding acute treatment decisions. Prompt diagnosis with emergent treatment decision and accurate prognostication is hence the cornerstone of acute stroke management. Over the recent years, a multitude of ML methodologies have been applied to stroke for various purposes, including diagnosis of stroke (12, 13), prediction of stroke symptom onset (14, 15), assessment of stroke severity (16, 17), characterization of clot composition (18), analysis of cerebral edema (19), prediction of hematoma expansion (20), and outcome prediction (21–23). In particular, there has been a rapid increase in the trend of ML application for imaging-based stroke diagnosis and outcome prediction. The Ischemic Stroke Lesion Segmentation Challenge (ISLES: http://www.isles-challenge.org/) provides a global competing platform encouraging teams across the world to develop advanced tools for stroke lesion analysis using ML. In this platform, competitors train their algorithms on a standardized dataset and eventually generate benchmarks for algorithm performance.

Deciding which type of ML to use on a specific dataset depends on factors such as the size of dataset, need for supervision, ability to learn, and the generalizability of the model (24). DL technology such as the deep neural networks has significantly improved the ability for image segmentation, automated featurization (e.g., conversion of raw signal into clinically useful parameter), and multimodal prognostication in stroke; and it is increasingly utilized in stroke-based applications (25–27). For example, DL algorithms can be applied to extract meaningful imaging features for image processing in an increasing order of hierarchical complexity to make predictions, such as the final infarct volume (27). Some commonly used ML types with their respective algorithms and practical examples are outlined in Figures 1–3. In the healthcare setting, supervised and unsupervised algorithms are both commonly used. In this review, we will specifically focus on ML strategies for stroke diagnosis and outcome prediction. Table 1 provides an overview of pertinent studies with use of ML in stroke diagnosis (Section A) and outcome prediction (Section B). A glossary of machine learning terms with brief description is separately provided in Supplementary Table 1.

Supervised learning. In supervised learning, a model is built by labeling images [Subarachnoid Hemorrhage (SAH) and Not Subarachnoid Hemorrhage (Not SAH)], a predictive model is created, and then tested for accuracy in reading unlabeled images (gray box). Source: WesternDigital BLOG.

Unsupervised learning. In unsupervised learning, the machine learning algorithm discovers structures within given data. The initial data is not labeled and a clustering algorithm groups unlabeled data together. Source: WesternDigital BLOG.

Created from the following referenes: Dey (28) Zhou (29) Geron (30).

Studies utilizing machine learning for stroke diagnosis and prediction.

References Study objective ML-based approaches Validation method Sample size Feature Optimal results Optimal ML approach Clinical implications Limitations
Section A: stroke diagnosis
Ischemic stroke
Garca-Terriza et al. (31) Stroke type diagnosis and mortality RF 10-fold cross validation resampling •119 •(AIS 105, ICH 14) •Type of stroke •Mortality •Non-invasive variables (cardiac and pulmonary) •Accuracy •Subtype - 92% •Mortality - 96% - May predict the type of stroke a patient is at risk for and outcomes Data obtained after event to for prediction models but do not include usual risk factors for consideration
Sung et al. (32) Ischemic stroke phenotype* Various models (C4.5, CART, KNN, RF, SVM, LR, with aggregation algorithms 10-fold cross validation 4,640 Clinical notes with preprocessing and MetaMap to identify medical entities +/- NIHSS •Accuracy; kappa •NIHSS + text •(0.489–0.583; 0.272–0.399) •NIHSS •(0.465–0.533; 0.254–0.344) •Text •(0.465–0.533; 0.170–0.328) - Clinical text plus validated scoring tools might aid in phenotyping of stroke •Phenotype based on OCSP definitions,* •Difficult delineating certain phenotypes, •Unclear who were the authors of the clinic notes
Giri et al. (33) Ischemic stroke diagnosis by EEG 1D CNN vs. various models (NB, Classification Tree, ANN, RF, kNN, LR) Leave-one-out cross-validation •32 – AIS •30 – Controls 15-min EEG with 24 chosen features •Accuracy - 0.86 •F-Score 0.861 Leave-one-out scenario of 1D CNN In areas with limited access to CT imaging may help diagnosis AIS Time to apply EEG electrodes may result in delays of care
Lee et al. (14) Identify patients within 4.5-h thrombolysis window LR, RF, SVM •85% training •15% test 355 MRI features •Sensitivity 75.8% •Specificity 82.6% •AUC 85.1% RF Improved sensitivity than human readings in identifying stroke patients within thrombolysis window Assessed only dichotomized visibility of signals in the lesion territory
Ho et al. (15) Classifying onset time from imaging LR, RF, GBRT, SVM, SMR 10-fold cross validation on training data with optimal hyperparameters 104 MRI •Sensitivity 78.8% •AUC 76.5% LR with deep autoencoder features Improved stroke onset detection compared to DWI-FLAIR Trained on MRI only
Takahashi et al. (34) Detection for MCA dot sign in unenhanced CT SVM Not described 297 images Unenhanced CT Sensitivity 97.5% SVM Accurately detect hyperdense MCA dot sign Data from 7 patients
Chen et al. (35) Automatically segment stroke lesions in DWI CNN Train / Test 741 subjects DWI Dice score 0.67 CNN Segment stroke lesions automatically Improved Dice scores on larger lesions
Bouts et al. (36) Depict ischemic tissue that can recover after reperfusion GLM, GAM, SVM, Adaptive boosting, RF Generalized cross validation with unbiased risk estimator scoring 19 rats MRI Dice Score 0.79 GLM MRI-based algorithms could estimate extent of salvageable tissue Varying efficacy in differentiating between areas irreversibly damaged vs. salvaged after reperfusion
Chen et al. (37) Quantify cerebral edema following infarction via CSF quantification RF with geodesic active contour segmentation •10-fold cross validation •Train / Test 38 subjects CT Imaging •Baseline Dice Score 0.76 •6-h Dice score 0.73 RF with geodesic active contour segmentation Efficiently and accurately measure evolution of cerebral edema
Colak et al. (38) Stroke Prediction MLP ANN and SVM with radial basis function kernel Train / Test 297 subjects (130 sick and 167 healthy) 9 predictors (CAD, DM, HTN, CVA history, AF, smoking, carotid Doppler findings, cholesterol, CRP •Accuracy 85.9% •AUC 0.93 ANN Ability to screen patients at risk for stroke based on comorbidities Factors used to predict model are known to be risk factors for stroke
Maier et al. (39) Classify lesion segmentation KNN, GNB, GLM, RF, CNN Leave-one-out cross-validation 37 subjects MRI •RF: •Precision 82% •Recall 62% •CNN: •Precision 77% •Recall 64% •RF •CNN Future work may be able to segment lesions No methods achieved results in the range of the human observer agreement
Öman et al. (40) Detection of ischemic stroke 3D CNN Train / Test 60 subjects CT Angiography •Sensitivity 93% •Specificity 82% •AUC 0.93 •Dice 0.61 3D CNN Lesion can be detected with CNN Contralateral hemisphere data may reduce false positive findings
Chen et al. (41) Prehospital detection of large vessel occlusion ANN 10-fold cross validation 600 subjects Baseline demographics, medical history, NIHSS, risk factors •Youden index 0.640 •Sensitivity 0.807 •Specificity 0.833 •Accuracy 0.822 ANN Known patient risk factors may help in predicting large vessel occlusion Cohort included stroke patients and not those with mimics or hemorrhagic stroke
Hemorrhagic stroke
Dhar et al. (42) Hemorrhage and perihematomal edema (PHE) quantification CNN •10-fold cross validation •Train / Test 124 24-h CT head scans •Dice score •0.9 – hemorrhage •0.54 - PHE - Rapid and consistent measurements of supratentorial ICH -IVH not delineated from ICH
Arab et al. (43) Hematoma segmentation and volume quantification CNN with deep supervision based on reader labeling Train / Test 55 64 axial slices of 128 × 128 voxels •Dice score •0.84 ± 0.06 •Precision •0.85 ± 0.07 •Recall •0.83 ± 0.07 •F-Score 0.84 CNN with deep supervision Fast and reliable quantification of hematoma volume •False positives observed with calcifications •False negatives observed with blood close to bone
Ko et al. (44) ICH detection CNN and long-short term memory Train / Test 5,244,234 Pre-processed CTH to balance subtypes and window settings •Classification accuracy •92 – 93% - Identification of ICH and subtypes -Preprocessing of data required to attain accuracy
Irene et al. (45) ICH segmentation and volume approximation Dynamic Graph CNN •4-fold cross validation •Train / Test 27 CTH •Accuracy 96.4% •Precision 0.93 •Recall 0.98 •F-Score 0.96 SVM method with radial basis function kernel Identification of ICH and blood volume prediction Small dataset
Arbabshinrani et al. (13) Diagnose ICH and prioritize radiology worklists Deep CNN •Training (75%) •Cross validation (5%) •Testing (20%) 46,573 studies Preprocessing of CTH images •ROC 0.846 •Specificity 0.8 •Sensitivity 0.73 - Assist in upgrading image reads to “stat” from “routine” Did not identify location of ICH
Sage et al. (46) ICH subtype detection Double-branch CNN of SVM, RF Concatenation of double-branch features and classification 9,997 subjects 372,556 images (11,454 CT scans) •Accuracy range •SVM •76.9 – 96% •RF •74.3 – 96.7% - Identify and classify ICH EDH performed the worst in SVM and RF possibly due to under representation in data
Ye et al. (47) ICH subtype detection 3D joint CNN – recurrent NN •Training (80%) •Validation (10%) •Testing (10%) 2,836 subjects 76,621 slices from non-contrast head CT scans •AUC for +/- ICH • 0.98 •AUC range for subtypes •0.89 – 0.96 - Identify and classify ICH SAH classification may have been more difficult due to blended ICH examples
Chang et al. (48) ICH detection and volume measurements Hybrid 3D/2D CNN 5-fold cross validation 10,841 Scans Non-contrast CTH •ICH detection •Accuracy 0.97 •Sensitivity 0.951 •Specificity 0.073 •Volume •Dice score 0.772–0.931 - Identification of ICH and blood volume prediction Generalization needs to be confirmed in other institutions
Subarachnoid hemorrhage
Capoglu et al. (49) Vasospasm prediction Sparse dictionary learning and covariance-based features Not described 20 3D brain angiograms ROC 0.93 - Proof of concept to predict those who might have vasospasm Small dataset
Ramos et al. (22) DCI Prediction LogReg, SVM, RF, MLP Monte-Carlo cross-validation with 100 random splits (75% training / 25% test) and 5-fold cross-validation 317 Non-contrast CT image data and 48 clinical variables •ROC 0.74 •Specificity 0.67 •Sensitivity 0.75 RF with clinical variables and image features ML improved prediction of DCI especially when image features included (aneurysm height / width) Manual extraction of features from medical images is time-consuming
Tanioka et al. (50) DCI prediction RF Leave-one-out cross-validation 95 Clinical variables and matricellular proteins (MCP) on days 1 – 3 •Accuracy •93.9% - clinical variables •87.2% - MCP only •95.2% - clinical variables + MCP - MCP might play a role in predicting DCI but further data needed Other biomarkers not assessed
Miscellaneous
Ni et al. (12) Stroke Case Detection LR, SVM-P, SVM-R, RF, ANN Two iterations of 10-fold cross validation 8,131 Medical record information compared to ICD codes •Accuracy 88.6% •Precision 93.8% •Recall 92.8% •F Score 93.3% •AUC 89.8% •AUC-PR 97.5% RF Detection of stroke diagnosis through EHR data that was miscoded Accurate ICD codes limit utility of the algorithm
Park et al. (16) Autonomously grade NIHSS and MRC scores through wearable sensors •SVM •Ensemble 5-fold cross validation searched by Bayes optimization in 30 trials 240 Wearable sensors •NIHSS: •Accuracy 83.3% •AUC 0.912 •MRC: •Accuracy 76.7% AUC 0.87 SVM Automatic grading in real time of proximal weakness Requires sensors to be applied
Section B: stroke outcome prediction
References Study objective ML-based approach Validation method Sample size Feature Optimal results Best predictors Clinical implications Limitations
Radiological outcomes
Nielsen et al. (26) Prediction of final infarct volume CNNdeep 85% training/15% testing 222 MRI images AUC 0.88 ± 0.12 - Facilitates treatment selection No external validation, retrospective
Giacalone et al. (51) Prediction of final infarct volume SVM K-fold cross-validation 4 MRI images 95% accuracy - “ ” Small sample size, Retrospective
Grosser et al. (52) Prediction of final infarct volume XGBoost Leave-one-out cross-validation 99 MRI images AUC 0.893 ± 0.085 Spatial lesion probability “ ” Retrospective, Limited generalizability (patient data is from 2006 to 2009)
Foroushani et al. (53) Prediction of malignant cerebral edema LR 10-fold cross-validation 361 Serial, quantitative CT images AUC 0.96 Reduction in CSF volume “ ” No external validation
Bentley et al. (23) Prediction of sICH SVM K-fold cross-validation 116 Unenhanced CT images AUC 0.744 Baseline NIHSS, CT evidence of acute ischemia “ ” Image processing took ~30 min; Small number of sICH cases
Yu et al. (54) Prediction of HT SR-KDA Leave-one-out cross-validation 155 MRI images 83.7 ± 2.6% accuracy - “ ” Single-center, Retrospective
Scalzo et al. (55) Prediction of HT SR-KDA 10-fold cross-validation 263 MRI images 88% accuracy - “ ” Retrospective, current limitations in measuring BBB permeability
van Os et al. (56) Prediction of reperfusion after EVT (mTICI <2b vs. ≥2b) LR (using backward elimination) Nested cross-validation, consisting of an outer and an inner cross-validation loop 1,383 EHR data, CT/CTA images AUC 0.57 - “ ” Retrospective; Only moderate predictive value, LR outperformed machine-learning
Hilbert et al. (57) Prediction of reperfusion after EVT (mTICI <2b vs. ≥2b) RFNN-ResNet-AE fine-tuned 4-fold cross-validation 1301 CTA images Average AUC 0.65 - “ ” Retrospective; Only moderate predictive value
Rondina et al. (58) Comparison of imaging approaches (lesion load per ROI vs. pattern of voxel) to predict post stroke motor impairment GPR 10-fold cross-validation 50 Post stroke MRI Best prediction was obtained using motor ROI and CST (derived from probabilistic tractography) R = 0.83, RMSE = 0.68 Patterns of voxels representing lesion probability produced better results Informs appropriate methodology for predicting long term motor outcomes from early post-stroke MRI. Small sample size, no external validation
Discrete morbidity and mortality clinical outcomes
Matsumoto et al. (59) Prediction of all-cause, in-hospital mortality LASSO 10-fold cross-validation 4,232 EHR data AUC 0.88 - Facilitates GOC decision making Retrospective, Single-center, Limited generalizability (ETV used in only 1.5% of patients), Low rate (3.5%) of in-hospital mortality
Scrutinio et al. (60) Prediction of 3-yr mortality after severe stroke SMOTE RF 10-fold cross-validation 1,207 EHR data AUC 0.928 Age Facilitates GOC decision making No external validation
Ge et al. (61) Prediction of SAP at 7 and 14 d Attention-augmented GRU 10-fold cross-validation 13,930 EHR data •7 d: AUC 0.928 •14 d: AUC 0.905 PPI use Facilitates early detection and targeted application of prophylaxis interventions Single-center, No external validation
Li et al. (62) Prediction of SAP at 7 d XGBoost 5-fold cross-validation 3,160 EHR data AUC 0.841 Age, Baseline NIHSS, FBG, sex, Premorbid mRS score, & History of AF “ “ Single-center, No external validation
Wang et al. (63) Predicting functional outcome (mRS) at 1st and 6th months RF 10-fold cross-validation 333 Demographics, labs, CT brain •1 month outcome: AUC 0.899; •6 months outcome AUC: 0.917 •1 month outcome= 26 attributes; •6 months outcome: 22 attributes Use of ML to predict functional outcome after ICH is feasible, and RF model provides the best predictive performance Small sample size, excluded large hematomas, did not evaluate hematoma or edema expansion, no external validation
Functional outcomes
Heo et al. (64) Prediction of mRS score (0–2 vs. 3–6) at 90 d Deep neural network 67% training/ 33% testing 2,604 EHR data AUC 0.888 - Informs patient expectations, Facilitates GOC decision making Single-center, No external validation
Lin et al. (65) Prediction of mRS score (0–2 vs. 3–6) at 90 d SVM 10-fold cross-validation 35,798 Registry data f1-score 87.9 ± 0.2% (92.9 ± 0.1%, with follow-up data) mRS score at 30 d, toilet use degree of dependence “ “ More severe strokes accounted for most prediction errors
Brugnara et al. (66) Prediction of mRS score (0–2 vs. 3–6) at 90 d “Machine-learning models with gradient boosting classifiers” Not specified 246 Clinical data, radiological data (CT, CTA, CTP, and angiographic images) AUC 0.856 NIHSS score at 24 h, Premorbid mRS score, Final infarct volume on CT “ “ Single center, No external validation, Retrospective
Forkert et al. (67) Prediction of mRS score at 90 d SVM (specifically the Extended Problem- specific model) Leave-one-out cross-validation 68 Clinical data, MRI images •mRS score ± 1: 82.4% accuracy •mRS score 0–2 vs. 3–6: 85.4% accuracy •L-hemisphere strokes: lesion-based t-score sum •Rt-hemisphere strokes: Lesion volume “ “ No external validation, Retrospective
Monteiro et al. (68) Prediction of mRS score (0–2 vs. 3–6) at 90 d RF 10-fold cross-validation 425 Clinical data, CT or MRI images AUC 0.936 ± 0.34 Baseline NIHSS score, Baseline NIHSS score on subsection 2 (Best gaze, horizontal EOMs) “ “ Single center, No external validation, Retrospective, Performed worse than non-imaging model
Jang et al. (69) Prediction of mRS score (>1 vs. >2) at 90 d XGBoost 3-fold cross-validation and a random search strategy 6,731 Registry data •mRS >1: AUC 0.84 •mRS >2: AUC 0.87 “ “ Treatment-related factors were not included, No external validation
Hope et al. (70) Prediction of speech production scores GPR Leave-one-out cross-validation 270 Clinical data, Assessments, MRI images R2 0.59 Time post-stroke, Lesion site Informs patient expectations Post-stroke imaging obtained over a wide range of times (<1 month to +30 y), No external validation, Retrospective
Lopes et al. (71) Prediction of cognitive functions at 3 y after minor stroke Ridge Regression 3-step nested leave-one-out cross-validation, consisting of inner, middle, and outer loops 72 Clinical data, Assessments, functional MRI images R2 values for attention, memory, visuospatial functions, and language functions: 0.73, 0.67, 0.55, 0.48 - “ “ Limited generalizability (mean NIHSS on admission was 1.5 ± 2.2), Retrospective
Sale et al. (72) Prediction of change in BI score and FIM score during inpatient rehab SVM Nested 5-fold cross-validation 55 Clinical biomarker data, Assessments Discharge cognitive FIM score: MADP 17.55%, RMSE 4.28 Cognitive FIM score upon admission Informs patient expectations, Facilitates GOC decision making Small sample size, included hemorrhagic stroke patients
Iwamoto et al. (73) Prediction of ADL dependence after inpatient rehab CART method Not specified 994 Clinical data, Assessments AUC 0.83 FIM transfer score (≤ 4 or >4) “ “ Single center, Retrospective
Lin et al. (74) Prediction of BI score (<60, 60–90, >90) upon discharge from inpatient rehab LR, RF 5-fold cross-validation 313 Clinical data, Assessments LR: AUC 0.796, RF: AUC 0.792 BI, IADL, and BBT scores on admission “ “ Limited generalizability due to aggressive rehab strategy, No external validation
Tozlu et al. (75) Prediction of post-intervention UE motor impairment in chronic stroke Elastic net Nested 10-fold cross-validation with outer and inner loops 102 Clinical data, Assessments Median R2 0.91 Pre-intervention UE-FMA, difference in MT between the affected and unaffected hemispheres Informs patient expectations, Increases rehabilitation efficiency Retrospective, No external validation
Stinear et al. (76) Predicts potential for UE recovery Cluster analyses Not applicable 40 Clinical assessments ± neurophysiological assessments and MRI images Partial η2 0.811 - “ “ Small sample size, Single center, No external validation

Section A and B

ADL, Activities of daily living; AE, Auto-encoders; AF, Atrial fibrillation; AIS, Acute ischemic stroke; ANN, Artificial neural network; AUC, area under the receiver operating characteristic curve; BBB, blood-brain barrier; BBT, Berg balance test; BI, Barthel Index; CART, Classification and regression tree; CNN, convolutional neural network; CSF, cerebral spinal fluid; CST, Corticospinal tract; CT, computed tomography; CTA, Computerized tomography angiography; CTP, Computerized tomography perfusion; CXR, Chest radiograph; D, days; DCI, delayed cerebral ischemia; DTI, Diffusion Tensor Imaging; DWI, diffusion weighted image; EDH, epidural hematoma; EEG, electroencephalogram; EHR, electronic health record; EOMs, Extra-ocular movements; EVT, endovascular treatment; FBG, Fasting blood glucose; FIM, Functional independence measure; GAM, generalized additive model; GBRT, gradient boosted regression tree; GLM, generalized linear model; GOC, Goals-of-care; GRU, gated recurrent unit; GPR, Gaussian Process model Regression; H, hours; HT, hemorrhagic transformation; IADL, Instrumental activities of daily living scale; ICH, Intracerebral hemorrhage; IVH, intraventricular hemorrhage; KNN, K nearest neighbor; L, Left; LASSO, Least absolute shrinkage and selection operator regression; LR, logistic regression; MADP, Mean absolute percentage deviation; MCA, middle cerebral artery; MCP, matricellular proteins; Min, minutes; MLP, multilayer perceptron; MRC, medical research council; MRI, magnetic resonance imaging; mRS, Modified Rankin Score; MT, motor threshold; NB, naïve bayes; NIHSS, National Institutes of Health Stroke Scale; PHE, perihematomal edema; PPI, Proton pump inhibitor; RF, Random forest; RFNN, Structured Receptive Field Neural Networks; RMSE, Root mean square error; ROI, region of interest; Rt, Right; SAP, Stroke-associated pneumonia; sICH, symptomatic intracranial hemorrhage; SMOTE, synthetic minority oversampling technique; SMR, stepwise multilinear regression; SR-KDA, Kernel Spectral Regression for Discriminant Analysis; SVM, support vector model; SVM-P, support vector machine with polynomial; SVM-R, support vector machine with radial basis function; UE, Upper extremity; UE-FMA, Upper extremity Fugl-Meyer Assessment; XGBoost, Extreme gradient boosting; Yr, year.

NB: List of ML terms with definitions is provided in Supplementary Table 1.

Section B

Many of the listed studies utilize a variety of machine learning (ML)-based approaches. The approach listed on the table is the approach with the optimal result from each individual study.

Phenotype based on Oxfordshire Community Stroke Project (OCSP) (total anterior circulation infarcts, lacunar infarcts, partial anterior circulation infarcts, posterior circulation infarcts).

 Methods

We searched PubMed, Google Scholar, Web of Science, and IEEE Xplore® for relevant articles using various combination of the following key words: “machine learning,” “artificial intelligence,” “stroke,” “ischemic stroke,” “hemorrhagic stroke,” “diagnosis,” “prognosis,” “outcome,” “big data,” and “outcome prediction.” Resulting abstracts were screened by all authors and articles were hand-picked for full review based on relevance and scientific integrity. Final article list was reviewed and approved by all authors.

Machine Learning in Stroke Diagnosis

The time-sensitive nature of stroke care underpins the need for accurate and rapid tools to assist in stroke diagnosis. Over the recent years, the science of brain imaging has vastly advanced with the availability of a myriad of AI based diagnostic imaging algorithms (77). Machine learning is particularly useful in diagnosis of acute stroke with large vessel occlusion (LVO). Various automated methods for detection of stroke core and penumbra size as well as mismatch quantification and detection of vascular thrombi have recently been developed (77). Over the past decade, 13 different companies have developed automated and semi-automated commercially available software for acute stroke diagnostics (Aidoc®, Apollo Medical Imaging Technology®, Brainomix®, inferVISION®, RAPID®, JLK Inspection®, Max-Q AI®, Nico.lab®, Olea Medical®, Qure.ai®, Viz.ai®, and Zebra Medical Vision®) (78). The RapidAI® and Viz.ai® technology have been approved under the medical device category of computer-assisted triage by the United States Food and Drug Administration (FDA). The RAPID MRI® (Rapid processing of Perfusion and Diffusion) software allows for an unsupervised, fully-automated processing of perfusion and diffusion data to identify those who may benefit from thrombectomy based on the mismatch ratio (79). Such commercial platforms available for automatic detection of ischemic stroke and LVO have facilitated rapid treatment decisions. When compared to manual segmentation of lesion volume and mismatch identification from patients enrolled in DEFUSE 2, the RAPID results were found to be well-correlated (r2 = 0.99 and 0.96 for diffusion and perfusion weighted imaging, respectively) with 100% sensitivity and 91% specificity for mismatch identification (80). Since 2008, the RapidAI® platform has expanded to include other products (Rapid® ICH, ASPECTS, CTA, LVO, CTP, MRI, Angio, and Aneurysm) that assist across the entire spectrum of stroke. Viz LVO® was the first FDA-cleared software to detect and alert clinicians of LVO via the “Viz Platform” (81). In a recent single center study with 1,167 CTAs analyzed, Viz LVO® was found to have a sensitivity of 0.81 and a negative predictive value of 0.99 with an accuracy of 0.94 (82).

Other areas of stroke diagnostics that have seen an increase in attention over the past decade are the identification of intracerebral hemorrhage (ICH) and patients at risk for delayed cerebral ischemia in the setting of aneurysmal subarachnoid hemorrhage (aSAH). While most studies tend to have good accuracy in detecting an ICH there is more variability in subclassification and measurements of hematoma volume. A summary of recent publications on ML in stroke diagnosis is presented in Table 1 (Section A).

 Machine Learning in Stroke Outcome Prediction

Despite recent advances in stroke care, it remains the second leading cause of death and disability world-wide (4, 83). Although acute stroke diagnosis and determination of the time of stroke onset are the initial steps of comprehensive stroke management, clinicians are also often charged with the task of determining stroke outcomes. These outcomes range from discrete radiological outcomes (e.g., final infarct volume, the likelihood of hemorrhagic transformation, etc.), the likelihood of morbidity (e.g., stroke-associated pneumonia) and mortality, and various measures of functional independence (e.g., mRS score, Barthel Index score, cognitive, and language function, etc.).

Prognostication after an acute brain injury is notoriously challenging, particularly within the first 24–48 h (84). However, a clinician may be called upon to provide estimates of a patient's short-term and long-term mortality and degree of functional dependence to assist with decision-making regarding the intensity of care (e.g., use of thrombolytics or endovascular treatment, intubation, code status, etc.) (60, 64, 66, 67, 69, 70, 72–76). Like all medical emergencies, it is incumbent upon the stroke clinician to ensure that all care provided is concordant with an individual patient's goals (85). For example, a surrogate decision-maker may decline to reverse a patient's longstanding “do not intubate” order to facilitate mechanical thrombectomy if the clinician predicts the patient has a high likelihood of functional dependence or short-term mortality. Hence, accuracy in outcome prediction is critical in guiding management of our patients.

Determining a patient's likelihood of developing symptomatic intracranial hemorrhage (sICH) is of obvious, immediate value in acute stroke management in determining candidacy for thrombolytic therapy or endovascular treatment. Historically, clinician-based prognostication tools to predict the risk of symptomatic intracranial hemorrhage after IV thrombolysis, such as the SEDAN (Sugar, Early Infarct signs, Dense cerebral artery sign, Age, and NIHSS) and HAT (Hemorrhage After Thrombolysis) scores have been used to predict the risk of symptomatic intracranial hemorrhage after IV thrombolysis (23). Advances in ML and DL have allowed for the development of more accurate models which outperform the traditional SEDAN and HAT scores (23, 54, 55). Similarly, the ability to predict final infarct volume and the likelihood of the development of malignant cerebral edema have important treatment implications and remain a significant focus of ML in stroke (26, 51–53).

In patients with intracerebral hemorrhage (ICH), the ICH-score is one of the most widely used clinical prediction scores (85–88). Although ML technology for outcome prediction has rapidly advanced for ischemic stroke, recent ML studies predicting functional outcomes after ICH have also demonstrated high-discriminating power (63, 89). A recent study by Sennfält et al. tracked long-term functional dependence and mortality after an acute ischemic stroke of more than 20,000 Swedish patients (90). The 30-day mortality rate was 11.1%. At 5 years, 70.6% of ischemic stroke patients were functionally dependent (defined as mRS score of ≥3) or had died (5-year mortality rate of 50.6%). These sobering outcomes partially account for the development of many stroke prognostic models over the years, which frequently serve as benchmarks in stroke research. Recently, Matsumoto et al. compared the performance of six existing stroke prognostic models for predicting poor functional outcomes and in-hospital mortality with linear regression or decision tree ensemble models (59). The novel prediction models performed slightly better than the conventional models in predicting poor functional outcomes (AUC 0.88–0.94 vs. AUC 0.70–0.92) but were equivalent or marginally worse in predicting in-hospital death (AUC 0.84–0.88 vs. AUC 0.87–0.88). Many such stroke prediction models have emerged over the recent years. An overview of ML based automated algorithms for stroke outcome prediction is provided in Table 1 (Section B).

 Discussion

In recent years, some DL algorithms have approached human levels of performance in object recognition (91). One of the greatest strengths of ML is its ability to endlessly process data and tirelessly perform an iterative task. Further, creation of a ML model can be performed much faster (i.e., in a matter of 5–6 days compared with 5–6 months or even years) than traditional computer-aided detection and diagnosis (CAD) (92). which makes ML an attractive field for computer experts and scientists. Several ML tools are currently in use including the FDA-approved ML algorithms previously discussed for rapid stroke diagnosis which have significantly enhanced the workflow of acute ischemic stroke patients.

Despite the prolific advent of new and improved ML algorithms with increasing clinical applications, it is important to recognize that computer-based algorithms are only as good as the data used to train the models. For a reliable algorithm, it is important to develop well-defined training, validation, and testing sets. Testing should be done on a diverse set of data points reflective of a real-world scenario. Overfitting can be an issue in ML algorithms when the model is trained on a group of highly-selected, specific features, which when tested on a larger dataset with varied features, fails to perform adequately. Similarly, underfitting can occur when a model is oversimplified with generalized feature selection in the training set which then becomes unable to capture the relevant features within a complex pattern of a larger or more diverse testing set. The aphorism “garbage in, garbage out” remains true as the use of inadequate or unvalidated data points (e.g., unverified clinical reports from electronic health record) in the training set can lead to poor performance of the ML algorithm in the testing set. Hence, it is important to note that the algorithmic decision-making tools do not guarantee accurate and unbiased interpretation compared to established logistic regression models (56, 59, 93). Comparisons to well-established models should be standard when developing new ML algorithms given the high cost associated with ML (e.g., the time required to collect data, train the model, perform internal and external validations, cost of reliable and secure data storage, etc.) (94). Specifically, as it relates to diagnostics there are a myriad of considerations that must be taken into account. Not only should the algorithm provide accurate information quickly, but it should have the ability to integrate into the electornic health record (EHR) to improve end user experience and efficiency in workflow. Programs such as RAPID®, Viz.ai®, and Brainomix® have started to successfully integrate into the EHR, which has helped expedite acute stroke diagnosis and triage process. One of the major technical challenges of ML include the ability to develop an algorithm with a “reasonable” detection rate of pathology without an excessive rate of false-positives. For example, there are notable discrepancies among various ML studies for ICH diagnosis, with varying accuracy depending on the type of ICH (e.g., spontaneous ICH, SDH, aSAH, or IVH). Overfitting and underfitting of the model could lead to poor applicability and therefore, image preprocessing with meticulous feature selection is necessary. Furthermore, the “black-box” nature of ML precludes the clinicians from identifying and addressing biases within the algorithms (95, 96). Hence, proper external validation is necessary to ensure generalizability of the algorithm in diverse clinical scenarios.

For stroke prediction, most existing ML algorithms utilize dichotomized outcomes. Functional outcome is frequently defined as “good” when mRS score is 0–2 and “poor” when mRS score is 3–6 by convention and IS studies often measure mRS score at 90 days after stroke (64–69, 97). However, the medical community is increasingly embracing patient-centered outcomes. People are starting to recognize the need for longitudinal patient follow-up given potential for functional improvement beyond conventional norms of 90 days (98). Once patient-centered outcomes are clinically validated (e.g., MRS cutoff of 0–2 vs. 3–6, 0–3 vs. 4–6, or 0–4 vs. 5–6), new ML algorithms incorporating such outcomes would be increasingly helpful to the clinicians. The use of high-yield, ML programs using patient-centered outcomes could ease the commonplace but challenging discussions of the anticipated quality of life and the risk of long-term dependency or death before deciding on a patient's goals-of-care. It is however important to apply caution while using ML algorithms for outcome prediction as patient demographics and clinical practice continue to evolve and updates to the ML algorithms would be necessary to remain applicable to evolving patient populations and clinical standards. Additionally, developers often retrieve data from existing datasets (e.g., clinical trial data) with its inherent biases including selection bias, observer bias and other confounders (e.g., withdrawal of life supporting therapy may be more common in older patients with large hemispheric stroke compared to younger patients, which could confound outcome prediction in older patients compared to younger ones).

Overall, compared to other diseases such as Alzheimer's disease, there is a relative paucity of large, high-quality datasets within stroke. Some limitations that have stymied the development of large, open-access stroke registries include the need for data-sharing agreements, patient privacy concerns, high costs of data storage and security, arbitration of quality control of the input data, etc. (95). Cohesive and collaborative efforts across hospital systems, regions, and nations with data acquisition and harmonization is needed to improve future ML-based programs in stroke. With adoption of EHR systems, healthcare data is rapidly accumulating with an estimated over 35 zettabytes of existing healthcare data! (99). Adoption of AI and ML algorithms allow us to efficiently process the plethora of information that surround us every day. Nonetheless, as we continue to adapt to this evolving landscape of medical practice surrounding big data, clinicians need to remain aware of the limitations of this modern day “black box” magic.

 Conclusion

The emerging ML technology has rapidly integrated into multiple fields of medicine including stroke. Deep learning has significantly enhanced practical applications of ML and some newer algorithms are known to have comparable accuracy to humans. However, the diagnosis and prognosis of a disease, including stroke, is highly intricate and depends on various clinical and personal factors. The development of optimal ML programs requires comprehensive data collection and assimilation to improve diagnostic and prognostic accuracy. Given the “black box” or cryptic nature of these algorithms, it is extremely important for the end-user (i.e., clinicians) to understand the intended use and limitations of any ML algorithm to avoid inaccurate data interpretation. Although ML algorithms have improved stroke systems of care, blind dependence on such computerized technology may lead to misdiagnosis or inaccurate prediction of prognostic trajectories. At the current state, ML tools are best used as “aids” for clinical decision making while still requiring oversight to address relevant clinical aspects that are overlooked by the algorithm.

 Author Contributions

SM: substantial contributions including conception and design of the work, literature review, interpretation and summarization of data, drafting the complete manuscript, revising it critically for important intellectual content, and final approval of the manuscript to be published. MD and KS: contribution including conception and design of the work, literature review, interpretation and summarization of the data, drafting of critical portion of the manuscript, critical revision for important intellectual content, and final approval of the manuscript. All authors contributed to the article and approved the submitted version.

 Funding

This article was supported by the Virginia Commonwealth University, Department of Neurology.

 Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

 Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

 Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2021.734345/full#supplementary-material

 References Samuel L. Some studies in machine learning using the game of checkers. IBM J Res Dev. (1959) 3:210–29. 10.1147/rd.33.0210 Mitchell TM. Machine Learning. New York: Mcgraw-hill (1997). Tankovska H. E-mail Usage in the United States—Statistics and Facts. In, Statista. Hamburg: Statista (2021). GBD 2016 Neurology Collaborators. Global, regional, and national burden of neurological disorders:1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol. (2019) 18:459–80. 10.1016/S1474-4422(18)30499-X30879893 Ovbiagele B Goldstein LB Higashida RT Howard VJ Johnston SC Khavjou OA . Forecasting the future of stroke in the United States: a policy statement from the American Heart Association and American Stroke Association. Stroke. (2013) 44:2361–75. 10.1161/STR.0b013e31829734f223697546 Hacke W Kaste M Bluhmki E Brozman M Dávalos A Guidetti D . Thrombolysis with alteplase 3 to 45 hours after acute ischemic stroke. N Engl J Med. (2008) 359:1317–29. 10.1056/NEJMoa080465618815396 Bluhmki E Chamorro A Dávalos A Machnig T Sauce C Wahlgren N . Stroke treatment with alteplase given 30-45 h after onset of acute ischaemic stroke (ECASS III): additional outcomes and subgroup analysis of a randomised controlled trial. Lancet Neurol. (2009) 8:1095–102. 10.1016/S1474-4422(09)70264-919850525 Powers WJ Rabinstein AA Ackerson T Adeoye OM Bambakidis NC Becker K . 2018 Guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. (2018) 49:e46–110. 10.1161/STR.000000000000015829367334 Nogueira RG Jadhav AP Haussen DC Bonafe A Budzik RF Bhuva P . Thrombectomy 6 to 24 hours after stroke with a mismatch between deficit and infarct. N Engl J Med. (2018) 378:11–21. 10.1056/NEJMoa170644229129157 Campbell A Bruce CV Peter Mitchell J Timothy Kleinig J Helen Dewey M Churilov L . Endovascular therapy for ischemic stroke with perfusion-imaging selection. N Engl J Med. (2015) 372:1009–18. 10.1056/NEJMoa141479233632884 Powers WJ Rabinstein AA Ackerson T Adeoye OM Bambakidis NC Becker K . Guidelines for the early management of patients with acute ischemic stroke: 2019 update to the 2018 guidelines for the early management of acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. (2019) 50:e344–418. 10.1161/STR.000000000000021131765293 Ni Y Alwell K Moomaw CJ Woo D Adeoye O Flaherty ML . Towards phenotyping stroke: leveraging data from a large-scale epidemiological study to detect stroke diagnosis. PLoS ONE. (2018) 13:e0192586. 10.1371/journal.pone.019258629444182 Arbabshirani MR Fornwalt BK Mongelluzzo GJ Suever JD Geise BD Patel AA . Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration. NPJ Digit Med. (2018) 1:9. 10.1038/s41746-017-0015-z31304294 Lee H Lee EJ Ham S Lee HB Lee JS Kwon SU . Machine learning approach to identify stroke within 45 hours. Stroke. (2020) 51:860–6. 10.1161/STROKEAHA.119.02761131987014 Ho KC Speier W Zhang H Scalzo F El-Saden S Arnold CW . Machine learning approach for classifying ischemic stroke onset time from imaging. IEEE Trans Med Imaging. (2019) 38:1666–76. 10.1109/TMI.2019.290144530802855 Park E Lee K Han T Nam HS. Automatic grading of stroke symptoms for rapid assessment using optimized machine learning and 4-limb kinematics: clinical validation study. J Med Internet Res. (2020) 22:e20641. 10.2196/2064132936079 Kogan E Twyman K Heap J Milentijevic D Lin JH Alberts M. Assessing stroke severity using electronic health record data: a machine learning approach. BMC Med Inform Decis Mak. (2020) 20:8. 10.1186/s12911-019-1010-x31914991 Chung JW Kim YC Cha J Choi EH Kim BM Seo WK . Characterization of clot composition in acute cerebral infarct using machine learning techniques. Ann Clin Transl Neurol. (2019) 6:739–47. 10.1002/acn3.75131019998 Dhar R Chen Y An H Lee JM. Application of machine learning to automated analysis of cerebral edema in large cohorts of ischemic stroke patients. Front Neurol. (2018) 9:687. 10.3389/fneur.2018.0068730186224 Liu J Xu H Chen Q Zhang T Sheng W Huang Q . Prediction of hematoma expansion in spontaneous intracerebral hemorrhage using support vector machine. EBioMedicine. (2019) 43:454–9. 10.1016/j.ebiom.2019.04.04031060901 Dengler NF Madai VI Unteroberdörster M Zihni E Brune SC Hilbert A . Outcome prediction in aneurysmal subarachnoid hemorrhage: a comparison of machine learning methods and established clinico-radiological scores. Neurosurg Rev. (2021) 44:2837–46. 10.1007/s10143-020-01453-633474607 Ramos LA van der Steen WE Sales Barros R Majoie C van den Berg R Verbaan D . Machine learning improves prediction of delayed cerebral ischemia in patients with subarachnoid hemorrhage. J Neurointerv Surg. (2019) 11:497–502. 10.1136/neurintsurg-2018-01425830415227 Bentley P Ganesalingam J Carlton Jones AL Mahady K Epton S Rinne P . Prediction of stroke thrombolysis outcome using CT brain machine learning. Neuroimage Clin. (2014) 4:635–40. 10.1016/j.nicl.2014.02.00324936414 Géron A. Hands-On Machine Learning With Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. Sebastopol, CA: O'Reilly Media (2019). Feng R Badgeley M Mocco J Oermann EK. Deep learning guided stroke management: a review of clinical applications. J Neurointerv Surg. (2018) 10:358–62. 10.1136/neurintsurg-2017-01335528954825 Nielsen A Hansen MB Tietze A Mouridsen K. Prediction of tissue outcome and assessment of treatment effect in acute ischemic stroke using deep learning. Stroke. (2018) 49:1394–401. 10.1161/STROKEAHA.117.01974029720437 Yu Y Xie Y Thamm T Gong E Ouyang J Huang C . Use of deep learning to predict final ischemic stroke lesions from initial magnetic resonance imaging. J Am Med Assoc Netw Open. (2020) 3:e200772. 10.1001/jamanetworkopen.2020.077232163165 Dey A. Machine learning algorithms: a review. Int J Comp Sci Inform Technol. (2016) 7:1174–9. Available online at: https://ijcsit.com/docs/Volume%207/vol7issue3/ijcsit2016070332.pdf Zhou ZH. Ensemble learning. Encycl Biometr. (2009) 2009:270–3. 10.1007/978-0-387-73003-5_293 Géron A. Hands-on Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. Sebastopol, CA: O'Reilly Media, Inc. (2017). Garcia-Terriza L Risco-Martin JL Ayala JL Rosello GR Camarasaltas JM. IEEE: comparison of different machine learning approaches to model stroke subtype classification and risk prediction. In: 2019 Spring Simulation Conference. Tuscan, AZ: Springsim. (2019). 10.23919/SpringSim.2019.8732846 Sung SF Lin CY Hu YH. EMR-based phenotyping of ischemic stroke using supervised machine learning and text mining techniques. IEEE J Biomed Health Informat. (2020) 24:2922–31. 10.1109/JBHI.2020.297693132142458 Giri EP Fanany MI Arymurthy AM Wijaya SK. Ischemic stroke identification based on EEG and EOG using ID convolutional neural network and batch normalization. In: 2016 International Conference on Advanced Computer Science and Information Systems, ICACSIS 2016. Malang (2017). 10.1109/ICACSIS.2016.787278027295638 Takahashi N Lee Y Tsai DY Matsuyama E Kinoshita T Ishii K. An automated detection method for the MCA dot sign of acute stroke in unenhanced CT. Radiol Phys Technol. (2014) 7:79–88. 10.1007/s12194-013-0234-123996520 Chen L Bentley P Rueckert D. Fully automatic acute ischemic lesion segmentation in DWI using convolutional neural networks. Neuroimage Clin. (2017) 15:633–43. 10.1016/j.nicl.2017.06.01628664034 Bouts MJ Tiebosch IA van der Toorn A Viergever MA Wu O Dijkhuizen RM. Early identification of potentially salvageable tissue with MRI-based predictive algorithms after experimental ischemic stroke. J Cereb Blood Flow Metab. (2013) 33:1075–82. 10.1038/jcbfm.2013.5123571283 Chen Y Dhar R Heitsch L Ford A Fernandez-Cadenas I Carrera C . Automated quantification of cerebral edema following hemispheric infarction: application of a machine-learning algorithm to evaluate CSF shifts on serial head CTs. Neuroimage Clin. (2016) 12:673–80. 10.1016/j.nicl.2016.09.01827761398 Colak A Karaman E Turtay MG. Application of knowledge discovery process on the prediction of stroke. Comput Methods Programs Biomed. (2015) 119:181–5. 10.1016/j.cmpb.2015.03.00225827533 Maier O Schröder C Forkert ND Martinetz T Handels H. Classifiers for ischemic stroke lesion segmentation: a comparison study. PLoS ONE. (2015) 10:e0145118. 10.1371/journal.pone.014511826871444 Öman O Mäkelä T Salli E Savolainen S Kangasniemi M. 3D convolutional neural networks applied to CT angiography in the detection of acute ischemic stroke. Eur Radiol Exp. (2019) 3:8. 10.1186/s41747-019-0085-630758694 Chen Z Zhang R Xu F Gong X Shi F Zhang M . Novel prehospital prediction model of large vessel occlusion using artificial neural network. Front Aging Neurosci. (2018) 10:181. 10.3389/fnagi.2018.0022230050431 Dhar R Falcone GJ Chen YS Hamzehloo A Kirsch EP Noche RB . Deep learning for automated measurement of hemorrhage and perihematomal edema in supratentorial intracerebral hemorrhage. Stroke. (2020) 51:648–51. 10.1161/STROKEAHA.119.02765731805845 Arab A Chinda B Medvedev G Siu W Guo H Gu T . A fast and fully-automated deep-learning approach for accurate hemorrhage segmentation and volume quantification in non-contrast whole-head CT. Sci Rep. (2020) 10:7. 10.1038/s41598-020-76459-733168895 Ko A Chung H Lee H Lee J. IEEE: feasible study on intracranial hemorrhage detection and classification using a CNN-LSTM network. In: 42nd Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society (EMBC). Montreal (2020). 10.1109/EMBC44109.2020.917616233018224 Irene K Masum MA Yunus RE Jatmiko W. Segmentation and approximation of blood volume in intracranial hemorrhage patients based on computed tomography scan images using deep learning method. In: 2020 International Workshop on Big Data and Information Security, IWBIS 2020. Depok (2020). 10.1109/IWBIS50925.2020.925559327295638 Sage A Badura P. Intracranial hemorrhage detection in head CT using double-branch convolutional neural network, support vector machine, and random forest. Appl Sci. (2020) 10:577. 10.3390/app10217577 Ye A Gao F Yin YB Guo DF Zhao PF Lu Y . Precise diagnosis of intracranial hemorrhage and subtypes using a three-dimensional joint convolutional and recurrent neural network. Eur Radiol. (2019) 29:6191–201. 10.1007/s00330-019-06163-231041565 Chang PD Kuoy E Grinband J Weinberg BD Thompson M Homo R . Hybrid 3D/2D convolutional neural network for hemorrhage evaluation on head CT. Am J Neuroradiol. (2018) 39:1609–16. 10.3174/ajnr.A574230049723 Capoglu S Savarraj JP Sheth SA Choi HA Giancardo L. Representation learning of 3D brain angiograms, an application for cerebral vasospasm prediction. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. Berlin (2019). 10.1109/EMBC.2019.885781531946608 Tanioka S Ishida F Nakano F Kawakita F Kanamaru H Nakatsuka Y . Machine learning analysis of matricellular proteins and clinical variables for early prediction of delayed cerebral ischemia after aneurysmal subarachnoid hemorrhage. Mol Neurobiol. (2019) 56:7128–35. 10.1007/s12035-019-1601-730989629 Giacalone M Rasti P Debs N Frindel C Cho TH Grenier E . Local spatio-temporal encoding of raw perfusion MRI for the prediction of final lesion in stroke. Med Image Anal. (2018) 50:117–26. 10.1016/j.media.2018.08.00830268970 Grosser M Gellißen S Borchert P Sedlacik J Nawabi J Fiehler J . Improved multi-parametric prediction of tissue outcome in acute ischemic stroke patients using spatial features. PLoS ONE. (2020) 15:e0228113. 10.1371/journal.pone.022811332163518 Foroushani HM Hamzehloo A Kumar A Chen Y Heitsch L Slowik A . Quantitative serial CT imaging-derived features improve prediction of malignant cerebral edema after ischemic stroke. Neurocrit Care. (2020) 33:785–92. 10.1007/s12028-020-01056-532729090 Yu Y Guo D Lou M Liebeskind D Scalzo F. Prediction of hemorrhagic transformation severity in acute stroke from source perfusion MRI. IEEE Trans Biomed Eng. (2018) 65:2058–65. 10.1109/TBME.2017.278324129989941 Scalzo F Alger JR Hu X Saver JL Dani KA Muir KW . Multi-center prediction of hemorrhagic transformation in acute ischemic stroke using permeability imaging features. Magn Reson Imaging. (2013) 31:961–9. 10.1016/j.mri.2013.03.01323587928 van Os HJA Ramos LA Hilbert A van Leeuwen M van Walderveen MAA Kruyt ND . Predicting outcome of endovascular treatment for acute ischemic stroke: potential value of machine learning algorithms. Front Neurol. (2018) 9:784. 10.3389/fneur.2018.0078430319525 Hilbert A Ramos LA van Os HJA Olabarriaga SD Tolhuisen ML Wermer MJH . Data-efficient deep learning of radiological image data for outcome prediction after endovascular treatment of patients with acute ischemic stroke. Comput Biol Med. (2019) 115:103516. 10.1016/j.compbiomed.2019.10351631707199 ARondina M Filippone M Girolami M Ward NS. Decoding post-stroke motor function from structural brain imaging. Neuroimage Clin. (2016) 12:372–80. 10.1016/j.nicl.2016.07.01427595065 Matsumoto K Nohara Y Soejima H Yonehara T Nakashima N Kamouchi M. Stroke prognostic scores and data-driven prediction of clinical outcomes after acute ischemic stroke. Stroke. (2020) 51:1477–83. 10.1161/STROKEAHA.119.02730032208843 Scrutinio D Ricciardi C Donisi L Losavio E Battista P Guida P . Machine learning to predict mortality after rehabilitation among patients with severe stroke. Sci Rep. (2020) 10:20127. 10.1038/s41598-020-77243-333208913 Ge Y Wang Q Wang L Wu H Peng C Wang J . Predicting post-stroke pneumonia using deep neural network approaches. Int J Med Inform. (2019) 132:103986. 10.1016/j.ijmedinf.2019.10398631629312 Li X Wu M Sun C Zhao Z Wang F Zheng X . Using machine learning to predict stroke-associated pneumonia in Chinese acute ischaemic stroke patients. Eur J Neurol. (2020) 27:1656–63. 10.1111/ene.1429533463872 Wang HL Hsu WY Lee MH Weng HH Chang SW Yang JT . Automatic machine-learning-based outcome prediction in patients with primary intracerebral hemorrhage. Front Neurol. (2019) 10:910. 10.3389/fneur.2019.0091031496988 Heo J Yoon JG Park H Kim YD Nam HS Heo JH. Machine learning-based model for prediction of outcomes in acute stroke. Stroke. (2019) 50:1263–5. 10.1161/STROKEAHA.118.02429330890116 Lin J Jiang A Ling M Mo Y Li M Zhao J. Prediction of neurologic deterioration based on support vector machine algorithms and serum osmolarity equations. Brain Behav. (2018) 8:e01023. 10.1002/brb3.102329888877 Brugnara G Neuberger U Mahmutoglu MA Foltyn M Herweh C Nagel S . Multimodal predictive modeling of endovascular treatment outcome for acute ischemic stroke using machine-learning. Stroke. (2020) 51:3541–51. 10.1161/STROKEAHA.120.03028733493061 Forkert ND Verleger T Cheng B Thomalla G Hilgetag CC Fiehler J. Multiclass support vector machine-based lesion mapping predicts functional outcome in ischemic stroke patients. PLoS ONE. (2015) 10:e0129569. 10.1371/journal.pone.012956926098418 Monteiro M Fonseca AC Freitas AT Pinho EMT Francisco AP Ferro JM . Using machine learning to improve the prediction of functional outcome in ischemic stroke patients. IEEE/ACM Trans Comput Biol Bioinform. (2018) 15:1953–9. 10.1109/TCBB.2018.281147129994736 Jang SK Chang JY Lee JS Lee EJ Kim YH Han JH . Reliability and clinical utility of machine learning to predict stroke prognosis: comparison with logistic regression. J Stroke. (2020) 22:403–6. 10.5853/jos.2020.0253733053956 Hope TM Seghier ML Leff AP Price CJ. Predicting outcome and recovery after stroke with lesions extracted from MRI images. Neuroimage Clin. (2013) 2:424–33. 10.1016/j.nicl.2013.03.00524179796 Lopes R Bournonville C Kuchcinski G Dondaine T Mendyk AM Viard R . Prediction of long-term cognitive functions after minor stroke, using functional connectivity. Neurology. (2021) 96:e1167–79. 10.1212/WNL.000000000001145233402437 Sale P Ferriero G Ciabattoni L Cortese AM Ferracuti F Romeo L . Predicting motor and cognitive improvement through machine learning algorithm in human subject that underwent a rehabilitation treatment in the early stage of stroke. J Stroke Cerebrovasc Dis. (2018) 27:2962–72. 10.1016/j.jstrokecerebrovasdis.2018.06.02130473028 Iwamoto Y Imura T Tanaka R Imada N Inagawa T Araki H . Development and validation of machine learning-based prediction for dependence in the activities of daily living after stroke inpatient rehabilitation: a decision-tree analysis. J Stroke Cerebrovasc Dis. (2020) 29:105332. 10.1016/j.jstrokecerebrovasdis.2020.10533232992179 Lin WY Chen CH Tseng YJ Tsai YT Chang CY Wang HY . Predicting post-stroke activities of daily living through a machine learning-based approach on initiating rehabilitation. Int J Med Inform. (2018) 111:159–64. 10.1016/j.ijmedinf.2018.01.00229425627 Tozlu A Edwards D Boes A Labar D Tsagaris KZ Silverstein J . Machine learning methods predict individual upper-limb motor impairment following therapy in chronic stroke. Neurorehabil Neural Repair. (2020) 34:428–39. 10.1177/154596832090979632193984 Stinear CM Barber PA Petoe M Anwar S Byblow WD. The PREP algorithm predicts potential for upper limb recovery after stroke. Brain. (2012) 135:2527–35. 10.1093/brain/aws14622689909 Murray NM Unberath M Hager GD Hui FK. Artificial intelligence to diagnose ischemic stroke and identify large vessel occlusions: a systematic review. J Neurointerv Surg. (2020) 12:156–64. 10.1136/neurintsurg-2019-01513531594798 Mokli Y Pfaff J Dos Santos DP Herweh C Nagel S. Computer-aided imaging analysis in acute ischemic stroke - background and clinical applications. Neurol Res Pract. (2019) 1:23. 10.1186/s42466-019-0028-y33324889 Straka MG Albers W Bammer R. Real-time diffusion-perfusion mismatch analysis in acute stroke J Magn Reson Imaging. (2010) 32:1024–37. 10.1002/jmri.2233821031505 Lansberg MG Straka M Kemp S Mlynash M Wechsler LR Jovin TG . MRI profile and response to endovascular reperfusion after stroke (DEFUSE 2): a prospective cohort study. Lancet Neurol. (2012) 11:860–7. 10.1016/S1474-4422(12)70203-X22954705 Justin P. FDA approves stroke-detecting AI software. Nat Biotechnol. (2018) 36:290. 10.1038/nbt0418-29029621226 Yahav-Dovrat A Saban M Merhav G Lankri I Abergel E Eran A . Evaluation of artificial intelligence-powered identification of large-vessel occlusions in a comprehensive stroke center. Am J Neuroradiol. (2021) 42:247–54. 10.3174/ajnr.A692333384294 GBD 2016 Causes of Death Collaborators. Global, regional, and national age-sex specific mortality for 264 causes of death:1980–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet. (2017) 390:1151–210. 10.1016/S0140-6736(17)32152-928919116 Stevens RD Sutter R. Prognosis in severe brain injury. Crit Care Med. (2013) 41:1104–23. 10.1097/CCM.0b013e318287ee7923528755 Holloway RG Arnold RM Creutzfeldt CJ Lewis EF Lutz BJ McCann RM . Palliative and end-of-life care in stroke: a statement for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. (2014) 45:1887–916. 10.1161/STR.000000000000001524676781 Morgenstern LB Zahuranec DB Sánchez BN Becker KJ Geraghty M Hughes R . Full medical support for intracerebral hemorrhage. Neurology. (2015) 84:1739–44. 10.1212/WNL.000000000000152525817842 AHwang Y Dell CA Sparks MJ Watson TD Langefeld CD Comeau ME . Clinician judgment vs. formal scales for predicting intracerebral hemorrhage outcomes. Neurology. (2016) 86:126–33. 10.1212/WNL.000000000000226626674335 Hemphill JC 3rd Bonovich DC Besmertis L Manley GT Johnston SC. The ICH score: a simple, reliable grading scale for intracerebral hemorrhage. Stroke. (2001) 32:891–7. 10.1161/01.STR.32.4.89111283388 Nawabi J Kniep H Elsayed S Friedrich C Sporns P Rusche T . Imaging-based outcome prediction of acute intracerebral hemorrhage. Transl Stroke Res. (2021). 10.1007/s12975-021-00891-833547592 Sennfält S Pihlsgård M Petersson J Norrving B Ullberg T. Long-term outcome after ischemic stroke in relation to comorbidity - an observational study from the Swedish Stroke Register (Riksstroke). Eur Stroke J. (2020) 5:36–46. 10.1177/239698731988315432232168 Russakovsky O Deng J Su H Krause J Satheesh S Ma S . ImageNet large scale visual recognition challenge. Int J Comput Vis. (2015) 115:211–52. 10.1007/s11263-015-0816-y Chan S Siegel EL. Will machine learning end the viability of radiology as a thriving medical specialty? Br J Radiol. (2019) 92:20180416. 10.1259/bjr.2018041630325645 ABeam L Kohane IS. Big data and machine learning in health care. J Am Med Assoc. (2018) 319:1317–8. 10.1001/jama.2017.1839129532063 SLiebeskind S Scalzo Editorial F. Machine learning and decision support in stroke. Front Neurol. (2020) 11:486. 10.3389/fneur.2020.0048634030088 Vayena A Blasimme A Cohen IG. Machine learning in medicine: addressing ethical challenges. PLoS Med. (2018) 15:e1002689. 10.1371/journal.pmed.100268930399149 Obermeyer Z Emanuel EJ. Predicting the future - big data, machine learning, and clinical medicine. N Engl J Med. (2016) 375:1216–9. 10.1056/NEJMp160618127682033 Marra A Ely EW Pandharipande PP Patel MB. The ABCDEF bundle in critical care. Crit Care Clin. (2017) 33:225–43. 10.1016/j.ccc.2016.12.00528284292 Cramer SC Le V Saver JL Dodakian L See J Augsburger R . Intense arm rehabilitation therapy improves the modified rankin scale score: association between gains in impairment and function. Neurology. (2021) 96:e1812–22. 10.1212/WNL.000000000001166733589538 Gui H Zheng R Ma C Fan H Xu L. An architecture for healthcare big data management and analysis. In: International Conference on Health Information Science. Cham: Springer (2016). p. 154–60. 10.1007/978-3-319-48335-1_17