We performed a systematic search in PubMed, Web of Sciences, Embase, and PsycINFO for relevant studies published before September 30, 2023. The search keywords consisted of three groups of keywords related to (a) ML, (b) SSD, and (c) VB. In this systematic review, the PICO (Population, Intervention, Comparison, Outcome) framework was employed with the following criteria:

Population: Schizophrenia spectrum disorder (SSD), including schizophrenia, schizoaffective disorder, and other delusional disorders (27);

Intervention: Machine learning models (ML);

Comparison: Medical records of patients or clinical violence risk assessment scales;

Outcome: Violent behavior (VB), defining as an attempt or action to harm a target, assault, robbery, aggression toward property, actions resulting in physical injury, child abuse, sexual abuse, threatening or causing injury with a weapon, verbal aggression or threatening, and violent crimes, e.g., attempted or completed homicide (28, 29).

This study was conducted in concordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) (30) and Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) guidelines (31).

All studies developing ML models for predicting VB in SSD patients were included. The development of a ML model in medicine includes the following stages: data acquisition, data preparation, ML model development, model evaluation, hyperparameter tuning, and model validation (32). We aimed to review the articles that developed and evaluated ML models for the prediction of VB in SSD patients. Hence, studies that only employed statistical models by using an ML subset (e.g., logistic regression) and did not either evaluate or validate the performance of their generated model were not included in this review. The exclusion criteria consisted of 1) Records that did not study patients with an SSD diagnosis, 2) records that did not predict VB, 3) records that did not employ an ML method, 4) records that were not available in the English language full-text, 5) editorials, commentaries, letters, conference abstracts, books, and review articles, and 6) animal studies.

The selection process began with removing the duplicated records. Then, two authors (MP and AA) independently reviewed the article titles and abstracts and selected the relevant papers for the full-text screening process. The same authors (MP and AA) independently conducted the full-text screening of the selected records for eligibility. Any discrepancies were settled by discussion and, if necessary, referred to a third author (GC).

Two authors (AA and MT) conducted the data extraction. We collected data about the authors, year of publication, sample size, characteristics of the patients, ML model and validation techniques, input variables (i.e., demographics), output variables (VB), additional assessments, and key findings from every included record. Also, reported measures of the area under the receiver operating characteristic curves (AUROC), balanced accuracy, predictive power, P-value, sensitivity, specificity, positive predictive values (PPV), and negative predictive values (NPV) were collected. Cross-validation performance was defined as a training dataset because it involved data “seen” by the machine, whereas “unseen” data from a held-out test set or external cohort was treated as validation.

To bypass the limitations of meta-analyzing heterogeneous datasets, one author (MT) implemented a novel comparative approach, ranking each ML model’s performance within individual studies and then averaging ranks across studies to identify the best overall performing ML model.

To assess the risk of bias (ROB), we employed the Prediction Model Risk of Bias Assessment Tool (PROBAST) (33). It is a tool for assessing ROB and the applicability of diagnostic and prognostic prediction model studies. PROBAST evaluates 4 domains of participants, predictors, outcome, and analysis in the study by 20 signaling questions. signaling questions of the PROBAST checklist and its guidance notes for rating ROB and applicability are fully provided in PROBAST checklist section of the Supplementary Material. These questions facilitate structured judgment of ROB in the studies of predictive models. We used the explanation and elaboration document that describes the rationale for including each domain and signaling question and guides researchers to use them to assess the ROB and applicability concerns. Also, to assess the ROB in the studies that employed more than one ML model, we selected the ML model with the best performance (best AUROC or accuracy).

The search strategy employed in this systematic review yielded 3941 articles. Following the removal of duplicates, 2142 articles remained for further assessment. After assessing the abstracts, 250 articles were deemed suitable for full-text screening. A total of 18 articles satisfied the eligibility criteria and were included in the final analysis ( Figure 1 ). Table 1 shows the characteristics and extracted data of the included articles.

Various features were identified in the included studies as the predictor variables of VB in SSD patients. We can classify most of them into sociodemographic, clinical, metabolic, and neuroimaging groups. Most of the features were consistent in multiple studies, except for some discrepancies, which will be elaborated upon.

Based on the results of our ROB assessment using the PROBAST guidelines, all studies except for two (43, 47), had some bias due to a small sample size, different violence definitions, and the inability to satisfy the study’s purpose. Although most of the studies had high ROB, the most important limitation arises from their limited sample sizes. According to the PROBAST guidelines, to achieve a low ROB in the analysis domain, the number of participants with the outcome relative to the number of the input variables should be equal to or higher than 20 (33). Only four reviewed studies had low ROB in the analysis domain (41, 42, 44, 47). Another reason for high ROB arises from the divergent definitions of violence and the use of different scales across the studies. We defined VB as an attempt or action to harm a target, assault, child or sexual abuse, and violent crimes. Whereas, Hofmann et al. (2022) included verbal aggression in the definition of VB (41) and four studies evaluated the ability of ML models to classify patients with previous criminal offenses (including VB) from non-offenders (46, 48–50). Also, some studies have evaluated the power of ML models in predicting VB (e.g., homicide) among offenders with SSD disorders (39, 42, 45). Although many studies represented high ROB in at least one field according to the PROBAST guideline (33), most of them (11/18) showed low concerns regarding applicability in the field of violence prediction in patients with SSD. Table 3 and Figure 2 illustrate the results of the quality assessment process.

Previous research has shown an acceptable power for ML models in predicting VB in populations broader than SSD patients (68, 69). In this article, we reviewed the role of ML in predicting VB in patients with SSD. According to our findings, the predictive performances of the ML models varied across the reviewed papers. However, ML models performed better in studies that employed more intricate methodologies for model development and evaluation. These findings suggest that a well-designed ML model could be a potential tool for VB prediction in SSD patients, and could be beneficial in warning the caregivers to seek prevention techniques and stop them from further harmful acts in clinical and forensic settings. Among the reviewed ML models, GB showed the best performance in VB detection. Also, we reviewed the most discriminating features in violence prediction of SSD patients. Age (34, 37, 43, 45, 47) and olanzapine equivalent dose at the time of discharge (39, 45, 46, 48, 49) were the most repetitive variables found to be associated with violence across the studies.

While direct comparison of results among studies was challenging due to the differences in sample characteristics, some insights were obtained. First, about two third of the studies (11/18) could reach values above 75% for both AUROC and accuracy, indicating that ML can be a promising tool for the accurate prediction of VB among SSD patients. Second, the studies demonstrated diverse performance in predicting VB among SSD patients, with an AUROC ranging from 0.56 to 0.95 and an accuracy range of 50.27% to 90.67%. However, the performance ranges within each study were narrower when comparing different ML models. Considering that many studies employed similar ML models and input variables, the observed diversity in performance appears to be partly influenced by the variations in study designs. This suggests that future similar studies could enhance their results not only by focusing on ML model selection or input variable choices but also by paying attention to the details of model development to mitigate biases.

In addition, there exists considerable divergence among the reviewed studies with regard to the methodologies employed for both feature selection and cross-validation. These two components play pivotal roles in the trajectory of ML model development, serving to mitigate overfitting and augment overall model performance (70). Within the included studies, a mere seven undertook data-driven feature selection utilizing ML techniques prior to model training as a preemptive measure against overfitting (38, 39, 41–43, 45). Notably, one study adopted a post hoc approach, selecting features subsequent to the computation of variable importance for each employed model independently (51). Additionally, sixteen studies embraced diverse methods for cross-validation, while two studies opted to forgo its application (35, 40). This heterogeneity in model development practices across the reviewed studies poses a significant obstacle to synthesizing their respective findings.

Therefore, our significant challenge was comparing ML models by integrating the results of different studies due to variations in sample characteristics, including differences in input and output variables distribution. To address this challenge, we devised a ranking method that enabled us to assess the overall success rate of commonly used methods. Based on our findings, the GB model exhibited notably superior average performance. However, it is essential to note that this does not necessarily imply inherent weakness in the other models. Instead, it highlights the favorable results achieved by the GB model in the specific context of the studied field.

GB is a subset of ensemble machine learning models, which also includes common models like classification trees and RF (32). This approach enables the effective handling of big data and also the handling of missing values in the predictors (32). While common ensemble techniques like RF rely on straightforward averaging of models within the ensemble, GB stands out for its step-by-step, sequential strategy for selecting the best predictor (71). This notable flexibility empowers GB to be highly adaptable to specific data-driven tasks (71). Due to its unique characteristics, GB outperformed other ML models in predicting VB in SSD patients, particularly due to its effective handling of a large number of predictors. Nevertheless, it is noteworthy that several other ML models, including SVM, LASSO, NNET, RF, decision trees, PDA, MLP, elastic net, and LR, in the studies reviewed, also achieved AUROC values exceeding 0.9 (41, 44, 47, 51). This highlights the substantial predictive potential of these alternative ML models in addition to GB.

Notably, various studies have explored the influence of age on VB risk, yielding diverse findings. For instance, Tzeng et al. (2004) associated younger age with a higher risk of VB (34). In contrast, four other studies observed that older age correlated with an increased tendency for VB in SSD patients (37, 44, 45, 47). Yet, Chen et al. (2015) found no significant correlation between patients’ age and VB risk. While the majority of previous research aligns with Chen et al. (2015) and negates the association between age and VB risk (35, 72), there are outliers such as Soyka et al. (2007) who identified older ages as linked to a higher VB risk in SSD patients (73). This variability underscores the need for further research to ascertain the precise impact of age on VB occurrence in SSD patients.

Contrary to age, the reported gender effect on VB occurrence risk was quite consistent among studies, which showed that male sex was associated with a higher risk of VB (34). These findings confirmed most of the previous studies that reported a higher prevalence of VB among male SSD patients (73, 74), as the general population (17). Furthermore, Gou et al. (2021) and Yu et al. (2022), but not Chen et al. (2015) and Yu et al. (2022a), found that lower educational levels could predict VB occurrence (38, 43). This was in line with previous research that found lower educational levels to be significant predictors of VB among SSD patients (75, 76) and the general population (17). These disparities suggest that further studies on larger populations are required to determine the exact effect of the educational level of SSD patients on their VB tendency.

Moreover, in terms of the effect of occupational status on VB tendency, Kirchebner et al. (2022) reported a significant relationship between unemployment and VB in SSD patients (42), which confirmed the findings by Karabekiroğlu et al. (2016). Conversely, Chen et al. (2015) and Wang et al. (2020) found no correlation between employment status and VB (35, 37). This divergence could be a result of different definitions of violence in these studies; indeed, Kirchebner et al. (2022) and Karabekiroğlu et al. (2016) studies, unemployment was able to differentiate SSD patients with serious VB (e.g., homicide) from patients with minor VB (e.g., property damage). On the other hand, two other studies trained ML models to differentiate SSD patients with any kind of VB (serious or minor) from patients without VB (35, 37). This suggests that unemployment does not seem to be associated with the overall risk of VB among SSD patients, but it increases the risk of serious VB among offenders.

Regarding the clinical features, two studies reported that the presence of positive symptoms (35, 43) was correlated with an increased risk of VB, which was consistent with previous research (72, 74, 77). However, another study suggested that the presence of specific positive symptoms, including delusion of persecution and auditory hallucination, decreases the risk of VB (40). This controversy indicates that different types of delusion may have varying effects on the occurrence of VB (40). Also, Sonnweber et al. (2021, 2022) found a favorable predictive power for the PANSS total score of the patients in two different studies (39, 45). This is in line with previous studies that demonstrated higher PANSS total scores in violent patients, compared to non-violent patients (78, 79). Moreover, consistent with previous research (80), Gou et al. (2021) found that the risk of VB occurrence is higher among SSD patients with higher scores in the BPRS hostility subscale. Furthermore, higher scores in BPRS total score, BPRS withdrawal factors, PCL-SV, HCR-20 (38), and SDSS (43) successfully predicted VB in SSD patients across the reviewed studies. While the BPRS and PANSS scales assess various domains of SSD, including positive and negative symptoms (55, 81), the PCL-SV scale is specifically designed to evaluate psychopathic traits in patients, which is not directly associated with SSD (82). This indicates that aside from psychotic symptoms, additional symptoms like patients’ personality profiles, including psychopathy and impulsivity, may have relevance in predicting VB among individuals with SSD. Altogether, these suggest that by training ML models with certified psychiatric rating tools, we can significantly improve the accuracy of predicting VB in SSD patients, which can be highly beneficial in clinical applications.

Chen et al. (2015) found negative symptoms to be correlated with a decreased risk of VB (35), which is in line with previous studies that found depressive and other negative symptoms to be associated with a lower occurrence of VB in SSD patients (73, 77). Furthermore, the effect of the age of disease onset was controversial across the reviewed studies. While Sonnweber et al. (2021, 2022) reported that younger age of disease onset correlated with the probability of VB, Chen et al. (2015) and Wang et al. (2020) did not find a significant relationship between the age of disease onset and VB occurrence. The findings of previous research in this field are also divergent. Indeed, while Caqueo-Urízar et al. (2016) found VB to be more prevalent among patients with younger age of illness onset, Nolan et al. (1999) did not find any significant differences between the age of onset of violent and non-violent patients. Therefore, further research is warranted to determine the effects of disease onset on the VB occurrence in SSD patients, as it can help the early detection and treatment of patients at higher risk of VB.

While most studies evaluating the prescribed daily olanzapine-equivalent dose at the time of discharge from previous hospitalizations have reported a positive association with the risk of VB (39, 46, 48, 49), there is an exception in one study that reported the opposite (45). The divergence in findings can be attributed to the different focus of the Sonnweber (2022) study, which specifically differentiated between homicide committers and patients who committed other types of VB (45). It is logical to assume that higher doses of antipsychotics are prescribed to patients with more enduring symptoms, as they are reported to be more prone to engaging in VB in some studies (83). However, some previous studies found no significant association between the disease severity or prescribed dosage of antipsychotics and the risk of VB in patients with SSD (84, 85). This highlights the need for further research to better comprehend the relationship between disease severity and prescribed antipsychotic dosages in the occurrence of VB among SSD patients.

Previous research has shown that SSD patients’ previous history of violence is significantly correlated with increased risks of VB, such as recent violence episodes (86), history of a recent assault (87), previous history of aggression (74), and a previous violent conviction (87). Consistently, Sonnweber et al. (2021) reported previous conviction history as a significant predictor of VB in SSD patients (39). Moreover, Wang et al. (2020) found that a history of more than five times of hospitalization increased the likelihood of VB tendency in patients. However, Tzeng et al. (2004) reported that the lifetime number of hospitalizations was not correlated with an increased risk of VB occurrence in SSD patients. This disparity could be due to the differences in the psychiatric history assessment across the studies, as Tzeng et al. (2004) evaluated a broader variable (lifetime hospitalization), while other studies assessed the recent hospitalization history (88, 89) or a more distinguishing variable (≥ 5-lifetime hospital admissions) (37).

Finally, two studies have observed that neuroimaging variables were robust predictors of VB in SSD patients. Yu et al. (2022) found decreased whole-brain gray matter volume, right inferior parietal thickness, and left frontal pole volume to be predictors of VB. Consistently, Gou et al. (2021) reported disruption in the structural and functional MRI of the temporal, frontal lobes, cingulate gyrus, and striatum can predict VB in SSD patients. Also, a systematic review of 21 studies, revealed that reduced volumes of the frontal lobe in patients with schizophrenia are associated with a higher rate of VB occurrence (90). This is not surprising, as previous research mentioned a prominent role for frontal and temporal lobes and cingulate gyrus disruptions in developing VB (91). Considering the role of the frontal cortex in controlling disinhibited behaviors (e.g., impulsiveness, aggressiveness, and violence), patients with disrupted frontal cortex are more likely to present VB (91, 92). Although previous research established the involvement of the hippocampus and amygdala in emotional processing and in the development of VB (91), the predictive value of these regions was not assessed across the reviewed studies. In conclusion, our knowledge in the field of ML-based prediction of VB in SSD patients by training MRI data is still limited, and future research is required to clarify its potential.

This study has some limitations. First, the sample sizes of most studies were small, considering the number of input variables, which can influence their analysis results. Second, the study samples across the reviewed articles were heterogeneous, as most of them studied clinical inpatients, while some studied forensic inpatients, and one included only outpatients’ data. Also, some studies only included male patients. Third, the outcome definitions differed within studies. For example, while most of the studies classified SSD patients into violent and non-violent, some others distinguished patients with serious types of VB (e.g., homicide) from other types of VB. Fourth, the reviewed studies were conducted in countries with different healthcare systems, which could have a significant impact on violence among SSD patients. Fifth, most of the studies did not select time-dependent features for VB prediction, which substantially lowers the ML model performance. Finally, none of the reviewed articles performed external validation, which can significantly diminish the generalizability of their findings. Therefore, future research with more homogenous methodologies and both internal and external validations seems to be necessary.

The outcomes of the ML models employed by the reviewed studies have yielded compelling findings, highlighting the significance of continuing along this research trajectory for further exploration and advancement. More in detail, while the ML models’ performance in VB prediction among SSD patients was divergent, yet promising, our comparative analysis demonstrated that GB outperformed other ML models. Considering the heterogeneity of ML model applications and study populations across the reviewed articles, there is substantial potential for further research in this field. Furthermore, the absence of external validation in the majority of the included articles reduces the generalizability of their findings. Indeed, subsequent research endeavors, employing comparable models, outcomes, and predictors, in extensive clinical samples, are imperative to substantiate the certainty of the current findings and ascertain the applicability of the developed ML algorithms.

Moreover, given the rapidly growing trend in the application of various artificial intelligence tools in medical contexts, it appears likely that in the next years ML models can be also utilized for VB prediction in SSD patients. Indeed, while the performance of ML models varied across the reviewed studies; several models demonstrated excellent predictive abilities with an AUROC exceeding 0.9. This highlights the potential for developing reliable ML models through further well-designed studies. Upon validation through external assessments, these models could effectively predict VB in real-world clinical settings. Consequently, the development of clinical assessment tools integrating patient data could facilitate the early identification of individuals highly susceptible to VB, whether in outpatient or inpatient settings. The utilization of such tools enables timely preventive interventions, such as providing social support and rehabilitation, adjusting medications, and considering more intensive therapeutic approaches, like electroconvulsive therapy. Implementing these measures could significantly alleviate the burden of VB on patients, healthcare systems, and society at large.