This study was based on information provided by the Enroll-HD database (9). Enroll-HD is a worldwide longitudinal observational study whose sites are located in North America, Latin America, Europe, Australia and New Zealand. Among other goals, Enroll-HD was designed to provide information about the dynamic phenotypic spectrum of HD and to promote the acquisition of knowledge about standards of care to inform clinical decisions, improving the health outcomes for the participant/family unit (9).

We used the periodic dataset containing Enroll-HD participants which meet the criteria for inclusion into the dataset as of November 1, 2015 (PDS2, Wave 1 sample). Cross-sectional data from baseline visits was obtained, with the data set being composed of 4,146 participants. From these, we excluded 971 participants whose number of CAG repeats was <36 (455 genotype negative and 516 family controls). The participants with a genetic diagnosis of HD (i.e., a CAG repeat length on the larger allele >=36) were divided into premanifest and manifest subjects. According to the Enroll-HD data handling manual, participants were classified as premanifest if they had the gene expansion for HD (larger CAG allele ≥36) but no clinical diagnosis of HD, [i.e., diagnostic confidence level (DCL) <4 on question 17 of the Unified Huntington's Disease Rating Scale (UHDRS)]. The manifest group was composed by participants with the gene expansion for HD (larger CAG allele ≥36) and a clinical diagnosis of HD based on the presence of unequivocal motor signs, i.e., DCL from the UHDRS = 4. A comparison between premanifest (N = 861) and manifest (N = 2,314) subjects revealed that while only 1.3% of premanifest patients presented history of psychosis, this number was much higher when considering manifest subjects (10.8%, p < 0.001, Fisher's exact test). Hence, we decided to run the analyses considering only manifest subjects. Figure 1A summarizes the exclusion process we applied in this study.

First, we performed univariate analyses in order to investigate differences between HD patients with (N = 248) and without (N = 2,055) history of psychosis. Associations between dichotomous variables were assessed with the Fisher's exact test. All continuous variables were tested to assess whether they follow a Gaussian distribution using the Shapiro-Wilk normality test. Two groups (history of psychosis vs. no history of psychosis) were compared using the Mann–Whitney U-test since data were determined to not follow a normal distribution. Then, a binary logistic regression was performed to determine which variables (among general clinical characteristics, medical history of substance abuse and psychiatric and behavioral problems, motor and functional capacity and cognitive performance) were significant associated with history of psychosis. A backward stepwise regression was used and all variables described in Table 1 were included in the initial model, except those exhibiting some degree of interdependency: (i) mother OR father affected by HD; and (ii) variables representing history of abuse of specific drugs (marijuana, heroin, cocaine, etc.), since they are dependent on the variable “history of drugs abuse.” Therefore, the following variables were included in the initial model: age, sex, age at motor symptoms onset, age at HD clinical diagnosis (based on the presence of unequivocal motor signs, DCL = 4), whether the mother was affected, number of CAG repeats, medical history of (alcohol use disorders, smoking, drugs abuse, depression, irritability, violent/aggressive behavior, apathy, perseverative/obsessive behavior, cognitive impairment, suicidal ideation), total motor score (TMS), total functional capacity (TFC) score, symbol digit modalities test (SDMT) (number of correct responses), verbal fluency test (number of correct responses in 1 min), Stroop interference test (number of correct responses), trail making test (TMT) parts A and B (time to complete and number of correct responses) and mini-mental state examination (MMSE) score. The backward stepwise selection was automatically performed using the SPSS software version 25.0 (SPSS Inc., Chicago, IL, USA) and the removal testing was based on the probability of the likelihood-ratio statistic based on conditional parameter estimates. The goodness of fit of the logistic regression model was assessed by the Hosmer-Lemeshow test as well as a Receiver Operating Characteristic (ROC) curve.

In order to validate our findings, the same variables included in the logistic regression model described above were tested in an independent cohort of patients with HD (Wave 2). Specifically, the Wave 2 data were composed of new Enroll-HD participants (N = 4,752 manifest subjects) whose information was released by the Enroll-HD as of October 31, 2016 (PDS3, Figure 1B).

Lastly, we evaluated the percentage of patients that presented pre-morbid psychosis (i.e., psychosis antedating the clinical diagnosis of HD) and the current presence of psychotic symptoms [defined as a score ≥1 in the psychosis subscale of the Problem Behaviors Assessment–short version (PBA-s)]. The PBA-s psychosis subscale is calculated as the sum of delusions / paranoid thinking score + hallucinations score (questions 9 and 10 of the PBA-s, respectively).

Machine learning—also known as pattern recognition—is a branch of computer science that involves developing algorithms that can learn from patterns of data, and subsequently able to make predictions on previously “unseen” observations (10). These algorithms are able to identify patterns of interactions among multiple variables and facilitate predictions and stratification of individual subjects' clinical outcomes (11). Machine learning has recently gained traction in biomedical studies due to their ability to analyze data from multiple observations and varied sources—also known as “big data” (11, 12). Machine learning approaches can result in highly accurate predictive models that support important clinical decisions such as selection of treatment options, preventive strategies, and prognosis orientations.

Machine learning algorithms are typically implemented in three stages: (1) algorithm training, (2) algorithm testing; and (3) validation. First, the data are divided into two groups (i.e., training and testing sets). Stage 1: the “training” set is used to train the algorithm and identify a set of optimal algorithm parameters. Stage 2: the “testing” set is used to examine whether the algorithm is able to generalize from the training set and calculate algorithm's prediction performance using accuracy, sensitivity and specificity metrics. Notably, it is a common practice to use a k-fold (i.e., k = 5 or 10) cross-validation method to separate algorithm training and testing data sets. Stage 3: once the algorithm has gone through the “training” and “testing” phase, it is evaluated using a “novel” evaluation data set—which was not included in the algorithm training or testing stages (13). In this study, we examined five machine learning algorithms: (i) Least Absolute Shrinkage and Selection Operator (LASSO) (14); (ii) Elastic Net (15); (iii) Support Vector Machines (SVM) (16); (iv) Random Forest (17); and (v) class-weighted SVM (18).

The same variables included in the Binary Logistic Regression (described in section Searching for Factors Associated With Psychosis: Conventional Statistical Analysis) were included in the machine learning approach. The main motivation here was to examine which demographic and clinical characteristics can individually distinguish between HD patients with and without psychosis. The machine learning algorithms were implemented using the Python programming language (19) through the Scikit-learn machine learning package (20). Missing data were imputed by replacing missing predictor variables with the mean. Predictor variables (Table 1) were normalized between zero and unity and together with corresponding categorical labels (0 – no history of psychosis; 1 – history of psychosis) used as an “input-target” pair for machine learning analyses.

Machine learning algorithms used in this study can be divided into three broad categories: (1) penalized linear regression (LASSO and Elastic Net), (2) Kernel-based (SVMs) and (3) Ensemble decision trees (random forests). Penalized linear regression methods use a classical linear regression approach albeit with additional penalty parameters to facilitate selection of most relevant variables or remove those that are redundant (21, 22). The penalized linear regression algorithms employed in this study use a logistic function to model probability and categorical outcomes (14, 15). On the other hand, Kernel-based methods use linear and non-linear kernel functions (e.g., polynomial, linear, and Gaussian) to “map” training data onto a higher dimensional space where a decision function able to separate both clinical groups is derived. Lastly, ensemble decision tree methods begin by constructing multiple decision trees which are subsequently combined by taking a majority (i.e., mode) of the predicted scores (17, 23).

Nearly 11% of motor manifest individuals with HD presented with a history of psychosis (N = 248 out of the 2,303). Demographics and clinical characteristics of manifest patients with HD with and without history of psychosis are shown in Table 1. The CAG trinucleotide repeat length was similar in both groups. Patients with a history of psychosis exhibited a higher frequency of behavioral problems and worse motor, functional capacity and cognitive scores than patients without a history of psychosis (Table 1).

Regarding the multivariate analysis, younger age at HD clinical diagnosis, lower number of CAG repeats, clinical history of [alcohol use disorders, depression, violent/aggressive behavior and perseverative/obsessive behavior], lower TFC score, and longer time to complete TMT-B (meaning worse cognitive performance) remained as significant factors associated with the history of psychosis in the final model (step 19). The results are presented in Table 2. The logistic regression model was significant [Hosmer-Lemeshow goodness of fit test (step 19): Chi-square = 11.4; p = 0.2] and the predicted variability resulted in an area under the curve (AUC) of 0.793 in the ROC analysis (Supplementary Figure 1A). The logistic regression results were corroborated by an external validation, since the relevant factors associated with psychosis in HD described above were very similar in the Wave 2 sample analysis [Supplementary Table 1; Hosmer-Lemeshow goodness of fit test (step 18): Chi-square = 7.2; p = 0.5. AUC = 0.816 in the ROC analysis (Supplementary Figure 1B)].

Our additional analyses revealed that among HD patients with history of psychosis, 31.6% currently have psychotic symptoms and 55.3% were pre-morbid (i.e., the age of psychosis symptoms preceded the age of clinical diagnosis of HD). Among these patients, the psychotic symptoms started in mean 4.11 (±6.26) years before the clinical diagnosis of HD. In addition, we observed that patients with HD who had a history of psychosis had higher scores in all the behavioral subscales of the PBA-s in comparison with patients with no history of psychosis (Supplementary Table 2).

All algorithms were capable of distinguishing individual HD patients with history of psychosis from those without a history of psychosis with prediction accuracy ranging from 71 to 73%. These results were established using a 10-fold cross-validation using the Wave 1 dataset only. All models were significant (chi-square p < 0.05, Table 3). A confusion matrix—which represents predicted labels against true labels (0 – no history of psychosis, 1 – history of psychosis) and a ROC curve for the weighted SVM algorithm are shown in Figures 3A,B, respectively.

Figure 3C shows a bar graph representing weighting factors assigned to each variable by a weighted SVM based on their relevance in distinguishing individuals with and without history of psychosis. The most relevant variables in identifying individuals with and without history of psychosis were the TFC score, worse performance at cognitive tests (longer time to complete the TMT-B and lower scores at SDMT), as well as history of perseverative/obsessive behavior. In addition, younger age at clinical HD diagnosis was found to be relevant in distinguishing individuals with and without history of psychosis.

To further validate our results, the machine learning algorithms developed using the Wave 1 sample were tested on “novel” or previously unseen observations from the Wave 2 validation sample. Results obtained by the machine learning models in identifying subjects with and without history of psychosis from Wave 2 sample are shown in Table 4. All validation models were significant (chi-square p < 0.05, Table 4). In particular, the weighted SVM reported the best accuracy on the Wave 2 validation sample.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.