Parkinson’s disease (PD) is a neurodegenerative disease characterized by the degeneration of dopaminergic neurons of the substantia nigra resulting in bradykinesia, rigidity, tremor, and postural instability (Revuelta et al., 2015; Li et al., 2019). Over 50% of the PD patients who have lived with the disease for more than 10 years are affected by freezing of gait (FOG) (Okuma, 2006). In addition, FOG may occur in 25% of patients with early PD (Moore et al., 2008). FOG is defined as not being able to start or continue walking and feeling as if the feet have been “glued” or “magnetized” to the ground (Handojoseno et al., 2012). This condition causes both physical and psychological distress. Approximately 60% of PD patients experience falls each year, and they also have to endure multiple falls caused by complications and fall-related injuries (Wood et al., 2002; Moore et al., 2008; Allcock et al., 2009; Latt et al., 2009; Contreras and Grandas, 2012).

The response of FOG in PD to pharmaceutical treatment is limited (Okuma, 2006). However, previous works have shown that cueing-based training has specific effects on gait, freezing, and balance (Lim et al., 2005). Many works have aimed to develop small wearable devices that can detect FOG episodes (Han et al., 2003; Moore et al., 2008, 2013; Bachlin et al., 2009; Jovanov et al., 2009; Mazilu et al., 2012; Pepa et al., 2015; Zach et al., 2015). Gait recognition is a gait assessment tool that uses a machine learning (ML) algorithm. Researchers have attempted to develop a wearable system to improve the accuracy of identifying FOG events. To optimize the system, multiple aspects were investigated, including different sensors and their locations, a subset of the features extracted, different gait recognition models, and hyperparameters. Bächlin et al. (2009) used a threshold algorithm based on the freezing index and signal power to recognize FOG. Their model was more sensitive when sensors were worn on the hips. Mazilu et al. (2012) showed that the mean latency of FOG detection increased linearly with the window size, but the classification performance increased quickly and then stabilized. Using an ML algorithm such as random forest (RF), a single sensor was sufficient for FOG detection, and the placement of this sensor had little effect on the detection. Ensemble methods such as boosting and bagging are more appropriate than other methods (Mazilu et al., 2012). Mikos et al. (2017) found that features maximized their mutual information (MI) at different window lengths. The efficient parameter optimization method in gait recognition has not been fully studied. We found that when the dataset is very large, optimizing the hyperparameters is a very time-consuming process, so an efficient parameter optimization method needs to be discussed. Some works have studied the effects of feature extraction parameters, while the interaction among parameters has not received enough attention. We propose a statistical experimental design, i.e., an orthogonal experimental design (OED), to optimize the hyperparameters and analyzes the interaction.

Furthermore, by utilizing such a wearable system, timely rhythmic cues can be provided after FOG is detected. These systems have advantages in shortening the duration of FOG and, by providing immediate (at least hundreds of milliseconds) rhythmic cues as a response to the FOG signal, will further improve these systems. Therefore, FOG prediction is necessary to solve the response problem and further avoid intervention failure. An unsupervised feature learning decision tree can be used to perform FOG prediction (Mazilu et al., 2013). Mazilu et al. (2015b) explored the association of gait with electrocardiogram (ECG) and skin-conductance response (SCR) features and then fit the data to a multivariate Gaussian distribution, which can be used for FOG detection. In previous studies, parameters such as the window size and the defined duration of precursor signs of FOG (preFOG; all the references to preFOG in this article mean preFOG in PD) are based on experience (Mazilu et al., 2013, 2015b). However, the principles of objectification, multipurpose use, and simplification (OMS) have been the trend in the development of a novel behavioral assessment for PD (Asakawa et al., 2016a, b, 2019).

In this study, an RF model evaluated with an episode-based strategy was developed to predict upcoming FOG. This model can be adopted by wearable devices to activate an early intervention to avoid some of the FOG episodes.

The parameters related to feature extraction included the window size, sliding step length, and preFOG duration. The preFOG duration (LogWorth: 2.364, p-value < 0.01) had the largest impact on the kappa value, followed by the window size (Figure 5). The window size (LogWorth: 3.216, p-value < 0.01) had the largest impact on the F1 score followed by the preFOG duration (Figure 6). Thus, there was a certain interaction between the window size and the preFOG duration defined in the experiment because of the statistically significant effect from the source item “window size^∗preFOG duration” (Figure 6).

Generally, the increase in the window size had an obvious positive effect on the F1 score, the increase in preFOG duration had an obvious negative effect on the F1 score, and the sliding step length did not affect our experiments (Figure 7). After analysis in the Minitab, the best combination of parameters in feature extraction was obtained and represented by a tuple (window size, step, and preFOG duration): (500, 20, and 250).

As the feature dimension is relatively high, this work also selects subsets of features and obtains the most important set of features {“F03,” “F05,” “F07,” “F09,” “F10,” “F11,” “F12,” “F13,” “F14,” “F15,” “F16,” and “F17”}. The OED was used to optimize the hyperparameters of the RF classifier. The optimal combination of parameters was presented as a tuple (n_estimators, max_features, max_depth, min_samples_split, and min_samples_leaf): (800, 0.1, 4, 4, and 4). In this tuple, max_features is the maximum ratio of features that can be used in a single subtree of the RF; “0.1” means a maximum of 924×0.1 = 92 features can be used.

With the optimized parameters, the RF model was trained and evaluated with an episode-based strategy. Under application conditions, FOG can be recorded as an episode, but normal gait cannot be evaluated as an episode. Therefore, we chose the false-positive rate, hit rate, and mean prediction time (MPT) for model evaluation. The false-positive rate, hit rate, and MPT were used as the indexes for evaluation of the prediction system. The false-positive rate increased with the hit rate. The false-positive rate, hit rate, and MPT were 27%, 68%, and 2.99 s, respectively.

Our key assumption was that FOG is not a sudden episode. In the period before FOG occurs, the patient’s kinematic features change. Based on this assumption, a prediction model was constructed, and the OED was used to optimize the hyperparameters to improve efficiency. To the best of our knowledge, this is the first time that the OED has been used for the ML-based prediction of FOG in PD. The interactions among the feature extraction parameters and the performance were first analyzed. The OED makes analyzing the effects of hyperparameters and finding the optimal ones more cost effective.

The OED was introduced to acquire optimized parameters involved in feature extraction. Having implemented the experimental design, the interactions were analyzed, the optimized parameters were obtained, and the efficiency of the test was improved. The existence of interactions hinted that it was necessary to tune the parameters simultaneously using combinatorial experiments and not a one-factor experimental design. Mikos et al. (2017) elucidated how the window size affects the MI between the feature and the FOG classification, which is a measure of the correlation between variables, and their works showed optimal window lengths for FOG classification vary across feature types. Mazilu et al. (2013) illustrated the effects of preFOG duration on the prediction and illuminated gait parameter changes prior to FOG (Ferster et al., 2015), but they still used the empirical and a priori preFOG duration. Sixteen runs from the OED were used to analyze the effects of parameters such as the window size, preFOG duration, and sliding window step length on the F1 score for preFOG identification, while the full factorial design of three factors with four levels required 64 runs. The workload was low, and the optimal parameters were obtained. The optimized preFOG duration and window size also implied the length of the preFOG period. With the improvement in the model training efficiency, more models can be trained with the same data scale and computing resources. Therefore, training personalized FOG prediction models for different individuals becomes possible. This is a possible direction to solve the problem of the poor generalizability of the model to different patients in real-world applications.

The classifier was evaluated by using a leave-one-patient-out, episode-based strategy and obtained an MPT of 2.99 s for FOG prediction with a false-positive rate of 27% and a hit rate of 68%. The performance of the model was not excellent in the patient-independent evaluation but was still comparable to previous works. As in the previous work by Mazilu et al. (2013) the FOG prediction performance was highly patient-dependent. The optimal parameters were not suitable for all patients, although our work solved these problems to some extent. We also developed a model with a patient-dependent method and obtained high performance with an F1 score of 0.89. In many ML models, the training and test data came from different patients, the identically distributed assumption is often violated, leading to poor performance. When the data from a patient were split into training and test data, the two datasets were often highly interdependent, leading to a good performance, but the model was overfit.

There were some limitations to our research. First, we did not include patients with a Hoehn–Yahr score greater than three since most of these patients have difficulty moving and are at risk of falling. The results might be affected if severe patients were included. Second, our method should be evaluated with the data of PD patients without FOG who were not included in our study. Third, the number of patients included in our study is small, weakening the experimental efficacy and supporting evidence. Future studies will be dedicated to improving the performance of the FOG prediction system by eliminating these defects.

Overall, a prediction classifier beneficial for early interventions for FOG was obtained. The OED was helpful in optimizing the hyperparameters. Larger-scale studies are needed.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

This study was approved by Ruijin Hospital affiliated to Shanghai Jiao Tong University School of Medicine. The patients/participants provided their written informed consent to participate in this study.

KR and SC designed and supervised the study, and double-checked the statistical analysis. ZC and GL collected the sample, performed the statistical analysis, and drafted the manuscript. All authors contributed to the article and approved the submitted version.

ZC, YL, XY, and KR were employed by the company Gyenno Science Co., Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.