The progress in micro-electromechanical system (MEMS) technology resulted in the availability of small, lightweight, and low-cost inertial measurement units (IMUs) conquering the motion tracking market (Seel et al., 2020). Therefore, IMUs are the most widely used type of wearable sensors for gait and balance analysis (de-la Herran et al., 2014; Shanahan et al., 2018); the quantification of gait with IMUs is sometimes referred to as InertiaLocoGraphy (ILG) (Vienne-Jumeau et al., 2020). IMUs typically consist of a combination of multi-dimensional gyroscopes, accelerometers, and often magnetometer sensors allowing the estimation of joint angles, gait and angular velocities, position and orientation in space via sensor fusion techniques (Sabatini, 2006). The wireless sensors can be mounted on various parts of the patient's body, for example, foot, lower leg, pelvis, torso, or integrated into garments and insoles, in order to measure movements of a specific body segment.

Accelerometers are most commonly used in motion analysis and assess the one-, two-, or three-dimensional acceleration of the sensor in terms of externally applied acceleration forces (Diaz et al., 2020). The measured signal is the sum of (1) the linear acceleration, namely the translation- and/or rotation-related instantaneous change of velocity, and (2) the earth's gravitational acceleration, which is approximately 9.81 m/s² in vertical direction near the earth's surface. However, these two components can only be differentiated completely in quasi-stationary scenarios.

Gyroscopes provide the one-, two-, or three-dimensional angular velocities of the body segment to which they are attached. The design typically relies on the Coriolis effect whereby a body moving freely in a rotating frame of reference experiences the Coriolis force acting perpendicular to the direction of applied motion and to the axis of rotation. Segment orientations and joint angles can be determined by integration of the resulting angular rates if initial values are known and measurement biases are removed. However, the biases of MEMS-based gyroscopes are temperature-dependent and time-varying, which makes it difficult to estimate them during movements (Woodman, 2007).

To overcome the disadvantages of both sensor types, accelerometers and gyroscopes, magnetometers are often included in IMUs [also referred to as magneto inertial measurement units (MIMUs)] to improve orientation measurements, namely heading. Heading describes the angle of the sensor with respect to the horizontal direction of the magnetic north. In a magnetically undisturbed environment, magnetometers measure this component and a vertical component of the local earth's magnetic field. However, these readings are typically noisy and affected by magnetic disturbances originating from objects containing ferromagnetic material or emitting magnetic fields as usually the case in indoor environments (Schauer, 2017).

As the measurement signals of inertial sensors are well-known to be subject to errors such as time-variant sensor biases and measurement noise, reliable motion tracking requires computationally complex algorithms including state estimation methods and kinematic models (Seel et al., 2020). Three-dimensional strap-down integration and suitable sensor fusion algorithms combining the different signals are applied to estimate the real-time orientation of an attached IMU (Schauer, 2017), as illustrated in Figure 3. More precisely, the orientation of the inertial coordinate system, which is aligned with the housing of the sensor, is estimated with respect to a three-dimensional inertial reference coordinate system. The accuracy and precision of a wearable sensor system depend on how many sensors are used, where and how the sensors are mounted, and on the utilized algorithms. For example, the sensor coordinate systems should be sufficiently well-aligned with a meaningful coordinate system of the body part of interest, to which the sensor is attached, or methods for non-restrictive sensor-to-segment calibration or automatic anatomical calibration should be applied (Seel et al., 2014b).

For clinical gait data analysis, further mathematical tools are required to extract spatiotemporal gait parameters and often anatomical models are utilized for extracting kinematic parameters. Commonly in a two-stage approach, first gait events and phases are detected and, secondly, spatial parameters are determined. Various approaches exist on how and to what detail gait phases are detected from IMU recordings. Reliable gait detection can be achieved by exploiting the angular rates from the gyroscopes (Bertoli et al., 2018) or by combining them with the accelerometer measurements using peak detection algorithms (Mariani et al., 2013). Automated methods deploy adaptive thresholds based on the subject's walking style (Bejarano et al., 2014; Seel et al., 2014a). Spatial parameters are obtained by either signal integration, kinematic gait models, or ML techniques (Yang and Li, 2012; Caldas et al., 2017). Major gait parameters, for example, stride length, walking speed, can be derived with the most commonly used setup of two inertial sensors that are placed on the feet/shoes (e.g., Schlachetzki et al., 2017) or on the shank. Especially when postural control and balance parameters shall be extracted, a third sensor is added either on the chest, pelvis or lumbar spine and the acceleration is used to calculate COM and sway parameters (Mancini et al., 2012b; Curtze et al., 2015; Hsieh and Sosnoff, 2021). Recently, it has been shown that several gait events can be obtained from a single IMU at the pelvis even in individuals with neurological conditions (Pham et al., 2017). Tracking of the lower body or full body motion tracking is facilitated by further sensors (e.g., Schepers et al., 2018; Teufl et al., 2019).

Available commercial systems for gait and balance analysis applied in neurological patients are, for example, Xsens MVN (Xsens Technologies B.V., Enschede, Netherlands) for full-body motion tracking with up to 17 sensors, Mobility Lab (APDM, Inc., Portland, OR, USA) with six IMUs and data analysis software for different test scenarios (Dewey et al., 2014; Mancini and Horak, 2016; Morris et al., 2019), or RehaGait (HASOMED, Magdeburg, Germany) with up to seven inertial sensors and gait data analysis software (Donath et al., 2016). The sensors and systems diverge in their software, namely algorithms for sensor fusion and parameter estimation, as well as in their communication and housing. The housing and its dimensions vary depending on the battery capacities and on-sensor storage (Diaz et al., 2020). Furthermore, different data collection modes are available, such as real-time streaming or post-recording data download, and their transmission to a computer or smart device. The provided sampling rates correlate with the number of utilized sensors and show a large range (22–320 Hz), although this parameter has a high impact on the accuracy (Caldas et al., 2017). Besides, the pricing for IMU-based gait analysis systems varies strongly with individual wireless inertial sensor being available at affordable prices. However, the more detailed the gait analysis software has been evaluated, the more expensive it is. This is one reason for the trend toward own investigations and open-source gait analysis software (e.g., Gurchiek et al., 2019). Therefore, before deciding on a sensor system, the complete application scenario and budget should be outlined and recent developments need to be taken into account.

Inertial sensors in PD. Inertial sensor technology applied in PD was able to reproduce the findings of distinct spatiotemporal gait characteristics including short steps, shuffling gait, and postural instability, specific for different disease stages and levels of motor impairment (Schlachetzki et al., 2017); cf. Section 2.3. A typical application is the instrumentalization of established clinical tests with IMUs with the goal of making the assessment rater-independent and gaining additional information (Palmerini et al., 2013). For example, Dewey et al. (2014) used the Mobility Lab system in an instrumented TUG and instrumented sway assessment in 135 PD subjects and 66 age-matched controls. For both tests, they identified multiple variables (e.g., stride length, turn duration, total sway area) that correlate with PD severity measures and differentiate PD subjects from controls. Instrumented gait analysis with IMUs offers the possibility to examine the differential effects of established and novel PD treatments on gait. A selection of exemplary studies is presented in Table 2.

For instance, Curtze et al. (2015) studied the effect of levodopa treatment on gait in a large cohort of patients and found that pace-related gait measures responded well to levodopa treatment, while balance parameters did not improve in the ON- compared to the OFF-state. Iijima et al. (2017) used 24 h single-accelerometer measurements from the trunk in order to track improvements in the gait fluctuations of PD patients after the addition/increase in dose of selegiline, showing a higher sensitivity than clinical scores. Recently, Cebi et al. (2020) used gait kinematics derived from IMU sensors placed at the hip and ankles to examine the therapeutic outcome of deep brain stimulation of the Nucleus subthalamicus (STN-DBS) on gait disorders in PD. Time to complete a 7 m walking task and number of steps were reduced and gait kinematics improved (stride length, ROM) 8 weeks after STN-DBS surgery in the DBS-ON compared to the DBS-OFF condition. In addition, freezers with a pre-surgical levodopa response of gait kinematics responded better to STN-DBS, indicating that the assessment with IMUss might be useful to predict the outcome of such treatments in specific patient subgroups.

Another popular application of IMUs in PD is as a tool to recognize and quantify FoG, a symptom which is rarely observed during clinical consultations since it occurs episodically. Freezing of gait usually appears in everyday life situations, i.e., during turning or walking through narrow doorways, and is associated with an increased risk of falls (Gray and Hildebrand, 2000; Bloem et al., 2004). Various sensor-based methods have been developed to objectively measure FoG in terms of number of episodes and episode duration (Moore et al., 2008; Rodrguez-Martn et al., 2017; Silva de Lima et al., 2017; Suppa et al., 2017; Pardoel et al., 2019). Sensor-based FoG detection opens up the possibility of monitoring FoG in the home environment of patients, which could facilitate the diagnosis and treatment of FoG (Suppa et al., 2017; Mancini et al., 2021).

Furthermore, IMUs are integrated in novel therapeutic cueing devices, which aim to monitor and treat gait disorders in PD. Cueing was shown to be effective in improving gait function in PD and a multitude of cueing paradigms exists (Muthukrishnan et al., 2019). Table 2 includes examples of cueing devices using IMUs to either administer gait-synchronized cues or to analyze the gait pattern in response to treatment. For example, the gait training tool CuPiD-system consists of wearable IMUs, a smartphone, and headphones to deliver intelligent auditory feedback on gait (Casamassima et al., 2014; Ferrari et al., 2016). Patients using the device showed improvements in maintaining cadence during prolonged walking, improved balance, and quality of life (Ginis et al., 2016, 2017). The GaitAssist system applies adaptive, rhythmic auditory cues and was used in the home environment of PD patients, who showed a trend toward reduced FoG episodes after several days of gait training with the system (Mazilu et al., 2015). Other cueing systems administer gait-synchronized sensory stimulation, which process IMU data online to analyze the gait while walking: Mancini and colleagues examined the effect of vibrotactile cueing at the wrist (VibroGait) and found reduced FoG, improved turning and trunk stability, increased first step duration, but reductions in gait speed and stride length (Harrington et al., 2016; Mancini et al., 2018; Fino and Mancini, 2020; Schlenstedt et al., 2020). Sijobert et al. (2016) developed a smart cueing device applying sensory, electrical stimulation at the lower leg via skin electrodes and found that the time to complete a walking task and the number of FoG episodes decreased.

These studies show that sensor-based gait measurements (1) might help to objectively examine treatment effects on gait disorders, (2) might facilitate the monitoring of treatment outcomes over longer follow-up periods, (3) may be used to predict the outcome of treatments in specific patient subgroups, and (4) could become integral part of new therapeutic methods.

Inertial sensors in MS. Shah et al. (2020) postulated that daily life monitoring with IMUs might be more sensitive to impairments from neurological diseases than laboratory IMU-based gait measures but that the analyzed neurological diseases (PD and MS) might require different gait outcome measures. Trunk-, shank-, or foot-placed IMUs have been frequently applied to measure gait and, especially, balance dysfunction in MS patients, commonly in the form of the instrumented TUG test (Shanahan et al., 2018): Spain et al. (2014) utilized IMUs to differentiate between mild MS, moderate MS, and control groups based on the variability in gait velocity, trunk motion, and sway (range, area). Craig et al. (2017) showed the reproducibility of the instrumented TUG results over two sessions and that stride velocity, cadence, and cycle time correlate significantly with disease severity and number of recent falls. IMU-based analysis has been found useful to detect even early changes in gait and balance in MS (Spain et al., 2012). Measurements with IMUs were able to reflect intra-individual changes in identified biomarkers associated with a change in clinical severity scores in a 12-month prospective study by Galea et al. (2017). Therefore, objective gait analysis with IMUs might increase the sensitivity of clinical and performance tests to monitor gait dysfunction in MS (Vienne-Jumeau et al., 2020). Moreover, spatiotemporal parameters from walking have been used to objectively measure MS disease specific characteristics, such as muscle fatigue, which could be helpful in monitoring and evaluating rehabilitation and treatment efficacy (Motta et al., 2016; Ibrahim et al., 2020). In the area of therapeutic aids and home-care, IMU-based fall detection is an emerging application for various gait disorders (Wang et al., 2020).

Although smart devices, such as smartphones or smartwatches, use inertial sensors as a technique, they are presented separately in this section due to their high presence and popularity in everyday life that makes them particularly interesting for long-term monitoring in home environments. Smart devices can be used as single sensor units like previously described IMUs, for example, by wearing a smart device on the hip for a postural control assessment (Kosse et al., 2015). The data can then be transferred and processed in the same way as previously described for IMUs. Usually, measurement setups are limited to two measurement locations, for example, one smartphone and a paired smartwatch. However, most frequently only one device is utilized (e.g., Chomiak et al., 2019; Hsieh et al., 2019). Available sampling rates for smart-device-based IMUs depend on the hardware (e.g., for Apple products³ (Cupertina, CA, USA), it is supposed to be at least 100 Hz), and can limit the range of applications. The reliability of smart device measurements for motion tracking is still being investigated (Vohralik et al., 2015).

In addition to regular IMUs, smart devices come with an integrated interface and specific software (“apps”) facilitating a user-friendly operation of the data assessment. Different apps provide different data collection modes, such as real-time streaming, recording, and post-recording wireless data download. In literature, mostly customized apps were used to record the sensor data and calculate gait or balance parameters on- or offline (Franco et al., 2012; Kosse et al., 2015; Chomiak et al., 2019), or to upload the data to cloud servers for offline gait analysis (Manor et al., 2018). As an advantage to standalone IMUs, smart devices promote direct biofeedback in the form of visual, auditory, or haptic signals. Due to these features, smartphones are often combined with IMUs for gait monitoring and therapy systems (e.g., Ferrari et al., 2016; Palmerini et al., 2017).

Although the use of smart devices to assess gait and balance is under extensive investigation, most applications are still under development. Multiple studies explore smart devices for balance assessments measuring trunk movements and postural stability but so far mostly in neurologically intact participants (e.g., Alberts et al., 2015a,b; Kosse et al., 2015; Hsieh et al., 2019). Roeing et al. (2017) reviewed 13 studies and found that five evaluated the validity of their smartphone applications for balance and risk of falls assessment; the results demonstrated strong concurrent validity with standalone accelerometry, 3D motion capture, and force plate measurements. Three of these studies included a measure of reliability revealing high ICC values for mixed variables (Mellone et al., 2012; Cerrito et al., 2015; Kosse et al., 2015; Roeing et al., 2017). Standardized clinical assessments, such as sit-to-stand evaluation (Cerrito et al., 2015; Marques et al., 2021), the TUG (Mellone et al., 2012; Ponciano et al., 2020), and postural balance (Hsieh and Sosnoff, 2021), were instrumented using a single smart device. Also applications in rehabilitation in the form of biofeedback loops with potential use at home, e.g., as a smartphone-based audio-biofeedback in order to improve balance during bipedal standing (Franco et al., 2012), are being evaluated. However, special research interest is on utilizing smart devices for gait assessment as there lies a great potential for long-term monitoring in everyday activities. Fall detection with smart devices is already available on the market in the form of Apple Watch Series 4–6 (Apple Inc, 2020). The accelerometer and gyroscope readings from the wrist are used in combination with a fall detection threshold yielding a high false-positive rate (Wang et al., 2020).

Most approaches aim at extracting gait parameters from the use of a single smart device (Ellis et al., 2015; Kosse et al., 2015; Manor et al., 2018). For example, Manor et al. (2018) created an app for systematic gait data recording and analysis that can be performed independently by the user either in the laboratory or at home with the smartphone placed in the trousers' front pocket. When comparing normal and dual-task trials in neurologically intact volunteers, average stride times derived from the app demonstrated high correlation with the simultaneously used instrumented mat in the laboratory. Lipsmeier et al. (2018) explored the potential of smartphones for assessing biomarkers in PD that might serve as outcome measures in clinical trials. The authors presented moderate to strong retest reliability and successful discrimination between PD and controls with increased sensitivity compared to traditional clinical scales (Buckley et al., 2019).

Despite regular gait assessment, smart devices were evaluated for continuous monitoring in PD for FoG detection and fall prevention. For example, Ellis et al. (2015) developed a mobile application with the smartphone at the front waist to track gait and its variability, an indicator for FoG in PD, presenting it as an alternative to conventional gait analysis technologies. Chomiak et al. (2019) utilized an iPod Touch, worn on the thigh, and ML to identify gait-cycle breakdown and freezing episodes of varying duration. Ahn et al. (2017) presented a system for FoG detection and visual cueing based on smart glasses (Android): The subject's movements are tracked using the inertial sensor from the glasses, which projects visual patterns in the case of a recognized FoG event. Furthermore, numerous smartphone applications have been designed for the assessment and monitoring of multiple health parameters in patients with PD (Monje et al., 2019). The apps combine questionnaires, cognitive, voice, and motor tasks providing repeated measures of the patients motor state along with valid and clinically meaningful knowledge of symptom evolution (Bot et al., 2016; Lakshminarayana et al., 2017; Lipsmeier et al., 2018). Similar applications are available for monitoring MS patients [e.g., elevateMS by Pratap et al., 2020].

These diverse applications of smart devices in gait and balance assessments reveal their future potential to be utilized for objective evaluation of treatments over short and long follow-up periods, closed-loop applications, and telemedical consulting in real-life environments.

Instrumented insoles are insoles that have integrated force or pressure sensors to measure changes in pressure between the foot and the ground. Force sensors measure the applied force discriminating the component of each axis that is measured, whereas pressure sensors are non-discriminating and thereby measure the combined ground reaction force (de-la Herran et al., 2014). Most commonly used insole sensors are capacitive, resistive piezoelectric, and piezoresistive sensors (de-la Herran et al., 2014). The measurement principle is based on the detection of voltage changes caused by fluctuations in electrical capacity or electrical resistance in semiconductor materials due to stretching or compression (Chen and Yan, 2020). The choice of sensor depends on the desired range of pressure/force, sampling rate, and sensitivity (Diaz et al., 2020). Insoles typically incorporate arrays of sensors measuring a spatial pressure/force profile over the plantar foot surface (Shanahan et al., 2018). The profile varies during the gait cycle and depends on a person's body weight: In healthy gait, the maximum vertical force is applied and, thereby, the maximum pressure occurs when the whole body weight is on one leg/foot during the stance phase (Clarke, 1980). No force is applied during the swing phase. The profile's spatial resolution depends on the number of integrated sensors in the insole, its temporal resolution on the applied sampling rate, and its sensitivity on the utilized sensor and analog-to-digital converter.

Available systems are among others the F-Scan (Tekscan Inc., Boston, MA, USA) with 3.9 force-sensitive resistors per cm², the Moticon SCIENCE pressure insoles (Moticon, Munich, Germany) with 16 capacitive pressure sensors, or WalkinSense (Kinematix SA, Sheffield, UK) with eight force-sensing piezoresistors. The latter two and other newer insole types often incorporate additional sensors such as an IMU (Arafsha et al., 2018). Besides, the available systems differ in the type of power supply, data transmission and storage, in the user operation, and associated analysis software.

The validity of discrete pressure and force measurements with insoles is comparable to optical motion capture and they show a high reliability within and between trials (Shanahan et al., 2018). From the profiles, spatiotemporal gait parameters (e.g., stride time, gait phases) can be extracted. However, patients with neurological gait disorders tend to walk slowly, shuffle, and perform short and dragged steps making it challenging for automatic gait event detection based on heel strike or initial contact (Pirker and Katzenschlager, 2017; Diaz et al., 2020). For balance analysis, insoles are regularly used to measure the COP to evaluate postural stability (Ma et al., 2016).

So far, the clinical application of instrumented insoles in PD patients has mostly been limited to the differentiation between PD and controls. Extracted gait and balance parameters have been used successfully for discrimination between the groups (Mazumder et al., 2018; Chatzaki et al., 2021), in line with the findings from established laboratory gait assessments. Furthermore, instrumented insoles have been investigated for their ability to recognize and quantify FoG in PD (Popovic et al., 2010; Shalin et al., 2020; Pardoel et al., 2021). Pardoel et al. (2021) combined features derived from a pressure-sensing insole and IMUs on the leg to detect FoG in 11 PD patients. The authors reported that the combination of both modalities outperformed classification models that used data from a single sensor type. In a small data sample (n = 5), Shalin et al. (2020) demonstrated that foot pressure distributions from 60 × 21 sensor-arrays could be used for FoG prediction (0.5–3 s before FoG onset). Therefore, together with inertial sensors, instrumented insoles could be integrated into therapeutic cueing devices for treating gait disorders in PD (cf. section 3.1).

Few studies utilized insoles to examine gait function in MS patients so far (Shanahan et al., 2018). Viqueira Villarejo et al. (2014) reported an increased plantar pressure during the stance phase and variability in step timing in MS compared to controls. Galea et al. (2017) quantified MS-related gait and balance deterioration over 12 months using EMG and insoles and observed decreases in gait speed and balance scores, and an increase in double support time. Doḿınguez et al. (2020) validated gait velocity and other parameters from a new insole system with an incorporated IMU against a common instrumented walkway in 205 MS patients. The results revealed a high correlation between devices in velocity, ambulation time, cadence, and stride length. Note that spatial parameters, such as stride length and stride wide, can only be derived from the IMU data (Farid et al., 2021).

Although the use of insoles is unobtrusive and, therefore, has a high potential in monitoring daily activities (Diaz et al., 2020), the hesitant use in research and clinical application may have practical reasons. For reliable measurements, diverse sole sizes must be available to cover the variety in foot sizes. Systems with multiple soles and validated analysis software can initially require a five-digit amount. Due to the mechanical stress during walking, the soles' life is limited. People must wear shoes that allow the use of additional insoles. Furthermore, shoes must be taken off and put on again to set up the measurement, an additional obstacle for elderly patient groups such as PD. However, for long-term monitoring in the future, the ease of integration in patients' everyday life could be an advantage.

Microsoft's Kinect is a motion-sensing device originally developed for gaming purposes and is one of the earliest motion capture technologies of its kind to be used in medical research. Owing to Kinect's long history in pose estimation, it has been well-assessed in various research settings and utilized in tracking different movement patterns. Throughout the last decade, several versions of Kinect have been produced: The introductory model Kinect 1 (2010) integrates a structured near-infrared light source with an accompanying sensor to capture the reflected light patterns, whereas Kinect 2 (2013) and Azure Kinect (2019) use wide angle ToF cameras (Zhang, 2012).

Although Kinect 1 is a well-established system, when compared to gold standard techniques, it provides only basic motion capture capabilities such as collecting temporal gait parameters, estimating single joint angles, or assessing postural control during reaching and balance tasks (Clark et al., 2012; Schmitz et al., 2014). In kinematic gait recordings, the system generally underestimates joint flexion and overestimates extension during walking in the sagittal plane. Here, stride timing measurements perform surprisingly well with the highest accuracy at low gait speeds, despite a high error in hip and knee displacement (Pfister et al., 2014). Clothing and different body shapes were discussed as possible reasons for measurement errors. Therefore, approaches that use additional multi-layer filtering, where the estimated pose is further refined through a synthetic library of posture variations, can alleviate some tracking inaccuracies, increase parameter precision, and allow for better recognition of occluded body parts (Shotton et al., 2011; Wei et al., 2012; Xu et al., 2013). Moreover, recording frameworks with multiple cameras were able to improve pose estimation accuracy and approximation of occluded segments at the costs of a higher setup effort (Gao et al., 2015).

Nonetheless, Kinect 1 has been used in several clinically-oriented studies to measure lower body biomechanics for determining stride time, length, and speed in healthy individuals (Gabel et al., 2012; Auvinet et al., 2015). In the context of PD, scientists used wavelet-based digital signal processing to analyze gait parameters and quantitatively distinguish gait phases with an accuracy of up to 93% (Muñoz et al., 2018). Spatiotemporal parameters were distinguishable in stage II and III PD patients compared to a control group, reaching a maximum accuracy of 97.2% after classification with a neural network (NN) (Ťupa et al., 2015). Likewise, Kinect 1 technology has been implemented in MS gait analysis to discern MS patients from neurologically intact controls by differences in the average walking speed and lateral body sway (Behrens et al., 2014), or ROM, stride length, and step width (Gholami et al., 2016).

The newer Kinect 2 system uses continuous-wave ToF technology instead of structured light, enabling a more stable data feed with an increased accuracy within the measurement range of 4m (Gonzalez-Jorge et al., 2015). In clinical assessments, Kinect 2 displayed an adequate performance when tracking joint center displacement (Napoli et al., 2017). The second generation demonstrates better accuracy in joint estimation and stays more robust to body rotation as well as occlusions during various movements like walking and jogging (Wang et al., 2015; Guess et al., 2017). Therefore, Kinect 2 seems to outperform Kinect 1 in locomotion tracking except for foot position tracking during standing, where a larger amount of noise is generated, possibly due to ToF artifacts (Otte et al., 2016). Kinect 2 reliably assessed spatiotemporal parameters during comfortable and fast-paced gait (Mentiplay et al., 2015). However, significant performance variations in different motion planes and incompatibility with certain functional movements still exist in Kinect 2 when compared to marker-based systems, especially in the context of compound joint movement (Mentiplay et al., 2015). The validity of lower limb joint kinematics depends on the camera's capture angle for recording the walking subject. Moreover, in treadmill walking, accuracy levels appear to vary across gait parameters, with temporal parameters based on heel strike having fewer errors than those based on toe-off, and their accuracy fluctuates with changing walking speeds (Xu et al., 2015). Linear pelvic and trunk ROM can still be tracked with reasonable precision at 70 and 90% of maximal locomotion speed, providing a reliable reference point across all velocities (Macpherson et al., 2016). In attempts to use multiple Kinect 2 cameras to achieve a higher tracking accuracy, several cameras have to be calibrated together via geometric trilateration. The distance between the subject and at least three recording cameras is measured through signal strength. When used to determine gait parameters, three Kinect 2 sensors show a much higher spatiotemporal reliability compared to a single Kinect 2 camera (Yang et al., 2016).

Kinect 2 has been applied in PD patients, where 92% of freezing episodes, 91% of tremor occurrences, and 99% of falling incidents could be detected with customized algorithms (Bigy et al., 2015). Moreover, Kinect 2 measurements in combination with customized algorithms were able to consistently produce results similar to a marker-based system and output significant differences between PD and control groups for stride length, gait, and swing velocity (Eltoukhy et al., 2017; Sabo et al., 2020). In MS patients, moderate and fast walking speed measurements agree with results derived from marker-based systems, but once more, only if combined with customized software or auxiliary ML-based classifiers (Bethoux et al., 2018; Elkurdi et al., 2018). Indeed, it seems that additional ML algorithms or NNs can frequently increase the validity and reliability with depth cameras (Rocha et al., 2018).

Microsoft's Azure is the most recent Kinect upgrade that supports additional features and several depth-sensing modes. Although part of the same production line, it has been specifically designed for distinctive non-gaming purposes such as research and health care use. Compared to Kinect 2, Azure has a higher angular resolution, lower noise, and better tracking accuracy (Tölgyessy et al., 2021). When used as a dual system consisting of two cameras, Azure outputs precise knee angles and demonstrates an overall improved validity over Kinect 2 (Ma et al., 2020b). During the estimation of sagittal hip and knee joint angles, a single Azure appears to have a superior depth resolution and shows better tracking performance when subjects walk at non-frontal camera viewing angles (Yeung et al., 2021). In treadmill walking, spatial gait parameters (e.g., step length and width) can be measured more reliably with Azure, though the accuracy of temporal parameters (e.g., stride duration) does not change significantly between the two models. Interestingly, Kinect 2 seems to outperform Azure regarding upper body tracking. However, an overall increase in the quality of lower extremity parameters and the additional introduction of integrated deep learning-based body tracking algorithms create appeal for Azure to be used in gait rehabilitation (Albert et al., 2020). As far as the application of Azure in PD and MS studies is concerned, to our knowledge there has not been any material published yet.

Intel's RealSense cameras stem from several generations of stereo depth cameras with a production start in 2015. The system comprises a left-right depth stereo camera pair and an additional color camera. The stereo cameras use textured light to ensure unambiguous image matching, which in turn enables more accurate depth measurements. Accordingly, stereotactic systems including RealSense are less sensitive to noise compared to other in-depth camera types, which allows for a more flexible experimental setup (Keselman et al., 2017; Zabatani et al., 2019). In motion analysis, the system can be used to measure a definite amount of spatiotemporal variables, however, joints with multiple degrees of freedom exhibit inaccuracies yielding difficulties for the forthcoming gait data analysis (Mejia-Trujillo et al., 2019). In general, RealSense seems to perform better at slow to normal walking speeds located in small to medium-sized environments (Hausamann et al., 2021). Auxiliary tools can be used to extend the current three-part system to up to six cameras to improve body shape and joint position tracking (Boppana and Anderson, 2019). Intriguingly, despite having an older production age, both Kinect 1 and 2 seem to rival RealSense's signal quality and capture range during walking (Mejia-Trujillo et al., 2019). Moreover, temporal parameters seem to exhibit slightly better accuracy when recorded with RealSense, whereas spatial parameters retain similar values to Kinect measurements (Gutta et al., 2021). While a few gait studies with Intel's RealSense exist, up to the present moment no publications known to the authors have used the technology to assess gait and balance explicitly in either PD or MS patient groups. However, RealSense convincingly holds the potential to be used in clinical research, whether as a new method for motion tracking or as a Kinect substitute (Clark et al., 2019).

Most recent 2D pose estimation approaches rely heavily on contemporary advances in deep learning, a branch of ML that employs NNs with many layers. With an annotated image data set where objects have been manually labeled, a NN can be trained via supervised learning to classify and track those objects. Pose estimation algorithms frequently use convolutional neural networks (CNNs) as their architectural foundation with multiple layers (e.g., Toshev and Szegedy, 2014). The greatest advantage of CNNs is their ability to learn feature representations directly from the data set in use, which removes the need for additional training data, thus, ensuring a straightforward experimental flow. Their convolutional structure produces 2D probability maps for the location of each body part after they had been trained to recognize image features that belong to specific shapes (e.g., knee, foot) (Wei et al., 2016). This establishes a statistical relationship between the input images and output pose key-points, which can be used to track pose in yet unanalyzed data and make predictions on the spatiotemporal appearance of tracked key-points.

In video data, 2D pose tracking represents a unique set of challenges and numerous network designs have been created to optimize both for speed and reliability in their specific study context. Contrary to static image analysis, images that have been extracted from video frames are often subject to motion blur, frequent body occlusions, unconventional subject positions and further represent large data sets due to the sheer amount of frames in a single video (cf. Figure 5). The continuously increasing amount of new pose algorithms also evokes the demand for largely manually annotated data sets that thematically represent the defined area of research: sports, outdoors, medical research, and many others (Sigal et al., 2010; Ionescu et al., 2013; Andriluka et al., 2014). To withstand these challenges, attempts have been made to increase the quality of parameter supervision by, for example, cross-correlating features in adjacent video frames or integrating various mathematical approaches with NNs, and to reduce the amount of required pre-labeled data (Ouyang et al., 2014; Szegedy et al., 2015; Tompson et al., 2015; Feichtenhofer et al., 2017).

In gait analysis, established preliminary models use standard cameras ranging from simple mobile phone cameras to multiple cameras accompanied with additional sensors such as IMUs or floor sensors (Alharthi et al., 2019; Viswakumar et al., 2019; Vaith et al., 2020; Stenum et al., 2021). The validity, reliability, and processing time of these models vary according to the type and quality of camera footage, computational system architecture, type and amount of training data used as well as many other factors. Therefore, choosing a suitable pose estimation model is strongly influenced by the experimental setting and might depend on the number of tracked legs, frequency of body part occlusions, subjects' clothing, and room background color. Until these and other issues are resolved, 2D pose estimation will not be applied on a wide scale in the clinical field. However, first studies indicating possible applications of this technique in neurological disorders have been published: Li et al. (2018) combined the outcome of convolutional pose machines with ML-based classification for discriminating disease and symptom severity in PD patients in tasks such as toe-tapping and stamping. Hu et al. (2019) successfully established a novel graph CNN to classify freezing episodes from regular gait in the TUG test of 45 Parkinsonian patients, recorded in frontal view.

Occlusion of body parts has continuously presented a challenge to 2D human pose estimation for gait analysis, especially when both legs are tracked simultaneously, as desired, for example, for analyzing gait symmetry in PD. Thus, advances have been made toward setting the pose in a 3D coordinate framework instead of 2D by subsequently generating a 3D environment from images obtained by a single RGB camera. Complementary 3D pose libraries can be used to create 2D projections from virtual camera views. In such cases, 2D pose estimation is performed on input images and then depth is calculated using an additional pre-existing 3D library as a reference (Chen and Ramanan, 2017). However, the employment of 3D libraries requires even larger amounts of annotated data. To address this challenge, specialized CNNs have been implemented to output 2D key-points together with body silhouettes, which are later synchronized with a mathematically generated 3D body mesh model to estimate full 3D pose (Loper et al., 2015; Pavlakos et al., 2018). Moreover, some networks specialize in the detection of individual people from group images and automatically crop out single subjects that are present on the input image, subsequently performing individual 2D pose tracking and later placing the obtained parameters into a virtual 3D environment (Moon et al., 2019). Recently, single-view 3D pose estimation has been integrated into the gait analysis of PD patients, where spatiotemporal parameters including step length, velocity, and cadence evaluated with a deep learning pose estimation algorithm seemed in good agreement with reference data obtained through pressure sensors (ICC > 0.9) (Shin et al., 2021).

The multi-view approach to 3D pose estimation is an alternative scheme that further reduces training set size and eliminates the need for large 3D libraries. One strategy is to train the network on images from multiple cameras before predicting the 3D pose from images obtained by a single camera (Rhodin et al., 2018). Specifically, a network is trained to predict the same 3D pose regardless of camera perspective and can perform 3D predictions solely based on 2D imagery. Other methods include algebraic and volumetric triangulation that are speculated to be more robust to occlusions or partial body visibility (Iskakov et al., 2019). Further strategies use so-called key-point coordinates instead of heatmaps (Pavllo et al., 2019) or employ a multi-stage architecture to reconstruct the 3D pose from 2D heatmap predictions at each CNN processing stage (Tome et al., 2018).

As multi-view 3D pose estimation approaches effectively deal with body part occlusions and simultaneously alleviate the need for large training libraries, they present a promising tool in gait tracking. Indeed, such models are able to qualitatively reproduce locomotion compared to marker-based motion capture, albeit still producing a small error rate in the final 3D pose (Nakano et al., 2020). Therefore, technical challenges of multi-view systems such as the setup of multiple cameras, triangulation, and more extensive processing make the experimental setting more demanding but at the same time offer an opportunity to improve the quality of gait parameters.

While numerous algorithms have been created in the attempt to improve the performance of 2D and 3D pose estimation algorithms, we will now briefly summarize several that have been pre-packaged as software toolboxes and are being used by a wider, non-specialist scientific community to track human motion promoting new fields of application (Table 3).

One of the earliest of such packages is DeeperCut, a multi-person pose estimation method based on the integer linear programming approach DeepCut (Pishchulin et al., 2016). Here, deep residual neural networks (ResNets) have been adapted inside a convolutional architecture in form of a sliding window-based body part detection (He et al., 2016). ResNets build on constructs known from pyramidal cells in the cerebral cortex: They utilize skip connections, or shortcuts to jump over some network layers and map nonlinearities. Moreover, DeeperCut features image-conditioned pairwise terms or architecture components that indicate the presence of other body parts in the vicinity of a tracked point and group these body parts into a valid pose configuration (Insafutdinov et al., 2016). Published in 2018, DeepLabCut is a more recent tracking toolbox. Although a CNN architecture as well, DeepLabCut significantly differs from its predecessor DeeperCut by implementing pre-trained ResNets, which fine-tune the already existing node weights following the tracked body part. Therefore, DeepLabCut exhibits a faster performance and requires a smaller amount of pre-labeled images for training. After network processing, the user can readily access spatial coordinates and the existential probability of every tracked body part, stored in the form of x- and y-coordinates for each video time frame (Mathis and Warren, 2018; Mathis et al., 2018). Figure 5 shows an example of 2D motion tracking with DeepLabCut. Additional reconstruction of 3D kinematics with DeepLabCut is possible by either establishing individual networks for each camera view or training a single network that generalizes across all views (Nath et al., 2019).

OpenPose is a real-time 2D pose estimation approach developed for motion processing of multiple individuals on a single image. An integral part are Part Affinity Fields (PAFs), a set of 2D vector fields that encode the orientation and location of limbs on the analyzed image. Moreover, PAFs are bottom-up representations of unstructured pairwise relationships between detected body parts that enable the reconstruction of the full-body pose while decreasing the total computational cost. As with any multi-person tracking algorithm, OpenPose faces obstacles like subjects present on the image at different positions or scales and body part occlusions (Cao et al., 2019). OpenPose has recently been implemented with multiple synchronized cameras to evaluate motor performance in a 3D pose framework. Compared to a marker-based system, the mean absolute error of points tracked during walking equaled less than 30 mm, excluding 10% of cases where OpenPose initially failed to recognize the correct body segment during 2D estimation (Nakano et al., 2020).

Several other prominent pose estimation toolboxes exist which have not yet been frequently featured in gait research: Anipose is an open-source toolkit designed to augment the existing 2D tracking methods for accurate pose tracking in a 3D setting. It deploys optimization on the calibration, triangulation, and filtering over multiple camera views that accompanies the processing by antecedent NN packages (Karashchuk et al., 2020). DeepPoseKit aims to resolve the limitations of over-parametrization by pre-trained ResNets and the lack of robustness in GPU-based approaches. The pipeline is based on alternative confidence map processing methods, multi-scale inference, and GPU-oriented convolutional layers (Graving et al., 2019). Lastly, AlphaPose is another open-source multi-pose estimator featuring a regional multi-person pose estimation (RPME) framework (Fang et al., 2017). During training, the RPME pipeline detects single humans on the image by establishing bounding boxes around each individual. Afterwards, single pose estimation is performed on each bounding box and the output is further refined (Xiu et al., 2018; Li et al., 2019).

In conclusion, several toolboxes have already been tested on human footage of walking and running. Among the software packages in Table 3, OpenPose has been most extensively evaluated both in 2D and 3D gait estimation. In 2D video analysis, mean absolute errors of temporal parameters are smaller than differences arising from natural variations in the walking pattern making temporal changes detectable in healthy gait (Stenum et al., 2021). Step length estimation accuracy depends on the participant's position in the camera field of view, with central positions resulting in lower error rates. Unlike gait speed that reaches accuracy levels similar to the gold standard, errors in sagittal hip, knee, and ankle angles are in proximity of test-retest errors in the same plane. In an underwater running setup, the accuracy of predictions for 2D-joint marker positions extracted with DeepLabCut seems to match manual labels with a mean difference of fewer than three pixels (Cronin et al., 2019). Although not compared to a marker-based system, DeepLabCut seemed sensitive enough to differentiate between closely-spaced running cadences with a high test-retest reliability of the mean stride data. In 3D motion capture obtained with OpenPose, DeepLabCut, and AlphaPose, significant kinematic differences at hip and knee occurred in comparison to marker-based systems (Needham et al., 2021). Here, tracking accuracy of the ankle unexpectedly performed better than other joints, possibly owing to more precise manual annotation during training due to its apparent anatomical position. When compared to IMUs, OpenPose seems to exhibit tracking discrepancies in joint angles of up to 14 (Gu et al., 2018; D'Antonio et al., 2020, 2021). Despite these reports, Sato et al. (2019) employed a pipeline with OpenPose to analyze cadence in daily clinical movies recorded from the frontal angle in healthy controls (n = 117) and two PD patients with prominent FoG. The authors reported a discrimination performance for mild PD gait from controls of 0.75–0.96 (area under curve) and for comparing gait sequences before vs. after DBS treatment (n = 1) of 0.98. On the whole, as the demand for efficient and cost-effective technologies for gait analysis grows, deep learning architectures are still lacking in precision but continue to improve rapidly and are increasingly being implemented into clinical studies and home assessments (Xue et al., 2018; Viswakumar et al., 2019; Sibley et al., 2021).