In-vivo measurement of joint contact forces (JCF) provides highly accurate measures of tissue loading (Bergmann et al., 2016). Tissue loading information is crucial for the understanding of disease progression, injury prevention, rehabilitation, and even the designing of new artificial joints or limbs. However, in-vivo measurements require invasive instrumentation (Bergmann et al., 2016), which cannot be readily extended to the study of both healthy and pathological participants. To circumvent the problems of in-vivo measurements, computational musculoskeletal models have been developed, which provide a non-invasive method of estimating JCFs (Delp et al., 2007). To calculate JCFs, musculoskeletal modelling software require two sources of information–body segment kinematics optical cameras, and ground reaction forces (GRFs) from force plates. Logistically, it is very challenging to acquire GRFs outside a laboratory, given that force plates have to be either embedded into the ground or integrated within a bulky instrumented treadmill.

Increasingly, researchers have turned to machine learning (ML) to train a statistical model which learns complex patterns that map easier-to-collect biomechanical predictors onto harder-to-collect biomechanical outcomes in the laboratory (Liu et al., 2009; Johnson et al., 2019b; Stetter et al., 2020; Liew et al., 2021; Boukhennoufa et al., 2022). Current studies have used predictors such as body segment kinematics from optical cameras (Johnson et al., 2019a; Giarmatzis et al., 2020; Liew et al., 2021; Boukhennoufa et al., 2022), inertial measurement units (IMUs) (Lee and Park, 2020; Mundt et al., 2020; Stetter et al., 2020; Wang et al., 2020), markerless motion capture (Boswell et al., 2021), or using a combination of wearable sensors like IMUs, electromyography (EMG), and pressure insoles (He et al., 2019; Rane et al., 2019; Zhu et al., 2020; Camargo et al., 2022; Moghadam et al., 2023). Most ML studies in biomechanics have focused on the prediction of GRFs and joint moments during various locomotion patterns, such as walking and side-step cutting (Liu et al., 2009; Johnson et al., 2019b; Stetter et al., 2020; Wang et al., 2020). Neural networks are the most common ML method used in contemporary biomechanics research for predicting kinetic variables (Johnson et al., 2019a; Johnson et al., 2019b; He et al., 2019; Stetter et al., 2020; Wang et al., 2020; Boswell et al., 2021; Boukhennoufa et al., 2022), although methods such as gradient boosting have been used (Wang et al., 2020; Camargo et al., 2022).

Although predicting GRFs and joint moments have their clinical utility, such measures may not accurately reflect tissue load measures, such as JCFs (Walter et al., 2010; Matijevich et al., 2019). The capacity to accurately estimate JCFs using ML may radically transform the way we measure the biomechanical markers of joint diseases and monitor the effects of treatments. Increasingly, studies have begun exploring the role of ML in estimating muscle forces (Rane et al., 2019; Moghadam et al., 2023) and JCFs (Ardestani et al., 2014; Rane et al., 2019; Giarmatzis et al., 2020; Zhu et al., 2020), by training ML models on outcomes derived from force-sensor implants (Ardestani et al., 2014; Zhu et al., 2020), or from musculoskeletal models (Rane et al., 2019; Giarmatzis et al., 2020; Moghadam et al., 2023). For JCFs, only the knee has been the focus of investigation (Ardestani et al., 2014; Rane et al., 2019; Giarmatzis et al., 2020; Zhu et al., 2020), and only walking has been studied (Ardestani et al., 2014; Rane et al., 2019; Giarmatzis et al., 2020; Zhu et al., 2020).

Although an increasing amount of research has been done in developing ML models for predicting joint kinetics, several methodological issues remain. If ML models learn statistical relationships between a set of inputs and output, it is interesting to speculate as to the nature of the relationships learned, and the transferability of such knowledge across different settings. At its core, the relationship that connects motion to forces is determined by Newton’s Laws of Motion. If the relationship learned by ML models are approximating known physical laws, it suggests that ML models can be applied to biomechanical domains beyond that trained by the model. For example, the ability of ML models trained during a walking task, but applied to a running task, and vice versa. A related issue that affects the transferability of ML performance is the number of epochs that neural networks are trained on (or the number of iterations when using boosting). In the wider ML fields, early stopping of the training process when a prior criterion threshold is reached is thought to reduce statistical overfitting and improve the transferability of performance–i.e. the Bias-Variance tradeoff. However, ML models in biomechanics have used a fixed number of training epochs/iterations, without evaluating if this affect prediction performance.

This study aims to develop ML models to predict the outcomes of lower-limb JCFs of the ankle, knee, and hip joints, using 3D joint angles obtained using optical cameras. We hypothesise that the prediction errors of our ML models would be less than current minimal detectable change (MDC) values of traditional musculoskeletal modelling practices (e.g., range between 0.43 and 1.53 bodyweight [BW] (Price et al., 2017). The secondary aim of the present study is to determine the effect of varying training epochs on ML performance, with the hypothesis that a greater number of training epochs will always lead to a drop in prediction performance. The third aim of this study is to determine if training an ML model on data from one gait type can predict outcomes from another gait type. We hypothesised that if ML models are learning statistical relationships that reflect fundamental laws of physics, then the performances of ML models trained and tested on different gait types will be similar to the performances of models trained and tested on identical gait types. The last aim is to determine if increasing the sample size by combining both walking and running datasets can improve the prediction performance, compared to ML models trained only on a single gait dataset.

Findings from this study represent several innovation points. First, we will be able to determine if ML can be used to predict JCFs across the three major lower-limb joints in both light and high-impact activities. Second, we will be able to determine the impact of over parameterisation in deep neural networks on the smoothness of predictions of our JCFs. Lastly, we will be able to determine if ML models can be translated across gait patterns, providing insights into the nature of relationships learnt by ML in biomechanics.

The raw waveform of the predictors and outcomes used in the present study can be found in the Supplementary Material. The performance metrics of the best model for each outcome can also be found in the Supplementary Material. The observed and predicted mean waveform for each of the nine outcomes are presented in Figures 2–4. A general observation is that training using too few epochs (<100) results in predicted JCF waveforms with high “wiggleliness”.

This is the first study to the authors’ knowledge to predict JCFs across all three major lower-limb joints in walking and running using motion capture-based kinematics. Our findings supported the first hypothesis in that our ML models could predict JCFs within the range of MDC values reported for JCFs. A previous reported study reported that the knee JCF MDC values were between 0.43 and 1.53 bodyweight (BW) (Price et al., 2017). However, these performances were not replicated when training and testing were performed on different gait types. The second hypothesis was not supported in that training up to 1000 epochs did improve the ML performance in some instances, but degraded performance in other instances. In contrast to the third hypothesis, ML models trained and tested on different gait types had clinically significantly worse performance (four times worse) than models trained and tested on identical gait types. Lastly, combining both walking and running gait data did not appreciably alter ML performance compared to ML models trained and tested on a single gait type.

Previous ML studies predicting the knee JCF have reported a correlation between the observed and predicted forces between 0.85 and 0.94, and a Normalised RMSE (NRMSE) (%) between 4.5% and 13.3% (Ardestani et al., 2014; Rane et al., 2019; Giarmatzis et al., 2020). For our knee JCFs, our average correlation magnitude was 0.87, and relative RMSE was 15.4%. A notary caution when comparing studies is the differences in error metrics used. Presently, the calculated relative RMSE reflects the integral of the errors across the gait cycle investigated, but the NRMSE used in previous studies reflected the average RMSE across the gait cycle (Ardestani et al., 2014; Rane et al., 2019; Giarmatzis et al., 2020). The much better predictive performance of Giarmatzis et al. (Giarmatzis et al., 2020) compared to the present study, could be due to the large number of test trials available for model training (n = 54, number of observations = 4784), and the inclusion of GRFs as well as joint angles for prediction.

When considering the relative RMSE of the ML models on JCFs presently, the performance was slightly worse than a previous study on running on joint moments (Liew et al., 2021). The best relative RMSE values of the vertical JCFs in running of the ankle (11.5%), knee (10.4%), and hip (11.4%) in the present study, but the same metric values of the sagittal plane joint moments of the ankle, knee, and hip, were 5%, 7%, and 12%, respectively (Liew et al., 2021). However, a previous study included 490 samples in their training dataset with 27 joint kinematics as predictors (Liew et al., 2021). Johnson et al. (Johnson et al., 2019b) reported a relRMSE in the 3D knee joint moments of 13.8%–31.8% in walking and 7.8%–31.7% in running, with 570–646 and 233–884 samples in the combined training and testing datasets. Interestingly, the same relative prediction performance was achieved in walking, as in running, even though the biomechanical variables were “noisier” in running compared to walking, due to established issues like soft-tissue artefacts. For example, the best relative RMSE values of the vertical JCFs in walking of the ankle (15.5%), knee (8.1%), and hip (8.9%) were similar to that of running in the present study.

Conventional ML wisdom of the Bias-Variance trade-off predicts that ML performance on a separate test set will degrade after an optimal number of training epochs is reached. However, a general pattern can be observed that the greater the number of training epochs, ML prediction performance in general improved. In addition, the predicted waveforms became “smoother” and more comparable to the original JCFs. One study predicting joint moment waveforms trained all ML models using 200 epochs (Liew et al., 2021), whilst another used 1000 epochs (Wang et al., 2020), and some others not reporting this hyperparameter (Johnson et al., 2019b). The present study findings suggest that the results of prior studies could have been improved if a different number of training epochs were to be used. It is challenging to suggest an optimal training epoch that is generalisable to all ML situations. However, the present study suggests that if the goal is to achieve high-performance “smooth” waveform predictions, training epochs should generally be >100.

It is interesting to speculate on the statistical mechanisms for why a low number of training epochs results in “wiggleliness” of the waveforms. For irregular data (functions not observed at equidistant time points), integration weights are required to weight the different RMSE values differently, which could influence the “wiggleliness”. However, even for regularly spaced time-series data “wiggleliness” can happen. This could be due to a lack of a missing smoothness penalty in the loss function during training. Interestingly, increasing the training epochs appeared to result in the network “learning” the smoothness of the outcomes. This present finding could be attributed to the double descent phenomenon (Nakkiran et al., 2021). This phenomenon predicts that with a greater number of training epochs, the learned relationship on the training data achieves near perfect fit, resulting in an interpolation through the data (Belkin et al., 2019). It may be that for prediction problems where smoothness in the prediction of outcomes are desired, early stopping for ML algorithms may not be optimal (Belkin et al., 2019). It is interesting to note also that ReLU networks, such as XCM, can approximate smooth functions of any order (Zhang and Wang, 2022) and are essentially spline interpolators (Savarese et al., 2019). Previous work has suggested that the number of layers with ReLU activation units could influence the smoothness of predictions (Savarese et al., 2019). Whilst both the activation function and number of layers were constant across all models presently investigated, future studies that investigate the effects of these architectural parameters on the smoothness of prediction is essential.

The poor ML performance when training and testing on data from different gait types suggest the statistical relationship learned by ML models may not fully approximate physical laws sufficiently, to allow for cross-gait prediction application. One reason for this could be that the values of biomechanical variables between different gait types have different peak values, different timing of the peak values, and different spread of values. The ML model algorithm used presently is modelling the conditional distribution of the mean values of the outcome given a set of predictors, which neglects other aspects of the outcome, such as its spread (e.g., variance, kurtosis). Increasingly, deep learning algorithms that can simultaneously model the conditional distribution of both the mean and spread of the outcome are being developed (Kook et al., 2022), although their potential impact on cross-gait prediction has not been explored.

Our experimental design precludes us from disentangling the effects of different gait types and the effects of speed, as the cause of the poor ML performance. Speculatively, we hypothesise that the poorer cross-gait predictive performance is primarily caused by the different biomechanical requirements of distinct gait types (Schache et al., 2015), rather than speed effects. This is because distinct biomechanical characteristics are observed when walking and running at the same speed, such as a much greater hip power contribution in the former than the latter (Schache et al., 2015). Also, the shape of kinematic and kinetic waveforms has greater variation between gait types, than between speed variations (Schache et al., 2015). Future studies investigating the limits in generalisability of the ML performance across different distinct gait types and variants within gait types (e.g., different speeds) should be explored.

This study is not without limitations. First, ideally, ML models to predict JCFs should be developed using direct in-vivo measurements collected via instrumented implants (Bergmann et al., 1993; Fregly et al., 2012). However, direct in-vivo measurements are very challenging to collect and are often performed on patients with orthopaedic disorders, where the performance of more strenuous physical activities is not possible (Bergmann et al., 1993; Fregly et al., 2012). This explains why the number of participants where direct in-vivo data are available is very low (e.g., n = 2 in the Knee Grand Challenge). Given that ML requires much more data than musculoskeletal modelling, training a ML model using current publicly available direct in-vivo data is not possible. Developing ML models to predict JCFs based on musculoskeletal models represents the most feasible way at present, to fully realising the potential of bringing biomechanical measurements from the lab into clinical environments. Second, there are many optimisation methods available to calculate the muscle forces needed to quantify JCFs (Trinler et al., 2018). The accuracy of our ML model in estimating JCFs will only be as accurate as the accuracy of the initial musculoskeletal modelling approach in quantifying JCFs, and the latter should be based on the intended application of the ML model.

Third, we used predictors derived from motion capture cameras which although portable, are not ubiquitously available in the clinic and the field. Wearable sensors, such as accelerometers, represent the most clinically feasible methods of measuring body motion. However, wearable sensor signals may be “nosier” than kinematics collected from optical cameras. For example, one study which used IMU variables as input resulted in a RMSE knee extensor moment of 1.13 Nm/kg (Stetter et al., 2020), whilst another study using optical camera inputs had a RMSE of 0.25 Nm/kg (Liew et al., 2021) during running. Whether the performance of ML modelling to estimate JCFs using these alternative motion technologies would match that of traditional motion capture camera needs to be investigated. Lastly, the present study used biomechanical features that have been averaged across trials and time-normalised. These processing steps may result in over optimistic ML performance, given that “noise” to the signals are reduced. The extent “noise” should be removed by signal pre-processing in ML studies should be based on the intended use case of the ML model. If the ML model is intended for real-time streaming of step-by-step JCFs, then ML models should be trained on the original signals (Wang et al., 2020; Camargo et al., 2022). However, if the ML model is intended for “post-hoc” prediction of the average gait cycle’s JCFs, then our approach may be suitable.

ML can be used to predict JCFs of the lower limb during walking and running, to a degree that is within the current MDC values of JCFs. When using deep learning models, like in the present study, training using too few epochs (<100) generally leads to not only poor prediction performances but excessive “wiggleliness” of the waveforms. ML models trained on one gait type cannot be applied to another gait type. If ML models are required for cross-activity usage, that training needs to be done on data from all intended activities.