We can adaptively operate our bipedal body by cooperating with others in an emergency (Hutchins, 1995; Fujii et al., 2016) and sometimes competing with others (Yamamoto et al., 2013; Fujii et al., 2015b). Current technology can succeed in reproducing such real-time adaptation in video game tasks (Mnih et al., 2015) and overcoming unpredicted deficits (Yoshihara et al., 2007; Cully et al., 2015), although such adaptation is limited to a certain part of the agent's body. However, with regard to bipedal locomotion, which is more dynamically unstable than that of more than four-legged species (Golubitsky et al., 1999), researchers have not paid attention to the adaptive movements when motor commands change suddenly in response to a change in the situation, such as chase-and-escape behavior (Kamimura and Ohira, 2010; Fujii et al., 2015b), which have been acquired in over the course of evolution as biological (Carvalho et al., 2012) and social (Helbing et al., 2000) features essential for life activities. For example, it is considered that a sudden intentional direction change opposite to the original direction, such as in interpersonal sports (Fujii et al., 2015b), is quite difficult to achieve and thus has been ignored in the fields of robotic engineering (Koolen et al., 2016; Kuindersma et al., 2016) and computational neuroscience (Taga et al., 1991; Song and Geyer, 2015), with the focus primarily being placed on bipedal adaptation to external disturbances. Here, we refer to this as switching adaptation in bipedal locomotion because both motor commands (i.e., situation or task) and motor system requirements will switch in this case.

Although most previous studies on human motor control were based on the optimal control theory (Todorov and Jordan, 2002; Scott, 2004), which is considered to be physiologically related to the cerebellum (Shadmehr and Krakauer, 2008), this theory cannot necessarily apply to universal motor control. The theory focuses on optimizing the system based on the centralized invariant cost functions, such as the deviation of target trajectory (Uno et al., 1989) or motor cost as muscle activity (Anderson and Pandy, 2003), such as in arm movement. However, an unstable multi-link closed-loop system with large inertia and a narrow base of support in abruptly changing situations, such as switching adaptation in bipedal locomotion, is difficult to control optimally. This is because it cannot determine the optimal target trajectory due to the large control component with physiological constraints (e.g., joints and muscles), the observation component with cognitive constraints (e.g., ground and opponent) and the context (e.g., the predicted optimal strategy could be defeated by the opponent's counter-attack; Fujii et al., 2015b). Thus, switching adaptation in bipedal locomotion, which is difficult to control even in current robotics (Koolen et al., 2016; Kuindersma et al., 2016), is an excellent example to shed more light on the mystery of universal motor control.

Human bipedality, which is considered to be the result of adaptations to environmental variabilities (Carvalho et al., 2012), is one of the controversial problems to control. While the efficiency of bipedal locomotion in the optimal control theory (Srinivasan and Ruina, 2006) was explained by the dynamics only in the ground phase, neural oscillator control (Taga et al., 1991) that is physiologically located in the spinal central pattern generator (Grillner, 1985; Dimitrijevic et al., 1998) can reproduce the whole of aperiodic adaptive bipedal locomotion in a self-organized manner rather than explicitly calculating the target trajectory or joint torques. However, the oscillator system is considered to be limited in cyclic movement with adaptation only to external disturbances (Thelen et al., 1987; Taga et al., 1991). For example, active adaptation to a changing situation will result in excessive deviation from the aperiodic locomotion generated by the oscillator (e.g., in the opposite direction) because the motor command itself changes drastically. In recent years, using a physiological reflex model, the diversity of walking including a direction change of 50° was reproduced (Song and Geyer, 2015), but in situations such as escape or pursuit, robustly faster direction change at any time (Fujii et al., 2015a) is needed. Furthermore, it is unknown which factors make such adaptive bipedal locomotion difficult because previous locomotion models (Taga et al., 1991; Song and Geyer, 2015) including multiple neural oscillators, peripheral reflexes and multi-link body dynamics were implemented in a complicated manner, whereas as far as the passive walk, the previous model simply accomplished it (McGeer, 1990). Therefore, as a prerequisite for such adaptability, it is important to examine a minimal control model that achieves direction change at any time in the opposite direction with a small number of components and a simple algorithm, and to establish a methodology for evaluating it.

Distributed autonomous control, in which autonomous components implicitly function as a whole by interacting with each other without central control, such as in multi-agent (Couzin et al., 2002) or multi-link (Watanabe et al., 2012) biological systems, is applicable to real-time adaptation to the rapid impairment of components (Yoshihara et al., 2007). This control system is biologically plausible than explicit simulation because the system can perform self-modeling (Bongard et al., 2006) to adapt to the situation beyond its framework. The differences and advantages of the distributed autonomous control compared with the neural oscillator control are that the local components autonomously set the local target and have flexibility in the rule-based interaction among components. Among the autonomous system, self-repairing robots (Bongard et al., 2006; Cully et al., 2015) are remarkable, but the switching adaptation task in this study requires more improvisational adaptation (e.g., within 1 s). The mobility control (Yoshihara et al., 2007) based on the design of autonomous systems, in which an autonomous mobile component moves prior to an immobile component, can execute arm reaching movement when confronting a real-time deficit of the component with improvisational adaptation. We thus assumed that mobility control can be a key factor in the switching adaptability with a minimal algorithm due to the real-time adaptability without the explicit control of the components. However, in bipedal locomotion, in addition to the control of the center of mass in locomotion (equal to endpoint control in arm movement), balance and leg swing control are necessary and often conflict, so not only the operation of equivalent rules for each component but also the switching of rules according to the situation should be important.

In this paper, we adopted switching autonomous system, which extended (i.e., incomplete) distributed autonomous control scheme, because the current task can be accomplished by solving multiple conflicting functions. For example, it would be more advantageous for multiple actuators to switch roles to maintain balance by the leading leg and to move the center of mass by the trailing leg (Yamashita et al., 2013). In neurophysiology, this mechanism may be related to postural control in the reticulospinal tracts found in cats (Mori et al., 1998), but its mechanisms of interaction and switching the function of actuators (i.e., muscles) have remained unknown. We therefore implemented an adaptive bipedal model into role-switching for propulsion, balance, and leg swing control with switching mobility control. The objective of this study is to propose a new control algorithm and evaluation methodology of a switching adaptive model for performing bipedal locomotion as a prerequisite for universal motor control.

In this study, we constructed a minimal distributed autonomous model achieving bipedal change in direction at any time with only three actuators, but without accurate features representing the whole human body structure, such as a central pattern generator and a lower limb joint, as reproduced in the previous comparable models (Taga et al., 1991; Song and Geyer, 2015), that is, without explicit optimization and cyclic movement in neural oscillators. Simultaneously, we also developed a new method for evaluating the ability to perform robustly faster direction change during bipedal locomotion (Figure 2E). In previous studies of adaptive bipedal locomotion in robotic engineering or neurophysiology, such as when encountering an obstacle in the environment or an external disturbance (Taga et al., 1991; Song and Geyer, 2015; Koolen et al., 2016; Kuindersma et al., 2016), it was not considered whether the bipedal model can achieve direction change at any time, which is not explicitly implemented (i.e., unpredictable for the model). Thus, to discuss this control problem, it is necessary to reconstruct the frameworks from the viewpoints of engineering control and neurophysiology, as explained below. The expansion of the frameworks should be necessary for understanding universal motor control.

From the viewpoint of engineering control, the switching mobility model showed switching adaptability at the expense of efficiency because it is difficult for the bipedal locomotion model to satisfy the criteria of both efficiency and adaptability. Previous research using an inverted pendulum locomotion model (Srinivasan and Ruina, 2006) explained the energy efficiency in various locomotion patterns based on the optimal control theory. With regard to efficiency, the switching mobility model showed highly costly movement in which a period of repeated grounding of the same leg occurred (Figure 3), in contrast to that in the conventional models (Taga et al., 1991; Song and Geyer, 2015), even without a change in direction. Furthermore, at the time of changing direction, it sometimes takes a long time to adapt to the command to change direction (Figure 3B) because of a greater number of steps (Figure 4A) and a higher leading leg raise (Figure 4B). This leg raise was controlled in the neural oscillator model (Taga et al., 1991) and strictly computed in the reflex model (Song and Geyer, 2015) at a lower level than in the switching mobility model without direction change (Figure 5). In adaptive bipedal locomotion, it is difficult to calculate the optimal trajectory, so we confirmed the higher motor cost in the switching mobility model, estimated from the much larger variance in step interval and much higher leg raise, than in the conventional models. This is also because there is no mathematical guarantee of efficient movement in mobility control (Yoshihara et al., 2007), which is different from the explicit optimal control or neural oscillator control in cyclic movement. Instead, switching mobility control in the switching mobility model would have an advantage regarding adaptability, even if the movement switches from a cyclic to a non-cyclic pattern and vice versa and has difficulty in control, such as large instability as in bipedal chase-and-escape in interactive sports (Fujii et al., 2015b,c). We were not able to perform a direct comparison with these control algorithms because there are no models to show the switching adaptability (and performing a sidestep); however, a future model can be compared by the proposed evaluation method in which the model performs direction change at various timings.

Neurophysiologically, our switching mobility control algorithm suggests the presence of reflex-like switching functions of propulsion, balancing, and leg swing within and between limbs to achieve the task. The algorithm does not directly reflect the neural mechanism, but we can consider similarity with human neurophysiology by a process of elimination. The proposed model does not explicitly control the actuator movements like cerebellum (Shadmehr and Krakauer, 2008) and does not directly generate quasiperiodic movements such as using neural oscillators (Taga et al., 1991) physiologically located in spinal central pattern generators (Grillner, 1985; Dimitrijevic et al., 1998). Instead, the previous work suggests that diverse movements can be reproduced by incorporating the multiple reflexes in the spinal feedback circuitry without the central pattern generator (Song and Geyer, 2015). Among these architectures, we can find the similarity between the reflex control and the autonomous distributed control (Yoshihara et al., 2007; Watanabe et al., 2012), which incorporates interaction among components and environments (i.e., feedback of ground reaction force) and can implicitly adapt to a rapid change of command. In other words, the point at which each actuator autonomously sets and executes a target movement according to the situation in the switching mobility model matches that in the physiological reflex mechanism in which the inputs and outputs are locally automated. Our switching distributed autonomous control algorithm provided the relationship among the actuators (i.e., muscles) to switch functions of propulsion, balancing, and leg swing within and between limb levels. In a manner similar to long-latency reflexes, which would possess an internal model of limb dynamics (Kurtzer et al., 2008), the actuator in the switching mobility model rapidly switched functions according to the situation. There is no physiological evidence from the neural circuitry of a response of such inter-limb reflexes to a drastic change of command in the long neural pathway, but we believe that it would be needed for the switching adaptation in bipedal locomotion. The neural mechanism involved would be complicated because of possible involvement with both voluntary and reflex control overlapping in their neural substrates (Kurtzer et al., 2008); however, it may be simply explained by the simple interaction rule of our switching mobility control. This constructive approach can contribute to understanding intelligent motor control including biologically (Carvalho et al., 2012) and socially (Helbing et al., 2000) essential activities, such as escape from enemies, pursuit of prey, and search for food.

However, there are some problems with the above neurophysiological claims. One is that it claims to be based only on the similarity in the architectures without neurophysiological evidences. This is considered as a general problem in finding evidences of long-latency reflexes, which overlapped with voluntary movements in their neural substrates (Kurtzer et al., 2008). Second is validation with real-world human data satisfying both sidestepping (Yamashita et al., 2013) and unpredicted change direction (Fujii et al., 2015b). The latter study showed that the unpredicted competitive situation (i.e., requiring the faster movement) delayed the first step initiation more than 100 ms, suggesting the human can decrease the delay more than the current switching adaptability model. The real human locomotion mechanism includes central pattern generators, peripheral reflexes, and spring-damper system. The spring-damper property, which contributes high speed locomotion according to a quadruped robot study (Kimura et al., 2007), should be further investigated.

Third is the sensitivity of the simulation to the choice of some of the model parameters. Our supplementary results (Figure S3) showed that the switching adaptability model had strong sensitivity to the parameters. We suppose the parameterization may be related with inherent adaptation to the individual musculoskeletal system and might be relatively independent of the motor control adaptation. As a further alternative approach, for example, evolutionary algorithm (Song and Geyer, 2015) and reinforcement learning (Lillicrap et al., 2015) efficiently worked in previous bipedal locomotion studies. These are complementary relationship in terms of inter- and intra-generation progress, respectively. The reflex model (Song and Geyer, 2015) optimized the control parameters with the covariance matrix adaptation evolution strategy (Hansen, 2006). Reinforcement learning enables the acquisition of efficient and adaptive locomotion by trial and error on models, like humans actually do, probably in the basal ganglia (Doya, 2000). However, if learning a case of sudden change through evolutionary algorithm and reinforcement learning (also in learning of humans as an experimental condition; Fujii et al., 2013), there is a possibility that it will not mean “unpredictable sudden change” when performing an adaptive bipedal locomotion (Shinya et al., 2009). In other words, it can be considered that the explicit control rule should be difficult to estimate in principle in these approaches (i.e., human bipedal locomotion can also predictively adapt to the situation; Shinya et al., 2009). The evolutionary algorithm and reinforcement learning will also have the advantage to acquire the efficient movement in our less efficient model. Satisfying the requirements of efficiency (e.g., obtained explicitly by using optimal control and implicitly by evolutionary algorithm or reinforcement learning) and switching adaptability proposed by our study will further contribute to solving the general motor control problem.

KF conceived the original idea of the model. KF and YYo designed the model. KF, YYo, HT, and YYa analyzed data and wrote the paper.