For the rhythmic pattern generation, the CPG is realized by a neural SO(2) oscillator² (Pasemann et al., 2003). The SO(2) oscillator has two fully connected standard non-spiking neurons (N1 and N2, see Figure 2A), both of which have a sigmoid transfer function. The activation a_i and output o_i of each neuron are provided by Equations (1), (2):

where w_ij is the synaptic weight of the connection from the jth neuron to the ith neuron. n indicates a time step of discrete-time equations. One time step is related to 1/update frequency. All the weights and MI are defined using Equation (3).

where MI is the modulatory input of the CPG synaptic weights (Manoonpong et al., 2013). Using different MI values lead to different CPG frequencies and, as a result, different walking frequencies. Note that the default synaptic weights of 0.21 and 1.4 are selected from the parameter domains that stay beyond a Neimark-Sacker bifurcation where periodic or quasi-periodic attractors exist (as investigated in Pasemann et al., 2003). This allows the CPG to produce basic periodic signals at a very low frequency, even when the MI value is zero. In Equation (1), f_i(n) represents the adaptive sensory feedback term that is induced by the APC to adaptively modulate the CPG's phase.

The outputs of the CPG are transferred to the MNs (M1 and M2), which are linear neurons with scaling factors that shape the CPG output signals. The outputs of the MNs are transmitted to the hip and knee motors as the inputs of the position control for driving the motors. The MNs can be represented as follows:

Where o_i(n) is the output of the ith neuron in a CPG and a_i and b_i are the slope and offset of the linear transformation of the MNs, respectively. θ_i(n) is the output of the corresponding MN. The hip and knee joints extend (clockwise rotation) as θ_i(n) increases, while the joints flex (counterclockwise rotation) as θ_i(n) decreases.

The CPG phase is directly modulated by the SFM that can be described by the following equations:

Where o_i(n) is the output of the ith neuron in a CPG. γ(n) is an adaptive feedback gain automatically tuned by the DL. F(n) represents the continuous actual GRF as sensory feedback to the CPG. Note that the GRFs of four legs are normalized to [0, 1) by dividing the measured GRFs with around half of the robot body weight, where zero and nonzero denote a foot in the swing phase and stance phase, respectively. Zero and nonzero denote a foot in the swing phase and stance phase, respectively. The DL is an error-based learning mechanism (Smith et al., 2006) that implements the adaptation of the physical communication.

Specifically, the DL tunes the gain γ(n) (see Equation 5) based on the error between the actual and expected GRF signals. Its function is represented by the following equations:

Where F^a(n) and F^e(n) are the actual and expected GRF signals, respectively. B_s and B_f correspond to the slow and fast learning rates of the slow and fast learners, whose slow and fast retention factors are A_s and A_f, respectively. The fast learner has a higher learning rate and the slow learner is characterized by a higher retention rate, i.e., A_s > A_f and B_s < B_f. The learning parameters used in this study were empirically set to: A_s = 0.99, A_f = 0.57, B_s = 0.0002, and B_f = 0.002.

The expected GRF (F^e(n)) is mapped from an efference copy of the knee joint by the FM. This is because, in an ideal state, the flexion and extension of the knee joint indicate that the foot should move up into a swing phase and down into a stance phase, respectively. Thus, in the swing phase, the expected GRF is zero; however, in the stance phase, it yields a higher value (> 0). Here, the input to the FM is a sine wave-like pattern (see Figure 2B). The FM can be described as follows:

Where F^e(n) is the expected GRF, α is a scaling factor used to scale the amplitude of the expected GRF such that it matches the actual GRF, ρ is a shaping parameter used to fine-tune the duty factor of the expected GRF. G(n) is a variable used mainly for switching the expected stance and swing phases depending on θ₁(n), where θ₁(n) is the output of the MN to the hip joint. In the following experiments, the parameters were set to α = 0.9, ρ = 0.99 (see Equation 10). Note that although the selected FM cannot create a complex waveform, a radial basis function network can yield a complex waveform (Thor et al., 2020).

Legged robots and animals show adaptive walking patterns with respect to the changes of their walking speed (Hoyt and Taylor, 1981; Owaki et al., 2012; Owaki and Ishiguro, 2017; Fukui et al., 2019; Nirody, 2021). In the physical simulation, we experimented the APNC on Lilibot to evaluate its adaptive walking pattern generation (self-organized locomotion) under different MI values (i.e., different walking frequencies, see Equation 3). The CPG parameter MI determines the CPG output frequency and robot step frequency (S_f), thereby regulating the robot's walking speed. There is an approximate relationship between them: S_f = 8.6 × MI + 0.5. Thirteen different MI values ranging from 0.04 to 0.28 (comparable to S_f from 0.8 to 2.9 Hz) were tested. The experiment was repeated 20 times for each MI value. Each trial ran for 45 s. The adaptability was evaluated using the CPG phase convergence time and duty factor (see Figure 4).

The CPG phase convergence time indicates the time required by the relative phases (ϕ₁₂(n), ϕ₁₃(n), ϕ₁₄(n)) of the decoupled CPGs to converge to a suitable state (e.g., π, π, 0) from the initial state (0, 0, 0).³ The dynamic transitions of the relative phases can be seen in Supplementary Figure S3.

The CPG phase convergence time and success rate were explored at different walking frequencies for the three control methods (Figure 4). The APNC and PR-based control methods exhibited faster phase convergence than the Tegotae-based control method. The APNC-based control method employed an adaptive feedback gain to properly accelerate the effect of the continuous phase modulation; thus, it achieved faster phase convergence than the Tegotae-based control method, which employed a fixed predefined feedback gain. The PR-based control method used discrete GRF feedback to periodically reset the CPG phases, which also led to fast phase convergence (see Sun et al., 2021b for further comparative analysis of the continuous phase modulation and PR).

The duty factor is defined as the proportion of the stance phase to a gait cycle (swing and stance phases). It can quantify the walking patterns according to the foot-end movement states. The average duty factors of the four legs are interestingly proportional to the MI value (see Supplementary Figure S4). This is due to the APC mechanism. The frequency of the CPG oscillation increases as the MI value increases (Manoonpong et al., 2013). This also increases the walking or step frequency, and the higher the step frequency, the greater the GRF feedback. As a result, the actual GRF signals might be greater than the expected GRF signals; thereby, the adaptive gains are also increased to transmit the actual GRF signals for adapting/inhibiting the CPGs' activations. Thus, the stance phase can become longer than the swing phase (see the gait diagram⁴ in Supplementary Figure S5). When MI ≥ 0.24, the effect weakens, and the average duty factor tends to be stable at approximately 0.72.

The update frequency of the robot system is an important factor for self-organized locomotion generation. We conducted robot walking experiments using different update frequencies ranging from 5 Hz to 60 Hz to investigate the effect of update frequency on control performance (Figure 5). The robot motor system has a maximum update frequency of 60 Hz. For the experiments, we tested three control methods (APNC, Tegotae, and PR) and compared their phase convergence time and success rate. Each trial lasted 60 s and was repeated 20 times. The MI value defining the robot walking frequency was set to 0.08.

Figure 5 shows the success rate and phase convergence time of each control method for different update frequencies. In general, as the update frequency increased, the phase convergence time of the three control methods decreased. This is because the high update frequency had low delay of the sensory feedback to the CPG, thereby yielding the optimal CPG modulation. Only the APNC method successfully generated self-organized locomotion with 100% success rate for all the update frequencies. The Tegotae and PR methods achieved 100% success rate only when the frequency was higher than 25 Hz. Furthermore, owing to its adaptive sensory feedback gain (see Figure 2) that properly enhances the effect of GRF modulation in the CPG phase, the APNC method achieved phase convergence faster than the Tegotae method across all the frequencies. The stability analysis of the control system under different update frequencies is provided in Supplementary Figure S10.

Adaptability to various uneven terrains (characterized by terrain roughness) is a critical property of adaptive interlimb coordination mechanisms for legged robots. To explore the adaptability of the APNC to uneven terrains, we set up different terrain conditions with many randomly distributed hemispherical obstacles with different diameters embedded in the ground. This was done to emulate varying terrain roughness conditions. The roughness (R) was defined as the percentage of the obstacle height (h_o) to the robot step height (h_s) (see Figure 6A). We tested the robot on 10 different terrain roughness setups ranging from 0% to 100% in the simulation. The experiment on each roughness setup was performed 20 times, and each trial was run for 45 s. The robot step height was empirically set to 0.01265 m in the experiment. Note that a higher step height will lead to unstable locomotion which requires additional posture balance control (Kimura et al., 2007).

The experiment results are presented in Figure 6B. As can be observed, the CPG phase adaptation required more time to converge with increasing roughness across all the methods. Moreover, the deviation of the phase convergence time also increased. When R was larger than 80%, the Tegotae was unable to form a gait in some trials (i.e., the success rates were less than 100%). This was because the CPG phase relationships could not converge to the desired state (see Supplementary Figure S6). The experiment results indicated that the APNC not only adapted to all the extents of terrain roughness, including relatively extremely uneven terrains (R > 60%), but also exhibited faster and more stable phase convergence than the Tegotae and PR at the extremely uneven terrains.

To validate the performance of the proposed control method in complex conditions, we tested the robot under four conditions: normal condition (C1) as a baseline, noisy feedback [GRF with noise (C2)], leg damage (C3), and carrying a payload (C4) (see Supplementary Table S5 for more details). The robot's walking performance was evaluated based on three common metrics: i) balance, ii) coordination, and iii) cost of transport (COT). The metrics are defined in the Supplementary Section S6 and Sun et al. (2021a). The walking trial was repeated 20 times for each condition and control.

The experiment results are depicted in Figure 7. While all the control methods (APNC, Tegotae, and PR) performed nearly equally well in the normal condition (C1), the APNC improved the robot's balance, coordination, and energy efficiency significantly, compared to the others in complex conditions (C2-C4). This is because, unlike those of the Tegotae and PR methods,⁵ the CPGs of the APNC-based control had adaptive neural connections that enabled the robot to effectively and robustly coordinate its leg movement even in complex conditions (C2-C4). In addition, a comparison of the three control methods using the real robot with a damaged leg is shown in Figure 8. As can be observed, the robot driven by the APNC-based control walked with the greatest speed.

In the first state (S1), we suspended Lilibot midair and initialized all the CPGs (MI = 0.15) with the same parameters and phases. After 1.7 s, Lilibot was placed on the ground (see Figure 10). Consequently, the GRFs were activated to modulate the CPGs. The feedback gain (γ(n) in Equation 5) was also automatically adjusted via the DL (Equation 9) to obtain the proper feedback strength to effectively modulate the CPGs. The sensory feedback to each decoupled CPG was implemented by physical body dynamics (Figure 2). This indicates that the APC occurred among the CPGs.

After 3.8 s, the GRF signals exhibited a periodic pattern (see Figure 10B). Furthermore, the clear swing and stance phases, where the GRF exhibited no activation (swing) and high activation (stance), could be observed. This resulted in a stable, self-organized trot gait (Figure 10D). The relative phases acquired by the APR (Figure 3) converged at approximately 9 s (Figure 10C). The relative phases slowly converged (after approximately 5 s) because of the effect of a low-pass filter in the APR of each CPG control circuit. The relative phases indicated that the outputs of the CPG were in phase for the RF and LF legs and in anti-phase for the other legs. This experiment result demonstrated that the APNC enabled swift generation of self-organized locomotion (within 5 s) and effectively established the ANC on an even ground within 9 s. The snapshots of the experiment are shown in Figure 10E and a video clip of the experiment can be viewed at http://www.manoonpong.com/AdaptiveCommunications/video4.mp4.

After the self-organized locomotion emerged, the phase relationships (ϕ_lk(n) in Equation 12) converged; thus, the physical communication gain (γ(n) in Equation 9) of the APC became small and the GRF feedback modulation (f_i(n) in Equation 5) decreased, while the ANC was activated to induce neural couplings (g_i(n) in Equation 17) among the CPGs, thereby storing the phase relationships (Supplementary Figure S7).

In the second state (S2), after Lilibot performed self-organized locomotion, the ANC was activated to compensate for sensor malfunction (Figure 9B). Thus, in this state, there were two communication mechanisms (APC and ANC) working in parallel. To simulate possible sensor malfunctions arising from an unexpected impact, the sensory feedback of the front legs was set to a high constant value (i.e., 0.9). To comparatively examine the effect of the combination of both communications, the state was also tested without the ANC. The experiment results of the adaptive neural control without and with the ANC are shown in Figures 11A, B, respectively.

In Figure 11A, the outputs of the CPGs of the front legs became constant. This indicated that the two CPGs stopped oscillating, owing to the abnormal sensory feedback of the front legs, which was very high, at 0.9, inhibiting the CPGs through the physical communication. As shown in the gait diagram in Figure 11A, the front legs remained in the stance phase all the time because they could not periodically move. Conversely, in Figure 11B, all the CPGs continued to oscillate, even after the front legs encountered the same abnormal situation as in Figure 11A, because the ANC among the CPGs coupled them, enabling them to synchronize their activations. This compensated for the inhibitory effect caused by the sensor malfunction of the front legs. Consequently, the robot successfully maintained a trot gait. The robot's GRFs and displacement can be seen in Supplementary Figure S9. The experiment results of this state revealed that the APNC-based adaptive neural control successfully improved the robustness of the self-organized locomotion by compensating for sensor malfunction or damage. The snapshots of this experiment are shown in Figures 11C, D, and a video of the experiment can be viewed at http://www.manoonpong.com/AdaptiveCommunications/video5.mp4.

In the last state (S3), we evaluated the effectiveness of the ANC for reusable locomotion or locomotion recovery. Therefore, we switched off the GRF feedback to the CPGs (i.e., sensory absence) such that there was no physical communication (see Figure 9C). Thus, only the ANC remained active. Furthermore, all the CPGs were initialized to prevent oscillation by setting MI (see Equation 3) to zero, thereby removing the formed patterns of the CPGs. S3 was performed under the adaptive neural control without and with the ANC. The results are shown in Figures 12A, B.

As shown in Figure 12, from 20 s to 24 s, the two cases yielded the same results during initialization. The CPGs stopped oscillating and the robot was motionless. At 24 s, the CPGs were reactivated to oscillate (we set MI of the CPGs back to 0.15). Subsequently, the behaviors of the robot were obviously different in both cases. In Figure 12A, although the CPGs generated commands to move the robot joints, the four legs moved in phase. The legs could not be lifted off the ground during the swing phase (see the gait diagram). Thus, the robot was rooted to the ground and unable to move forward. This was because without the ANC, the previously formed gait could not be recovered. Conversely, as shown in Figure 12B, the relative phases returned to the same values as observed in S1 (Figure 10). Accordingly, the trot gait was immediately recovered. This was attributed to the ANC, because the phases among the CPGs were stored in the couplings (i.e., motor memory; see Equation 17). The robot's GRFs and displacement can be seen in Supplementary Figure S8. The snapshots of this experiment are shown in Figures 12C, D, and a video clip of the experiment can be viewed at http://www.manoonpong.com/AdaptiveCommunications/video6.mp4.

The legs received the same motion commands if their CPGs were initialized using the same parameters. Thus, in an ideal situation, they would perform the same movements in response to the same commands. Accordingly, in the ideal condition, the sensors on the legs would acquire the same sensory information (e.g., GRFs) and pass it on to the CPGs. This resulted in the same modulation for all the CPG activations, such that the legs would move in the same phase forever. Therefore, in this ideal case, the physical communication makes it impossible for the robot to form a gait. However, the actual GRFs of all legs were not completely identical, even when the legs moved in the same phase. This is because of the existence of some non-ideal factors such as sensory noise, joint movement trace error, measurement error, and imperfect structural symmetry. These factors cause slight differences among GRFs, thus creating initial phase shifts among the CPGs. Subsequently, the slight phase shifts increase rapidly and converge to values that represent a particular gait when using sensory feedback modulation (see Equation 5).

The adaptive feedback gain mechanism employed in this study augmented the differences among the GRF feedback to accelerate gait formation. This is because of the ability of the DL to derive the optimal feedback gain for each leg, thus enabling the GRF to appropriately influence the CPG. In Owaki et al. (2012), Owaki et al. proposed a novel approach using four decoupled phase oscillators serving as CPGs with local-only GRF feedback to generate self-organized locomotion in a quadruped robot. They discussed the phase dynamics of the sensory-modulated CPG and indicated that the CPG had two states, namely, the oscillatory and excitatory states. The autonomous switch from one state to the other plays an essential role in the generation of adaptive interlimb coordination. However, the sensory feedback gain was predefined in their work, although it has a significant effect on the convergence time of self-organized locomotion. Therefore, we induced an adaptive sensory feedback gain realized by the DL (see Figure 2A). The DL adjusted the gain depending on the error between the actual and expected GRFs such that the actual leg movement matched the expected one with greater precision. The DL with fast and slow learners quickly adapted the feedback gain online toward achieving rapidly self-organized locomotion generation (i.e., within 5 s see Figure 10). The error between the actual and expected GRFs utilized by the DL was limited to positive values (Equation 6). Consequently, the feedback gain was non-negative (γ ≥ 0) and had a positive value (γ > 0) only when the corresponding foot remained in the actual stance phase; thus, the DL was active when the robot interacted with the ground. This feature prevented the legs' swing phase from affecting the retention of the DL such that the adaptive sensory feedback modulation provided a rapid reaction to the stance phase movement.

The automatic sensorimotor coordination was significantly influenced by the functionality of the sensory feedback, which determined the walking patterns. Over the years, various forms of sensory information have been exploited to modulate CPGs for adaptive gait generation and transition. For instance, Fukuhara et al. investigated a combination of the support and propulsion force as sensory feedback to adjust the CPG phase, through which they demonstrated not only self-organized locomotion generation, but also gait transitions on a quadruped robot (Fukuhara et al., 2018). Fukui et al. (2019) modulated the CPG phase using vestibular feedback, which enabled a quadruped robot to achieve autonomous gait transition and galloping, even when the CPGs had predefined and fixed weak connections. In addition to the use of multiple sensory feedback, Aoi et al. (2010) studied the effect of the stiffness of the backbone joint on gait transitions under CPG-based control. They achieved gait transitions by changing the waist joint stiffness. Similar biological examples have been observed in mammals (e.g., dogs Schilling and Carrier, 2010, cheetahs Hildebrand, 1959 and horses Hildebrand, 1959) that exhibit distinctive spine movements when they use different gaits.

As a matter of fact, the backbone joint significantly affects the functionality of the physical communication because its stiffness influences the dynamic interactions among the legs. In summary, these results support the argument that the functionality of sensory feedback to the CPGs not only plays a key role in interlimb coordination generation, but also affects the gait selection. In this study, we used Lilibot, a robot with a rigid trunk that provided a relatively fixed transmission over GRFs (i.e., a support force) regardless of changes in the walking speed and direction changes. Thus, in future work, we plan to implement multiple sensory feedback modulation and incorporate a backbone joint with adaptive stiffness to extend the flexibility of the physical communication channel and thereby enable the robot to perform adaptive gait transitions.

Owing to its high dependence on sensory feedback, there are two drawbacks associated with using the APC for interlimb coordination generation. First, once a sensor malfunction occurs, the robot cannot sustain its stable self-organized locomotion. Second, the robot cannot memorize the formed gait for later recovery. However, biological findings have demonstrated that animals have the capability to maintain locomotion even when they encounter unexpected situations (Graham, 1977) such as sensory damage or leg amputation. A general neurophysiological fact is that the neural couplings between neural circuits form a basis for memory. An evidence from Giovanni et al. demonstrated that neural circuits and their couplings support imitation learning of hand actions (Buccino et al., 2004). Many biologically inspired control approaches employed for locomotion generation utilize neural couplings or connections to predefine movement behaviors for legged robots (Cruse et al., 1998; Kimura et al., 2007; Ajallooeian et al., 2013; Fukuoka and Kimura, 2014; Liu et al., 2018).

The generated self-organized locomotion was robust against sensory feedback damage. In our experiments, we studied a sensor malfunction case in which the GRFs to the front legs took on a high constant value to simulate an unexpected collision triggering sensor failure in the feet (see S2 in Figure 9). The abnormal GRF signals could not be used to distinguish the movement state (stance or swing phase) and strongly inhibited the CPGs continuously, as a result of which the CPGs stopped oscillating (Figure 11). However, the control with the additional ANC was able to synchronize and enforce the oscillation of the CPGs, even if some of them are inhibited heavily by abnormal sensory feedback. This is because the activations of the CPGs were controlled by the CPGs themselves through the ANC (see Equation 17). Therefore, the control using a combination of the APC and ANC enabled a legged robot to exhibit not only rapidly self-organized locomotion generation but also robust locomotion against sensor malfunction or absence. This result indicates that the ANC mechanism plays an important role in supplementing physical communication to generate stable movement in a legged robot. Furthermore, in contrast to some existing CPG coupling approaches using phase oscillators (Collins and Richmond, 1994; Aoi et al., 2010), we achieved neural communication based on the abstract version of biological neurons (i.e., the neuron model in the SO(2) CPG). This biological plausibility tends to support the claim that adaptive neural couplings (e.g., synaptic adaptation) play a crucial role regarding interlimb coordination after a limb injury (e.g., leg loss Cully et al., 2015 or amputation Dasgupta et al., 2015).

Another important effect attributed to the ANC is the reusability of the formed locomotion even when the CPGs are reinitialized. The robot exhibiting this behavior demonstrated that the proposed adaptive neural control can memorize the stable movement formed in a self-organized manner. This feature was derived from the inherent properties of the adaptive neural couplings, which stored the stable phase relationships among the CPGs. Motor learning has increasingly attracted researchers aiming to develop adaptive movement in robotics (Reinkensmeyer et al., 2004). This study illustrates a means to convert the adaptive movement formed online (using physical communication) into an adaptive neural circuit network (see Figure 1B). It further provides insight into the underlying mechanism of motor memory regarding how to encode and store information in the neural system for movement recovery in robots.