From a neurocybernetics perspective the dynamical systems approach to embodied cognition can be traced back to the work of Ashby (Ashby, 1960) and von Foerster (Von Foerster, 1960). The assumption is that a living organism, in order to survive, must be able to develop internally some stable “entities” (von Foerster) which refer to or classify objects and situations in the physical world. These “entities” are the result of cognitive and sensorimotor processes developing through continuous interactions of an individual with its specific environment. On the other hand, cognitive and sensorimotor processes, relevant for the behavior of the individual, depend on the formation of these stable structures; i.e., they are complementary in the sense that one defines or implies the other. The assumption was, that an organism must be able to relate discrete internal structures to relevant aspects of its own interaction with its environment.

Although, the underlying processes are continuous these internal “entities” have to be discrete because the referenced objects or situations are discrete features of the environment. They also have to be “stable” in a certain time domain. On the other hand, due to changing sensorimotor or cognitive processes, they have to get “unstable” in the sense that different references have to be built up; i.e., new “stability domains” have to be visited or formed.

To pursue the dynamical systems approach to embodied cognition in this spirit, this paper will consider an individual as an autonomous system called an animat. An animat Dean (1998) and Guillot and Meyer (2001) is a simulated or physical robot equipped with sensors and actuators, and a neural network for behavior control. The neural controllers then have to operate in the so called sensorimotor loop, getting inputs from sensory signals and generating motor signals, which in turn will lead to new sensor inputs. The essential role of these closed loop processes for living or live-like systems has been discussed over several decades now from various points of view (Bishop, 1960; Beer, 1995; Di Paolo, 2003; Philipona et al., 2004; Hülse et al., 2007; Zahedi et al., 2010; Sándor et al., 2015). Here we use a purely formal approach and carefully analyze the dynamical description by making successive approximations to these processes.

Neurocontrollers, mimicking their biological counterparts, are considered as recurrent neural networks which in general allow for dynamical properties. That is, for fixed synaptic weights, bias terms and inputs such a network can be described as a dynamical system. Then, assuming that a neurocontroller is driven by slow sensor inputs, it will be properly described as a parametrized family of dynamical systems, where sensor inputs (and proprioceptive signals as well) are considered in a first approximation as parameters of such a family. Furthermore, for every parameter value the corresponding dynamical system may have a manifold of different attractors. The postulated internal “entities” then will be identified with the basins of attraction of parametrically stable neurodynamical systems. The interaction with the environment then may change the references to situations in the external world by changing parameter values given, for instance, by the sensor signals. This process of changing references will be described by so-called bifurcations.

For theoretical reasons, parameters are assumed to change so slowly that the system can approach its asymptotic states. This is often not the case for realistic sensor inputs. So, in a second step we will introduce sequences of neural states called meta-transients as for instance in Negrello and Pasemann (2008) Negrello (2011), and Toutounji and Pipa (2014).

In general these meta-transients can not be given an interpretation as trajectories of a dynamical system, mainly because the inducing sequence of sensor signals is not a trajectory of a dynamical system on sensor space. Instead, because of the closed loop, it is superposition of movements in the environment and the result of motor actions. The case where one has access to controlling parameters has often been discussed in geometric control theory (Gardner, 1983; Sussmann, 1983; Respondek, 1996; Kloeden et al., 2013). There then one can generalize the concept of attractors and the like. Although we do not find this approach applicable for the dynamics in the sensorimotor loop we will work with a comparable view.

Finally, these meta-transients have to be mapped to motor neurons, inducing then actions of the animats body; i.e., its behavior. Due to this projection not all elements of the neural system will be involved directly in the generation of motor signals. This leads naturally to a fiber structure over the motor space allowing to introduce the concepts of vertical or internal neurodynamics, having no direct effect on behavior, and a horizontal or effective neurodynamics, the projection of which generates the movements of the animat.

To clarify concepts, the paper will address the discrete-time neurodynamics of networks composed of standard sigmoidal neurons of additive type. Using this simplifying setup, it is assumed that the aspects described in the following are transferable also to neural systems employing more biologically plausible or other types of neurons. The basic concern here is to specify the role of, for example, attractors, basins of attraction, transients, bifurcations and stability properties in the context of systems acting in a sensorimotor loop.

Approaching the description of neurodynamics in the sensorimotor loop in three steps, we will first define the type of neurodynamics studied in this paper (Section 2), exemplifying it by some well known results. Assuming that sensor inputs are slow when compared to the activity dynamics of the neural system, we argue in Section 3 that neural systems in the sensorimotor loop are effectively described by parametrized families of dynamical systems, were parameters correspond to the sensor inputs. Other parameters, not considered here, are, for instance, signals coming from proprioceptors and the synaptic weights of the network, the change of which usually is associated with learning. Referring to the more realistic situations, meta-transients are introduction in Section 4. Finally, Section 5 discusses the generation of motor signals resulting from a projection of attractor transients or meta-transients, respectively, to the motor space; this then allows to differentiate between so called effective and internal neurodynamics. Finally the sensorimotor loop is closed through the environment by a formal mapping from motor space M to sensor space S. The paper concludes with a discussion of the possible role the introduced concepts can play for understanding neural representations of behavior relevant situations in the external world and, correspondingly, for a notion of memory which is not based on specific attractors like, for instance, fixed point attractors in Hopfield networks.

Besides the body of an animat, three different parts of it will be discerned: The “brain” considered as a recurrent neural network N with n neurons. Its sensor neurons will prescribe the sensor space S, and the output neurons will define the motor space M. Sensor space S and motor space M are the interfaces of the “brain” N to the physical world. Assuming strictly the point of view of an animat, the world for an animat is what happens on its sensor surface. We describe these parts more concrete as follows.

A state a(t) ∈ A ⊂ ℝⁿ of the neural system N at time t is characterized by the activation of all its n neurons. Correspondingly, the state space A is called the activation space or phase space of N. It is a manifold of dimension dim (A) = n. Neural states may be represented in an equivalent way by the outputs o(t) of the n neurons, and we call the corresponding state space of the network N its output space A^*.

The sensor space S consists of all possible sensor inputs, i.e., a sensor state s(t) ∈ S at time t consists of all sensor values at time t. The sensor space is assumed to be a bounded manifold S of dimension dim S = m, where m denotes the number of distinct sensor elements. S may be subdivided into modality spaces corresponding, for example, to visual, acoustic, or haptic inputs.

A motor state m(t) ∈ M at time t is given by the activation of the motor neurons at time t driving the various actuators of the animat. Thus, the motor space of the animat is an open bounded manifold M of dimension dim M = k, the number of all its motor neurons. The motor space M may be segmented into different domains responsible e.g., for head movement, eye movement, driving wheels, arms, and so forth. Special domains may be related to corresponding domains in sensor space: Fixed infrared sensors may be, for instance, related only to the wheels domain; but with pan-tilt-camera vision is related to wheels, and pan-tilt-motors, et cetera.

In the last section the bias terms θ_i and synaptic weights w_ij, i, j = 1, …, n, were held constant, and one can consider them as parameters of the neural system (A, f). For different bias terms or synaptic weight one gets different dynamical systems. Thus, we introduce a parameter space Q ⊂ ℝ^q for a neural system (A, f) as a q-dimensional Euclidean manifold (Q, h) with metric h. A parameter vector ρ = (θ, w) ∈ Q is given by the bias vector θ and the weight matrix w of the network N(A). Thus, one has dim Q = q = n · (n + 1).

As a next step we argue that the sensor inputs to the neural system N(A) can be assumed to act as parameters of the neurodynamics. Because brain-like systems will always act in a sensorimotor loop, the sensor signals s(t) ∈ S will always drive the neurodynamical system N(A). Assuming in a first approximation that the sensor signals s(t) change so slowly that the orbits of the neural system are always able to converge to an attractor, then they can be considered as varying parameters. For that reason we will subsume the sensor signals s(t) as part of the bias terms θ(t):=θ^+s(t), with θ^=constant.

A neural system then has to be described as a parametrized family of discrete-time dynamical systems denoted by (A, f; Q), with A ⊂ ℝⁿ the activation space, Q ⊂ ℝ^q the parameter space, and a differentiable map f : Q × A → A. For a specific parameter vector ρ ∈ Q, we write f_ρ : A → A for the corresponding dynamical system, and denote the q-parameter family of neurodynamical systems also by (A, f_ρ), ρ ∈ Q. The only varying parameters considered in the following are the bias terms θ_i, i = 1, …, n. As stated above, other parameters of the animats brain, like synaptic weights w_ij are constant.

We may now look at the “brain” as a fiber structure over parameter space Q (compare Figure 6): To every ρ ∈ Q there is attached the activation space A together with the flow ψ_ρ corresponding to ρ ∈ Q; i.e., there is a whole attractor-landscape, denoted by [A]_ρ, attached to every parameter ρ ∈ Q.

In the next step we will have to ease the restrictions on the parameters by assuming that the sensor signals can change so fast that the activations a(t) of the neurodynamical system (A, f_ρ) can not approach an attractor Γ ⊂ A asymptotically.

In the following the considered parameters will be the sensor inputs s(t) of an animat, and all other parameters are fixed. Due to properties of the environment, or due to the behavior of the animat, its sensor inputs may change so fast that they can not be considered as parameters in the strict mathematical sense.

Such a situation is often described in terms of the dynamics of non-autonomous systems. But it is different from the situations covered by control theory (Gardner, 1983; Sussmann, 1983; Respondek, 1996) or by skew-product systems (Kloeden et al., 2013) in so far as a sequence of such sensor inputs is neither the trajectory of a dynamical system in parameter space, nor is it a well defined sequence leading to a preexisting goal. Here the sensor inputs depend on the dynamics of the physical environment (exo-motion) as well as on the movements/actions of the animat itself (ego-motion). We will come to that later again.

Assuming that parameters change almost as fast as the internal states, the resulting sequence of states is no longer that of a transient to one and the same attractor. Suppose the neural system at time t is in a state a(t) on a definite transient O(Γ_ρ(t)) to an attractor Γ_ρ(t) of the neural system (A, f_ρ(t)). If the parameter vector a short time later satisfies ρ(t + k) ≠ ρ(t) the corresponding state a(t + k) will be an element of a different transient O(Γ_ρ(t+k)) to a different attractor Γ_ρ(t+k) ⊂ A.

So, let σ_θ: = {s(t), s(t + 1), s(t + 2), …} denote such a sequence of sensor inputs represented by a sequence of parameter vectors θ(t) in Q. This will induce a sequence of states α(σ_θ): = {a(t), a(t + 1), a(t + 2), …} on A with

Such a sequence α(σ_θ) in A will be called a meta-transient (Negrello and Pasemann, 2008). Thus, a meta-transient is not a transient of a dynamical system, but it is a sequence of states a(t) ∈ A following the morphing attractors of a sequence of the parametrized dynamical systems (A, f_ρ(t)). The projection of such a meta-transient on A back to the parameter space Q then gives the sequence of “driving” parameter values σ_θ.

If we define a map Φ:Q × A → A associated with the given parametrized family of dynamical systems by

then the elements of a meta-transient α(σ_θ) are generated by this map according to

For example, if the input to a neuron with excitatory self-connection is slow when compared with the internal dynamics one will observe a clear hysteresis signal as in Figure 10. If the input signal changes much faster, then there will not be “jumps” at the boundaries of the hysteresis domains but a kind of “squashed” hysteresis loop will appear, as was observed for instance in Manoonpong et al. (2010) for the dynamics resulting from audio input signals.

Furthermore, if all the parameter values, corresponding to the sequence σ_θ of sensor inputs, lie in one and the same domain of parametric stability P, the behavior of the animat's body will not change dramatically, and one may describe it as “the same.” But if a sequence of parameter values crosses a bifurcation set K in parameter space Q the system may behave in a very different way.

All the dynamics discussed so far has the goal to generate appropriate body movements. Therefore, the only interesting thing here is the effect of the activities of the neural system which activate the motor neurons. Thus, we have to project the meta-transients α(σ_θ) on phase space A to the motor space M with dim (M) = k < n. This projection, denoted by Π : A → M, is assumed here to correspond to the application of a one-layer feedforward network (compare Figure 11). The activations of the k motor neurons then are spanning the output layer, and we define

where w_ji, i = 1, …, n, j = 1, …, k denote the weights from the n internal neurons to the k motor neurons. The activation of the jth motor neuron m_j ∈ M having a bias value θjM is then given by

Such a motor neuron in general will not be connected to all of the brains neurons. Therefore, there will be many internal states a ∈ A which will project to identical motor activations m ∈ M. This will give the second fiber structure of the sensorimotor loop, where the fiber F_m ⊂ A over m ∈ M is given by

Then, what is observable is the behavior of the animat generated by a sequence of motor states

which corresponds to a given meta-transient α(σ_θ) on A; that is, with ρ(t) ∈ Q, a(t) ∈ A, and bias terms of motor neurons θ^M ∈ ℝ^k one has

From the projection argument it is clear that not the whole state space A is of direct relevance for the behavior of the animat. It is obvious that the activity of neurons not connected to the motor neurons do not have a direct effect on the behavior of the animat. Therefore an attractor in A, if it is a fixed point, a periodic orbit or even a chaotic attractor, may be projected to only one and the same motor state m ∈ M; attractors, their transients or meta-transients may then have little or no effect on motor activities at all.

To reflect this property we introduce a splitting of every state a ∈ A into a so called horizontal and a vertical part; i.e.,

And due to this splitting we have a direct decomposition of the space of brain states A into horizontal and vertical parts; i.e.,

where A^v is given as A^v = ker Π.

Let there be l ≥ k internal neurons being directly connected with neurons in the motor layer; they serve as an l-dimensional input space B ⊂ A^h of the feedforward network (compare Figure 11). Furthermore, due to the geometry of feedforward networks (Pasemann, 1993b), in general there is a (l − k)-dimensional linear subspace C_m ⊂ B on which the activation of the motor neurons is constant.

The dynamics directly relevant for behavior then will actually live in the horizontal state space A^h ⊂ A. Correspondingly, what will lead to an effective behavior is a sequence of horizontal states given by a horizontal meta-transient on A

Going back to section 3 let us consider again a discrete-time dynamical systems f_ρ :A → A with fixed parameter vector ρ ∈ Q. Then, post hock, we can introduce a well-defined splitting of the dynamical system f_ρ into vertical and horizontal parts by

It is obvious that only the horizontal dynamics fρh:Ah→Ah contributes to the observable behavior of an animat, whereas the vertical dynamics fρv:Av→Av will describe brain processes which may be associated to a dynamical kind of memory, to association, planning, dreaming, contemplation, and the like; that is, to the cognitive faculties of the brain.

Furthermore, suppose that two dynamical systems f_ρ and fρ′ with ρ,ρ′∈P, P⊂Q a domain of parametric stability (compare section 3.1), are topologically conjugate. Then it is reasonable that their horizontal components fρh and fρ′h will generate motor states in M which lead to variants of a specific behavior. The next example gives a demonstration of this situation.

Example 3: In evolutionary robotics one often used the motor dynamics of a system as a fitness criterion to reduce the “ineffective” higher dimensional neurodynamics of evolved controllers to analyzable, minimalistic solutions for which the discussed effects could be studied (Wischmann and Pasemann, 2006; von Twickel et al., 2011; Pasemann et al., 2012). Here only a simple example of a neurocontroller may be given by the following recurrent neural network (Figure 11). It provides an obstacle avoiding behavior of a Khepera-like Robot (Toutounji and Pasemann, 2016). It uses five distance sensors (sensor layer) and two motor neurons (motor layer).

The hidden layer (the “brain”) has eight neurons, but only two of them project to the two-dimensional motor space M. Though the brain dynamics runs in an 8-dimensional state space A only a 2-dimensional subspace B ⊂ A determines the motor activity directly. What is going on in the 6-dimensional vertical state space has no immediate effect on the behavior of the robot. Indirectly, of course, the dynamics on A can influence the behavior of the robot; for instance, the over-critical excitatory self-connections of the input neurons I (Figure 11) control the turning angle of the robot at walls. The submodule in N(A) composed of the two neurons A and B has an interesting dynamics not influencing the motor behavior. They display a “chaotic” meta-transient while the the robot is turning, ending up in a period-2 attractor after a complete right turn, and in a period-4 attractor after a complete left turn. This internal (vertical) dynamics does not contribute to the behavior of the robot, but can be used as a kind of memory for subsequent decisions. The over all performance of this controller is comparable to that of the 2-neuron network called the MRC (minimal recurrent controller) in Hülse et al. (2004).

The description of biological brains as dynamical systems is often assumed to be an appropriate approach to describe cognition and the behavior of animals (Port and Van Gelder, 1995; Thelen and Smith, 1996). Based on the observation that the typical activity of an animat is a reaction to its environment, we used the sensorimotor loop to carefully approach the dynamics hypothesis in three steps. Relying on experiences in the field of evolutionary robotics (Nolfi and Floreano, 2000) we used discrete-time neurodynamics to, first, describe the (isolated) brains as dynamical systems. Having realized that (living) brains are always driven by sensor inputs, we made clear that the description of brains as parametrized families of dynamical systems is more appropriate. This allowed to introduce the concept of parametric stability which helped to formalize the general observation that a certain behavior is robust against “noise,” and can be classified as “the same,” although the initializing sensor inputs vary over a larger domain.

In a third step, assuming that sensor inputs may change so fast that they can not be assumed to serve as parameters in the mathematical sense (compare for instance Manoonpong et al., 2005), we were compelled to introduce the concept of meta-transients to describe the brains activity in a sensorimotor loop. These meta-transients in general will not be describable as orbits of a dynamical system. Finally, we used the fact that not all of the brains activity is directly reflected in the motor performance to discern between the brains effective (horizontal) and internal (vertical) activations.

In a more general sense the horizontal part is associated more with the sensorimotor pathways, whereas the vertical part is assigned to the higher centers of the brain, associated with cognitive faculties of a system. Of course horizontal and vertical processes are not decoupled and depend on each other; they are processes on one and the same highly recurrent network. As usual, higher centers are assumed to check the adequacy of the activities along the sensorimotor pathways; they are modulating the sensorimotor flow of signals. On the other hand, the vertical processes are permanently restricted by the “horizontal” flow of signals; otherwise, that is, without sensor inputs, they will run freely into perhaps noxious states of brain and body.

Following a purely formal approach to neurodynamics, we introduced in Section 3.2 the concept of parametric stability and the associated concept of a dynamical form. We think that these concepts may help to discuss questions concerning the representation of objects or, in this context better, behavior relevant situations in the external world.

From the dynamical point of view certain patterns of sensor inputs will be associated with the existence of certain attractors in activation space A; or otherwise stated, with the existence of a certain attractor-landscape. Because one has to assume that the brains dynamics is always driven by sensor inputs (including proprioception) it is more plausible to refer to a basin of attraction as a candidate for representing an external situation. Taking our argument for meta-transients serious it becomes obvious, that a dynamical form, associated with a certain type of behavior, is a reasonable representative for behavior relevant situations in the external world. Thus, taking parametric stability as essential for the reproducible identification of “the same” situations gives a reasonable conceptual basis for treating brain dynamics induced by an ever changing complex environment.

If one approves this interpretation then it will also allow for a less restrictive dynamical view on memory. Neural memories usually are represented by asymptotically stable fixed points, like in Hopfield's associative-memory model, or are conceived as periodic, quasiperiodic, or even chaotic attractors of neural networks. In fact, the correspondence between attractors and memories is one of the fundamental aspects of neural networks. But, as we have seen, situated in a sensorimotor loop and driven by sensor inputs, the best we can expect is that attractors of a neural network serve as kinds of symbols, while the system always runs on transients to these attractors (or on meta-transients). So in a first step memory should be associated with the basins of certain attractors. Taken that the natural situation is such that neural systems in the sensorimotor loop run on meta-transients, we have to assume that the union of all basins of attraction, belonging to the possibly morphing attractors of a dynamic form, should be identified with the memory of certain behavior relevant external situations. We will call this kind of memory model a blurred memory. The relation between learning and blurred memory will be the subject of further research.

The author confirms being the sole contributor of this work and approved it for publication.