Let us consider in all the following an agent endowed with motor and sensing capabilities. In the referenced previous contributions, the sensorimotor interaction is defined by the internal motor configuration m and the sensory configuration s of this agent, which lie, respectively into some sets M and S. Both of them define the internal agent configuration (m, s), i.e., the sensorimotor flow the agent has access to. There is a clear dependency between the sensory and motor configurations that can be captured by the sensorimotor maps ψ:M×E→S, such that ψ(m, ϵ) = s, where ϵ ∈ E represents the state of the environment. As said in the introduction, other contributions already exploited this kind of parameterization (Philipona et al., 2003; Laflaquière et al., 2015; Marcel et al., 2017). In all these contributions, only fixed base agents are considered, since a single internal motor configuration m∈M is only mapped to a single sensory configuration s∈S for a fixed environment configuration ϵ.

To illustrate this point, Figure 1 represents a 2D-agent able to translate itself only along one dimension x. This agent is able to move inside an environment made of colored walls thanks to five rotating joints whose states m_i, i = 1, …, 5, are captured in its motor configuration m = (_mi)i (where the second i subscript in (_.i)i denotes the collection being taken with i for ranging variable). To begin, let us consider the case where m₁ and m₂ are fixed, so that the agent is only able to move its arm supporting a camera-like sensor generating a sensation s, i.e., m is restricted to (m₃, m₄, m₅) only. In such a scenario, one has a fixed base agent for which each motor configuration m can be mapped to one corresponding sensor pose, which is itself mapped to a sensation s. This simple statement allows to build structures in M by exploiting only the sensorimotor flow (m, s), structures that can be leveraged to build an internal representation of the agent body; they can be further refined into a representation of its peripersonal space (Marcel et al., 2017; Marcel et al., 2019). In these works, m carries all spatial data, possibly with some redundancy, about the coupling between the agent and its environment: the combination of states m and e is sufficient to determine the resulting sensory output s (as described by the formal sensorimotor map ψ).

However, what would happen if the same agent was able to perform translations in its environment? Let us now focus on the case where all motor states m_i are actually used, as depicted in Figure 1. Indeed, one can imagine a case where the agent moves in its environment along the x axis from (external) position τ (with internal configuration m) to τ′ (same m). In this case, the sensor samples two different parts of the color wall so that its generated sensations s and s′ from these two different positions are different. Then two identical internal configurations m give two different sensations: there is no more mapping between m and s, and all the mathematical developments performed in previous works can no longer apply. Therefore, it seems necessary to generalize these formalisms to cope with agents able to move freely in their environment. In this paper, one proposes a variational formulation of motor actions to deal with this issue. Importantly, the term variational refers in all the following to the focus given on specific sequences of states (e.g., motor, sensory, or external states) rather than any specific one of said states. It is introduced in the next subsections.

From previous arguments, the internal motor configuration m can not be mapped unambiguously to sensations without additional considerations. If one still insists on having a functional relation between motor data and sensations, one then needs to enrich the initial motor set. In this paper, one proposes to introduce some superset B of M as initial parameter space. This new set B can be thought of as the set of all absolute configurations b made of pairs (m, τ) where m is the internal motor configuration and τ represents an absolute measure of the pose of the agent in its ambient space (which would most commonly be position and orientation in 3D space). While posture or proprioception give m a definite meaning, one should instead only think of τ as a choice of reference frame in space. Indeed for spaces much like ours it ought to be somewhat arbitrary since any “displacement” from pose τ to τ′ could be instead realized as an opposite motion of the whole ambient space like for compensatory movements as initially introduced in Poincaré (1895) and dealt with in Philipona et al. (2003). This equivalency argument has been mentioned in previous contributions as a possible way to deal with mobile agents as proposed in this contribution. While this could be formalized in a consistent way, either from a quasi static or variational perspective, we argue the proposed point of view offers usability advantages, like for multi-agent situations (where space should emerge as a shared common playground), an easier formulation of compensability, or a clear separation between the internal and external points of view. It is important to understand that the agent itself has no knowledge of the current absolute configuration b of its interaction with its environment, retaining the same hypotheses about a priori structure. However, we may then consider the sensorimotor map as a function ψ:B×E→S instead of ψ:M×E→S to account for possible displacements in the environment. Defining such a new set B allows then to introduce the notion of external agent configuration as the tuple (b, s). As such, two different points of view must be stressed out: (i) the external point of view (i.e., coming from the designer of the system) will allow to characterize some properties of the agent interaction with its environment (through modelization, hypotheses, etc.), and (ii) the internal point of view which represents which data and concepts are available to the agent for its operations. This specific point is discussed in section 3.1.

Coming back to Figure 1, the agent moves to three successive absolute configurations b, b′, and b″. All of them are now different, which was not the case of the internal motor configurations: introducing b ∈ B apparently solves the issue raised at the end of section 2.1.1. Let us now explain how the agent actually reaches some given absolute configuration b.

As explained previously, the agent has no direct access to the configuration data b: it cannot know where it is in B. Instead we suppose it starts with some (very limited) knowledge of how it moves in this set, i.e., it is capable of performing some moves in B and of comparing any two moves for equality. To this end, we propose to introduce some new set A behaving in the following manner: an element a∈A can be applied to any absolute configuration b ∈ B to give a new configuration b′ = ab = a(b). Therefore, a can be seen as a function B→B. We will usually denote b→ab′ this situation, and call a a motor action. Such a definition for “actions” differs from many intuitions since it is restricted to quasistatic differences in posture and position; it does not account for a notion of dynamical effort exerted by actuators. In particular, no dynamical effects are considered at this level and no specification is made of the precise motor path taken from b to ab. Instead, only these pairs of related (b, ab) endpoints are relevant to characterizing any action a. This constitutes a present limitation pervading much of similar works to which a future—and assuredly significant—contribution shall be devoted.

Now as we intend to represent the way in which the agent can move in its environment, one can take for granted the existence of a special action e∈A that verifies ∀b∈B, b→eb: the agent may decide to stay still. Note that for certain systems, e.g., drones or bipedal walkers, this is distinctly different from doing nothing since constant posture and position must still be maintained. Moreover, considering it is able to do any moves a and a′, it may then chain them in one single move a″=a′a∈A which satisfies

so that A naturally carries the structure of a monoid. Remark that this composition operation is necessarily associative since motor actions are assumed to behave as functions B→B. In the following, we will further restrict ourselves to the case where individual actions are reversible, that is for any action a there exists an action a⁻¹ such that

making A into a group. Seeing as how actions can be thought of as mappings B→B, a necessary (and sufficient) condition is for all mappings in A to be bijective. It is clear that this assumption of invertibility may not apply in some experimental contexts, e.g., an agent may jump down a height which it cannot jump up. This constitutes a current limitation of the proposed framework, although several factors may limit its severity. In this example, even if the agent cannot jump up the height directly, it may still find a sequence of actions allowing it to climb back up to its original position. Corresponding to the definition of Equation (2), this would result in an inverse action in the formal sense, as illustrated later in section 4.3.3.

Figure 1 illustrates these notions, with the agent moving from external configuration b to b′ through an action a. This action, applied at b = (m, τ), happens to produce a translation of the agent so that its internal motor configuration finishes at the same m. Note that the agent would be able to return back to its initial absolute configuration by applying the inverse action a⁻¹ of a. Moreover, since a is a function defined on the whole B set, the same action can be applied at b′ = (m, τ′) to reach a third configuration b″ = (m″, τ″). This time, the same action a has conducted to a global displacement of the agent in the environment, combined with a change in its internal motor configuration. Indeed, while it represents cases which are mostly avoided for practical reasons, it is not required for a to only depend on m in the general case: the outcome of the same action a may depend on the position τ of the agent in the environment. Finally, the agent would have been able to move from b to b″ by applying the action a″ = a², as per Equation (1).

With these structure assumptions, for a given subset of motor primitives A′⊂A available to the agent, we can search for the set of composed moves the agent can actually reach by iteration of its known ones. We shall say an action a∈A decomposes over A′={ai}i∈I if it can be written in the form

This represents a formal property functionally similar to that of compositionality of motor trajectories, with A′ filling for actions the role of primitives (Flash and Hochner, 2006). Indeed, the interest of these decompositions appears because the effect of composed moves on motor configurations boils down to the effects of its components as per the following diagram:

In the example in Figure 1, it may well be that the agent can move to any configuration b, i.e., that its action set is A=ℝ5 (for the five possible angular increments of its five joints). But it may also be restricted to a limited set of moves, for example if it only can send discrete commands to its joints. For instance, if each actuator is a stepper motor, then its action set turns into A=ℤ5. In this case, a would be written as the tuple (Δ_qi)i, i = 1, …, 5, where Δq_i ∈ ℤ is the ith motor increment expressed in step increments. Consequently, any action a would decompose over A′={ai}i where action a_i corresponds to adding one step to the ith actuator. In this specific case, while A is infinite, it is sufficient for the agent to know the five motor primitives a_i to generate any action a∈A. This is very similar to the notion of reducing a (finite dimensional) vector space, which is usually infinite, to the very finite subset of a generating set or if possible a base. However it can be proven that any finite subset of ℝ will not generate it as a group, and that it will often only generate a discrete kℤ subgroup. This occurs in the proposed simulations in section 4, where all combinations of a finite subset of starting actions lead to the discovery of a discrete generated action group.

In previous works, a traditional way for parameterizing the environment was to introduce the environment configuration ϵ. The meaning of such a variable was often left unspecified, almost without any formal semantics linking it to the sensorimotor experience of the agent (Laflaquière et al., 2015). In this paper it is proposed to stress the difference between the ambient geometrical space—in which sensorimotor experience occurs—and the environment itself—that is the state of “things” lying in this space. The former takes the form of some set X endowed with a spatial structure as encoded by a group G(X) of admissible transformations. These spatial transformations are mappings X→X preserving some “geometry” of X. The most common illustration is the usual affine geometry of ℝ³ given by the group SE3(ℝ)=SO3(ℝ)⋊ℝ3 of its rigid transformations, made of 3D rotations, translations and their compositions. On this basis, one chooses to particularize a “state of the environment” as a valuation that maps each point of X to its corresponding physical properties, such as temperature, color, luminance, etc. These states are therefore best represented as functions ϵ:X→P where P is a set describing the different physical properties the agent can observe. Consequently, ϵ(x) represents the observable physical properties at point x∈X. We will henceforth denote E the set of environment states, i.e., a set of such functions ϵ.

Figure 2 illustrates these considerations. In this simple case, the geometrical space X is monodimensional, represented as an axis where each point x is assigned a color through a function ϵ₁ or ϵ₂. Interestingly, one can now distinguish points in X on which the physics described by different environment states ϵ₁ and ϵ₂ locally coincide, as far as the agent is able to observe this coincidence. Particularizing the former unspecified ϵ state to a function ϵ of the spatial variable will allow to express new properties of the sensorimotor experience, as in the next subsection.

Now that we have formally defined what is “out there” from an external point of view, let us now focus on the sensory capabilities of the agent. On this specific point, most previous contributions were considering the full sensory output as atomic data: although it is implemented as a possibly high dimensional vector, elements and subarrays were generally kept from scrutiny. On the contrary, we now take interest at the subarray level and accordingly adapt the formalism. Therefore, in all the following the sensorimotor map is written as ψc:B×E→Sc where the c subscript outlines that the sensory map is explicitly written for a sensory element c (or sensel, i.e., one pixel for a camera, the cochlea cell coding for one sound frequency, etc.). Thus, the sensorimotor map ψ_C for the entire sensory apparatus is made of the aggregate of all sensels along ψC:B×E→S=∏c∈CSc with C the set of all sensels.¹ An illustration of these points is proposed in Figure 2 for a similar agent endowed this time with two cameras so as to better show the descriptive capabilities of the formalism. In this case, the sensels c_i—each depicted as elements in a color array—represent the pixels of either camera. Separate sensors in the apparatus thus appear as sub-arrays in C: the first (resp. second) camera is figured by C₁ (resp. C₂). Note that this decomposition of C as C₁ ∪ C₂ directly comes from our external understanding of the agent structure (i.e., with one camera corresponding to one set of sensels, i.e., one sensor). One could have selected others sub-arrays to form a distinct set of (virtual) sensors not necessarily corresponding to their (physical) implementation on the agent.

With space and sensors made formally precise we can now proceed with the (spatial) receptive field of sensor C′ ⊂ C, that is a region of space which environment state suffices to determine the output of C′. This region as a subset of X should naturally depend on the current configuration b since moving causes one's sensors to sample new parts of space, so that it takes the form of a map b∈B↦FC′(b)⊂X. Then, its characteristic property is

Figure 2 represents some of the receptive fields for the two cameras agent. The first one, F_ci(b), is the receptive field of a single sensel/pixel c_i ∈ C. The receptive fields F_C1(b) and F_C2(b) of each camera can be obtained as the union of the receptive field F_cj(b) of their respective pixels. In the same vein, the overall receptive field of the agent F_C(b) is also given by F_C1(b) ∪ F_C2(b). From the same figure, it is clear that even if ϵ₁≠ϵ₂ (since there are areas of different colors on the X axis), the sensation captured by the agent is the same since the aforementioned differences are restricted to areas of space unseen to the agent.

It is important to notice that this is the formal step where the notion of receptive field formalizes an implicit relation between the sensations of the agent and spatial structure. This constitutes one fundamental property sufficient to leverage spatial knowledge from the agent interaction with its environment. The application of these theoretical elements is proposed in the next section.

The result obtained in the previous subsection exhibits an important property making internally available spatial matching between receptive fields at different timesteps of motor exploration. But given that the actual motor exploration follows the algebraic structure of actions A, it still remains to be shown how these two structures are consistent. This can be made apparent by introducing conservative actions as those a of A for which all sensels of the agent exchange the places they sample: there is conservation of the (spatial) information available. In terms of the formalism, a∈A is conservative if it verifies

generalizing somewhat Equation (7). This characterization makes apparent that many actions can't be conservative: for example, “turning back” may only be conservative for the rare agent that “sees” precisely as much forwards as it does backwards. In fact, the spatiality of the condition on receptive fields makes it so that all readily found conservative actions correspond to displacements of the body of the agent. In the following, “∀b∈B,Fc(ab)=Fc′(b)” will be shortened to the more legible c→ac′, and c (resp. c′) is said to be the predecessor (resp. successor) of c′ (resp. c) by a. It is proven in Appendix 1 that for conservative actions a, the relation →a can be made into a successor function

where c′=σa(c) is a sensel verifying c→ac′. Therefore, conservative actions can equivalently be thought of as permutation of sensels. Importantly, conservative actions provide a natural framework for exploiting Equations (7) and (8) during motor exploration. Indeed, it is proven in Appendix 2 (see the Supplementary Materials) that conservative actions form a subgroup AC⊂A for its composition operation. That is to say, chaining conservative actions yield other actions which are necessarily conservative, and the inverses of conservative actions are themselves conservative.

At this stage, it has been shown how the spatial property of permutation of the receptive fields relates to the intrinsic motor structure of the agent. However, this does not suffice to make this group structure of conservative actions accessible to the agent given the a priori knowledge we discussed in subsection 3.1, since the dependency of the sensorimotor process on the spatial variable is implicit. We must therefore go through one final step to relate the available informational content (i.e., sensory reading) to the motor structure.

Let us consider the agent at any point (b, s) of its sensorimotor experience. Its sensory output is s = ψ_C(b, ϵ) = (_{sc)c ∈ C}, and for any action a it may perform this sensory output should shift to s′ = ψ(b′ = ab, ϵ) provided the environment state stays constant throughout the action. If we now restrict ourselves to the case of conservative actions, we get

so that performing motor action a only results in a permutation of the components of the sensory output. This permutation is exactly σ_a, and therefore is a constant of the agent which does not depend on the actual current configuration (b, ϵ). Equation (11) shows that any conservative action a∈AC corresponds to a sensory function

which verifies the property

Per this property, Π_a is a function which given any starting sensory reading of the agent can determine the sensation it would experience after performing action a (provided the environment state stays constant during a). It must be reiterated that a crucial part is that this function operates on sensory data, which is precisely the only data available to the agent.

The first experiment performed by the agent is computing, where possible, the permutation matrix associated to each of its available motor actions. This is done according to the following procedure: at the beginning of the sensorimotor experience, to each starting action ak∈Ainit associate a N_c × N_c matrix M_ak where N_c is associated the number of sensels. This matrix is initialized so that all of its coefficients are 1. Then, at the end of timestep t_n where it performed action a_k (that is a[t_n] = a_k), the agent uses its sensory output arrays both previous (s[t_n]) and current (s[t_n+1]) as per the update rule:

Let us first observe that in this rule the only possible change in coefficients is going from 1 to 0: whenever a coefficient (_{Mak[tn])i, j} is already 0, the condition of the first case automatically fails so that its value stays at 0. Therefore, the rough dynamics of the update is that while all coefficients start at 1, some are eventually switched to 0 upon exploration until matrices converge to a final (possibly null) form.

One can note that this is a very drastic choice compared to the more usual soft incremental rules. This offers increased simplicity, such as in Appendix 4 (see the Supplementaty Materials) where an argument is provided that for any conservative motor action this algorithm makes the empirical matrix M_ak converge to the associated permutation matrix M_σak. Moreover, we argue that obtaining said convergence with such an unforgiving rule is strong evidence toward the systematic, rather than statistical, nature of the supporting mechanism. The argument also proves that for non-conservative actions, under the same richness hypothesis the associated empirical matrix will converge to the null matrix. This fact allows the robot to naively distinguish between conservative and non-conservative actions, should he be given the capability to perform both on startup.

In the second phase of the experiment, the agent uses the prediction functions it discovered for elementary conservative moves to infer how combinations of these moves relate to each other. Indeed, it was proved in the previous part that for any conservative actions a and a′ with associated permutation matrices M_σa and Mσa′, it is true that

In the case of actions which are not strictly conservative, such as those in the simulation, equality in the previous equation is not guaranteed. This happens because in the Mσa′Mσa expression, all the loss of information of a and a′ on their respective boundaries is accumulated, whereas a′a might recoup some of it, e.g., when a′ = a⁻¹. However, multiple expressions should at least yield non-contradictory prediction, that is whenever one specifies a pair c_i→c_j another other cannot assert c_i→c_k with j≠k. As long as these combinations are kept short enough to limit the accumulation, this non-contradiction criterion can be used by the agent to internally infer the sensory prediction of any combination of the moves it empirically learned. This is used in a Dijsktra-like process to build a graph of prediction matrices, which runs as follows, see Algorithm 1: starting from a prediction matrix M₀ corresponding to any origin action a₀, each of the known matrices Mak, ak∈Ainit are applied to yield both a set of new neighboring “end points” NM0:={MakM0, ak∈Ainit} and for each pair (M₀, M_akM₀) a directed edge M_ak. This is then recursively applied to all newly discovered end points, while those that were previously visited (as the prediction matrices can be compared for equality) are discarded. However, the resulting graph would in most cases be infinite, therefore a stopping rule must be chosen. In our case, we chose to explore up to a given depth parameter in graph edge distance.

From the previous experiment, we now have as many permutation matrices M_ak as we have actions in Ainit. As outlined in section 4.2.2, one can then use them to build a graph of prediction matrices by following Algorithm 1. Recall that in this graph, a node is a permutation matrix obtained as a combination of the M_ak matrices, while there is a M_ak edge from matrix M to M′ iff M′=MakM. Therefore, all edges in the graph correspond to the permutation matrices built during experiment 1. According to Equation (15), this graph is isomorphic to the graph of corresponding actions, meaning all properties discovered of any combination of matrices holds true for the corresponding combination of actions. As an example, if one discovers that M_a1 = M_a2M_a3, then one also has a₁ = a₂a₃.

As a first step, let us consider only the actions corresponding to translations in the environment, i.e., a₆ and a₇ are discarded from Ainit. This a priori selection is only made to simplify the visualization of the graph at first. After applying Algorithm 1 to the matrices shown in Figure 5, one gets the directed graph in Figure 8A, where all the color conventions are consistent with experiment 1. This particular graph has been built for a maximum depth set to 3 and with M_a1 taken as the origin of the graph. Note that the depth of this graph has been maintained voluntarily low so as to help in the reading of the graph. Note also that the arbitrary choice of origin makes all of its neighbors themselves correspond to one of the M_ak discovered in experiment 1 since they all occurred as M_akM_a1 = M_ak products, whereas all other nodes are indeed new matrices.

This graph mirrors many algebraic properties of the M_ak as captured by the internal experience. Indeed one can first observe that the light blue arrow leads from any given node M to itself, which corresponds to M_a1 being the identity matrix I_Nc. Furthermore, one can note that the graph obtained is, up to its borders, completely homogeneous; that is the neighborhoods of each interior nodes share the same geometry. This even extends to the color of edges matching, so that some of them form pairs. One can for example verify that whenever a yellow edge goes from node M to M′, there is a green edge from M′ to M and no other one. This identifies the corresponding actions to be inverses w.r.t. successive execution since from any starting node, taking first the green (resp. yellow) edge then the yellow (resp. green) one forms a loop. The same can be said of the red and purple colors, which are found to correspond to another pair of inverse actions. At last, the four central squares correspond to the commutativity of the a_k used: indeed one can see on the graph that taking the red edge first, then the green one always leads to the same node as green first, red second.

While those observations were discussed as properties of the permutation matrices, the actual result is their representing properties of the abstract motor actions a_k. And indeed one can check that the blue arrow corresponds to the identity action a₁, that the inverse pairs (yellow, green) and (orange, purple), respectively correspond to (rightward, leftward) and (forward,backward) translations, and that the commutativity discussed is that of “forward then left” being the same as “left then forward.” While these facts seem obvious from an external point of view, they were not part of the initial knowledge of the agent discussed in section 3.1. This only appears as a consequence of the agent capability to predict the sensory consequences of its own actions built during experiment 1. On a functional level, this is very similar to the property of motor sequence compression exhibited by RNNs performing sensorimotor prediction in Ortiz and Laflaquière (2018); in fact we argue that it is the same phenomenon that is picked up on by the neural networks and that it is intrinsically related to sensorimotor prediction as developed in section 3.

This also applies to the graph shown in Figure 8B obtained when considering all seven actions, i.e., the two rotations corresponding to actions a₆ and a₇ are now included in the analysis. This plot, obtained through a classical force-directed algorithm, shows the same 2D graph of translations obtained before, but enriched with a third dimension supporting the change of orientation induced by rotations. Again, the depth of the graph is maintained low to keep things legible. The global structure of the graph can be described as a disjunction of 2D subgraphs corresponding to translations at fixed orientation. Each subgraph is therefore equivalent to each other up to a rotation as can be seen by the edges colors shifting between the planes. As an example, one can see that the same node in the graph can be reached by following either (green, dark blue) and (dark blue, purple), or (left, turn left), and (turn left, forward) in terms of actions seen from an external point of view. Figure 8B also shows that rotations are limited to the third vertical dimension in which they form cycles at constant position in the planar subgraph. This property is highlighted in the graph by the four circled nodes which figure the same agent position for the four possible orientations. The cycle simply mirrors the external observation that taking four π/2 rotations successively takes one back to the initial orientation. Importantly, this could constitute an internal signature of rotations as opposed to translations.

In the end of this second experiment, the agent has thus been able to discover a structure of its actual group of motor actions. The agent now has access to algebraic relations between its own actions which relate to its motor capabilities. This knowledge also allows it to generalize the sensory prediction it discovered in experiment 1 to all the combinations considered in the graph. Nevertheless, one has to keep in mind that all the actions considered in these experiments are not exactly conservative in the sense of Equation (9). Indeed, they fail at conserving spatial information on the border of the simulated camera. However, results show that conservativity holding true for all internal pixels allows for discovering the aforementioned properties. Because all calculations are made on actual current outputs of the pixels, one obvious consequence is that the agent has no way to predict what happens outside of its field of view, and so far it keeps no memory of it. Therefore, the results only hold for a very local movement w.r.t. the dimension of the agent. However, the discovered structure stays true whatever the initial position of the exploration.

In the previous results, we have run the experiments with an experimental starting action set Ainit conveniently set to A, the set of externally defined movements. While this has been useful at first to yield easily recognized structure in Experiment 2, it is a crucial point that the results do not depend on this strong assumption. Therefore, the same two part experiment is conducted with the difference that the starting action set Ainit the agent can run is not arbitrarily set to A anymore. Instead, it is now drawn in the set of combinations of actions a_k ∈ A. Three important cases are now possible : first, it may be that some of the a_k are “missing” in Ainit; on the contrary, duplicates may have been drawn so that the agent can run a,a′∈Ainit which are effectively the same action (i.e., ∀b ∈ B, ab = a′b). Finally, it may even have drawn “complex” actions a∉A, that is actions that can only be obtained by combining some of the a_k.

The three situations are illustrated in Figure 9, where the graphs obtained at the end of Experiment 2 are drawn for various starting action sets. In both cases, both the complexity of the starting action set and the depth of the depicted graph have been limited to keep the discussed structure as readable as possible. Figure 9A depicts the first two cases: the agent was given a duplicate action from A as well as missing one. This can be assessed in the resulting prediction graph by the yellow and black arrows which relate the same ordered pairs of nodes, e.g., from the highlighted red to blue nodes, and the lack of an inverse arrow that would match them. Note that while the absence of this inverse (green) arrow represents the lack of a “direct” inverse action in Ainit, the emerging structure from the graph allows for the determination of an inverse path as highlighted by the bold (red, orange, blue) arrows. From an external point of view, this basically means that if the agent has no action to translate itself to its left, it can instead rotate clockwise, then move backward, and finally rotate counter-clockwise to reach the correct orientation. Interestingly, this phenomenon where a missing inverse can be otherwise obtained by combination of other actions can only occur when the agent is able to rotate. The third situation is depicted in Figure 9B, where the experiment was conducted with the robot given the additional action a₈ “forward then rightward” along a₈ = a₅a₃ in addition to the translations of A. The choice not to give the agent its defined rotations a₆ and a₇ serves only to get an easily legible picture of the resulting graph, much like in Figure 8A, and does not impact the following. This additional move a₈, which we as an external observer know to be a combination, is studied like all other basic actions by the agent during the motor babbling phase. It means that the agent has no cue about a₈ being an actual combination of two other actions. In the end of the experiment, the obtained graph of action exhibits this additional action a₈ as black edges, as shown in Figure 9B. From this graph, one can easily see that, from any point, it is indeed equivalent to follow either the black arrow or first the orange and then the yellow ones. The agent has thus been able to discover the action combination property.

These graphs therefore show two important results. The first conceptual comment is that the validity of the proposed experiment is not conditional to a perfect match between ideal, “objective” moves of the agent and actions it is effectively able to perform at start. This is a desirable property for genericity and our goal of bootstrapping, for it allows to avoid justifying said match. The second, more practical, comment is that the graph resulting from the experiment can be used by the agent to select an action set “better” than Ainit. Indeed, the redundancy between edges (or paths of edges) in Figures 9A,B represent the agent discovering it can discard the actions corresponding to black edges without losing capabilities, i.e., while keeping all nodes reachable. It can then be used to prune the available action set to a minimal set generating the same group, in the sense of Equation (4). Determining a criterion for selecting which actions are kept and which are discarded could functionally correspond to a basis for invariant principles in motor actions of the agent (Flash and Hogan, 1998). The agent may also expand its action set with new actions that verify useful properties: for example, in the case depicted in Figure 9A it can package the bolded path of edges into a single action so that it gets a missing inverse.