<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xml:lang="EN" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Neurorobot.</journal-id>
<journal-title>Frontiers in Neurorobotics</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Neurorobot.</abbrev-journal-title>
<issn pub-type="epub">1662-5218</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fnbot.2022.847054</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Neuroscience</subject>
<subj-group>
<subject>Original Research</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Using enactive robotics to think outside of the problem-solving box: How sensorimotor contingencies constrain the forms of emergent autononomous habits</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes">
<name><surname>Egbert</surname> <given-names>Matthew D.</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
<xref ref-type="corresp" rid="c001"><sup>&#x0002A;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/125494/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Barandiaran</surname> <given-names>Xabier E.</given-names></name>
<xref ref-type="aff" rid="aff3"><sup>3</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/77199/overview"/>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>School of Computer Science, University of Auckland</institution>, <addr-line>Auckland</addr-line>, <country>New Zealand</country></aff>
<aff id="aff2"><sup>2</sup><institution>Te Ao M&#x00101;rama, University of Auckland</institution>, <addr-line>Auckland</addr-line>, <country>New Zealand</country></aff>
<aff id="aff3"><sup>3</sup><institution>IAS-Research Center for Life, Mind, and Society, Department of Philosophy, University of the Basque Country (UPV/EHU)</institution>, <addr-line>Bilbao</addr-line>, <country>Spain</country></aff>
<author-notes>
<fn fn-type="edited-by"><p>Edited by: In&#x000EA;s Hip&#x000F3;lito, Humboldt University of Berlin, Germany</p></fn>
<fn fn-type="edited-by"><p>Reviewed by: Alban Laflaqui&#x000E8;re, Softbank Robotics, France; David Vernon, Carnegie Mellon University Africa, Rwanda</p></fn>
<corresp id="c001">&#x0002A;Correspondence: Matthew D. Egbert &#x02709; <email>m.egbert&#x00040;auckland.ac.nz</email></corresp>
<fn fn-type="other" id="fn001"><p>This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Neurorobotics</p></fn></author-notes>
<pub-date pub-type="epub">
<day>21</day>
<month>12</month>
<year>2022</year>
</pub-date>
<pub-date pub-type="collection">
<year>2022</year>
</pub-date>
<volume>16</volume>
<elocation-id>847054</elocation-id>
<history>
<date date-type="received">
<day>01</day>
<month>01</month>
<year>2022</year>
</date>
<date date-type="accepted">
<day>29</day>
<month>09</month>
<year>2022</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x000A9; 2022 Egbert and Barandiaran.</copyright-statement>
<copyright-year>2022</copyright-year>
<copyright-holder>Egbert and Barandiaran</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p></license> </permissions>
<abstract>
<p>We suggest that the influence of biology in &#x02018;biologically inspired robotics&#x02019; can be embraced at a deeper level than is typical, if we adopt an enactive approach that moves the focus of interest from how problems are solved to how problems emerge in the first place. In addition to being inspired by mechanisms found in natural systems or by evolutionary design principles directed at solving problems posited by the environment, we can take inspiration from the precarious, self-maintaining organization of living systems to investigate forms of cognition that are also precarious and self-maintaining and that thus also, like life, have their own problems that must be be addressed if they are to persist. In this vein, we use a simulation to explore precarious, self-reinforcing sensorimotor habits as a building block for a robot&#x00027;s behavior. Our simulations of simple robots controlled by an Iterative Deformable Sensorimotor Medium demonstrate the spontaneous emergence of different habits, their re-enactment and the organization of an ecology of habits within each agent. The form of the emergent habits is constrained by the sensory modality of the robot such that habits formed under one modality (vision) are more similar to each other than they are to habits formed under another (audition). We discuss these results in the wider context of: (a) enactive approaches to life and mind, (b) sensorimotor contingency theory, (c) adaptationist vs. structuralist explanations in biology, and (d) the limits of functionalist problem-solving approaches to (artificial) intelligence.</p></abstract>
<kwd-group>
<kwd>enactive robotics</kwd>
<kwd>sensorimotor contingencies</kwd>
<kwd>problem-solving</kwd>
<kwd>habits</kwd>
<kwd>IDSM</kwd>
<kwd>sensorimotor autonomy</kwd>
</kwd-group>
<counts>
<fig-count count="16"/>
<table-count count="1"/>
<equation-count count="17"/>
<ref-count count="57"/>
<page-count count="23"/>
<word-count count="13071"/>
</counts>
</article-meta>
</front>
<body>
<sec sec-type="intro" id="s1">
<title>1. Introduction</title>
<p>Artificial Intelligence and the scientific approach to mind (what is known as <italic>cognitive science</italic>) was born (or rather raised) as a problem solving discipline (Newell et al., <xref ref-type="bibr" rid="B33">1958</xref>; Putnam, <xref ref-type="bibr" rid="B42">1965</xref>; Fodor, <xref ref-type="bibr" rid="B20">1968</xref>). Deprived of life, the machine metaphor was one of symbol manipulation and rationality (deductive, inferential, heuristic or otherwise). The unprecedented potential of Universal Turing Machines (computers) was the driving metaphor to study the mind. The software, the mind, was the problem solving method, the hardware, the brain, its implementation.</p>
<p>Alternative conceptions of the mind were available at the origins of Artificial Intelligence and Robotics (Grey Walter, <xref ref-type="bibr" rid="B23">1950</xref>; Ashby, <xref ref-type="bibr" rid="B2">1952</xref>) but the rapid success of computer science left them aside. Over time, the limitations of the problem-solving centered computational theory of the mind became apparent and the biologically inspired, embodied and later enactive conceptions of the mind gained momentum. We are now immersed in a mesh of hybrid architectures, applied to a wide range of practices, from industrial to scientific modeling applications, and a new summer of Artificial Intelligence is rising, with robotics as a major container of social and technological expectations.</p>
<p>There are good reasons for why problem-solving attracts so much attention from researchers, but it is pertinent to ask: what aspects of minds are omitted or obscured by the problem-solving focused perspective? what can life teach us about what intelligence holds before and beyond problem-solving? and even when problem solving is addressed &#x02026;how is it that natural agents have and become concerned by <italic>their own</italic> problems?</p>
<p>This paper has two goals. The first is to argue that by abandoning problem solving (or at least putting it down for a time), other useful explanatory targets and ways to explain minds are given space to emerge.</p>
<p>The second, more specific goal is a case in point: we use simulated robots to show how sensorimotor contingencies influence the formation of self-maintaining patterns of sensorimotor activity &#x0201C;habits&#x0201D; in regular ways that depend upon sensory modality. By de-emphasizing problem solving, we are able to take a fresh look at the relationship between sensory modalities, sensorimotor contingencies and habitual behaviors. But to explain these results, we first need to provide more context.</p>
<p>The paper proceeds as follows. The next section explains what we mean by &#x0201C;problem-solving,&#x0201D; why it has been a popular target within the cognitive sciences, and what we see as the primary disadvantage of excessive attention being given to the topic. We then explore the intimate relationship between robotics and biologically inspired and embodied problem-solving paradigms. Section 2.3 introduces the enactivist concept of autonomy, providing an alternative framework for developing Sensorimotor Contingency Theory outside of the problem-solving approach. The remainder of the paper presents and analyzes a simulation model that is used to explain: (i) that robots must first have their own problems instead of solving those posited by external observers; (ii) that, in doing so, they must assert a way of life whose structure and form must be taken as the object of study. We finally discuss some of the larger implications of our enactive approach in connection with wider theories of biological explanation and inspiration.</p></sec>
<sec id="s2">
<title>2. From problem solving to enactive robot</title>
<sec>
<title>2.1. Problem-solving in minds and machines</title>
<p>We use the term &#x0201C;problem-solving&#x0201D; to refer, in a broad and inclusive manner, to the kinds of things that we associate with being capable or clever. Nowadays, for many, &#x0201C;the ability to solve problems is not just an aspect or feature of intelligence&#x02014;it is the essence of intelligence&#x0201D; (Hambrick et al., <xref ref-type="bibr" rid="B24">2020</xref>, p.553). It is certainly not a new idea. The very birth of Artificial Intelligence owes much to it (Newell et al., <xref ref-type="bibr" rid="B33">1958</xref>). As Newell and Simon later stated: &#x0201C;Since ability to solve problems is generally taken as a prime indicator that a system has intelligence, it is natural that much of the history of artificial intelligence is taken up with attempts to build and understand problem-solving systems&#x0201D; (Newell and Simon, <xref ref-type="bibr" rid="B34">1976</xref>, p. 120). The task of artificial intelligence was thus to devise potential solution-generators and to design tests that could evaluate them. This assumed that the problem space was well fixed so that solutions could both be evaluated and generated. Decades later, Artificial Intelligence handbooks still devote their first central sections to problem solving (e.g., see part II of Norvig and Russell, <xref ref-type="bibr" rid="B38">2021</xref>, 4th edition).</p>
<p>A problem is a context in which behaviors can be evaluated according to a norm of success at &#x0201C;solving&#x0201D; the problem. Problems vary from being trivial to challenging to impossible as the proportion of behaviors that are good (out of all possible behaviors) shrinks. They include &#x0201C;high-level&#x0201D; human problem-solving, such as the skills that are taught in schools or universities, as well as embodied problems such as balancing on two feet or swimming efficiently. For a system to be evaluated in terms of problem-solving, one must first have the specification of the context and of a normative evaluation so that behaviors within that context can be compared as more or less successful. Defined as such, just about anything can be seen as a problem solver. A bottle lid solves the problem of preventing spills; a car&#x00027;s differential solves the problem of distributing force effectively to its wheels; a computer program solves the problem of beating a human at chess.</p>
<p>Herein lies both the advantage and the disadvantage of placing problem solving at the center of the cognitive sciences: <italic>almost anything can be evaluated in terms of its problem solving ability</italic>. On the one hand this is a great boon. Quantifying problem solving ability is relatively straight-forward and the ability to quantify how effective a system is at solving one or more problems facilitates technoscientific progress where measurements can play an important role in defining progress. This is apparent in artificial intelligence (AI) research where benchmarks such as chess and other games (Canaan et al., <xref ref-type="bibr" rid="B11">2019</xref>), hand-writing recognition (e.g., Graves and Schmidhuber, <xref ref-type="bibr" rid="B22">2009</xref>), image classification, speech recognition, etc. (e.g., MLPerf benchmarks MLPerf, <xref ref-type="bibr" rid="B32">2021</xref>), are used to quantify progress. Problem-solving similarly provides metrics for studying the minds in psychology and neuroscience, where problem-solving related notion of &#x02018;tasks&#x02019; (e.g., the Simon task) are used to structure human activity and performance related metrics such as reaction speed or error rate are seen as providing key insights into how our minds operate (Simon and Wolf, <xref ref-type="bibr" rid="B47">1963</xref>).</p>
<p>The ease of measuring problem-solving ability sometimes leads to it (problem-solving ability) becoming the <italic>explanandum</italic>&#x02014;the thing we strive to understand. This is seen in research questions like: <italic>How do people recognize faces so well? How do babies come to understand the motives of other people? How do we play chess? How can we make a safe self-driving car?</italic> etc. Problem-solving also sometimes becomes the <italic>explananda</italic>&#x02014;the terms in which we explain what minds are, how they work, or what they do. Evolutionary psychology (Pinker, <xref ref-type="bibr" rid="B41">1997</xref>; Buss, <xref ref-type="bibr" rid="B10">1998</xref>), for example, emphasizes the evolutionary advantage of problem-solving ability and in this context, explanations that are provided in terms of problem-solving ability are seen as complete, as evolution can be invoked to explain why such mechanisms exist. The evolutionary advantage of having a mind is in its contribution to problem solving and therefore, minds are best understood as problem solving machines.</p>
<p>However, it has been argued that: &#x0201C;The essence of intelligence is to act appropriately when there is no simple pre-definition of the problem or the space of states in which to search for a solution. Rational search within a problem space is not possible until the space itself has been created, and is useful only to the extent that the formal structure corresponds effectively to the situation&#x0201D; (Winograd and Flores, <xref ref-type="bibr" rid="B55">1987</xref>, p.98). Thus, even from a problem solving perspective, intelligence is not really the capacity to solve a problem but to bring a situation into a fabricated frame where it can be treated as a problem to be solved.</p>
<p>Moreover, the problem with excessive focus upon problem-solving is that there are other unique and important features of minds that are worthy of study&#x02014;features that may only indirectly relate to problem-solving ability or perhaps not at all. The problem, in a nutshell, is the conflation of (i) &#x0201C;problem-solving ability&#x0201D; with (ii) all of the other phenomena associated with &#x0201C;being a mindful body.&#x0201D;</p>
<p>The mainstream computational functionalist approach to the mind (Putnam, <xref ref-type="bibr" rid="B42">1965</xref>; Fodor, <xref ref-type="bibr" rid="B20">1968</xref>) doesn&#x00027;t really help much addressing what mindful bodies are beyond problem-solving devices. For Putnam, the very definition of the mental is always in reference to a Turing machine table that works out rational transitions (e.g., computing and storing preferences over a utility function or solving problems in problem representation space). Deviations from this rationality are treated as pathological. All human mental life is, according to Putnam, not perfectly normal, thus relatively pathological. Putnam acknowledges &#x0201C;our model is an overly simple and overly rationalistic one in a number of respects. However, it would be easy, in principle, although perhaps impossible in practice, to complicate our model in all these respects&#x02014;to make the model dynamical, to allow for irrationalities in preference, to allow for irrationalities in the inductive logic of the machine, to allow for deviations from the rule: maximize the estimated utility. But I do not believe that any of these complications would affect the philosophical conclusions reached in this paper&#x0201D; (Putnam, <xref ref-type="bibr" rid="B42">1965</xref>, p. 43).</p>
<p>Deviations from the abstract rational rule are pathological. Explanation lies on the pure domain of abstract problem solving, the deviations make it all more complicated (as if dirt in the form of a set of exceptions where to be added to the pure explanation) but change fundamentally nothing. As we are about to see, embodied approaches, and particularly enactivism, bring these &#x0201C;pathological&#x0201D; expressions to the center of the explanation (of which rational thinking is the complicated achievement) turning it into the core constitution of mind. Biologically inspired robotics has a lot to contribute in this direction.</p></sec>
<sec>
<title>2.2. From biologically inspired problem-solving to enactive robotics</title>
<p>Enactivism was born under the conviction that robotics, as a field, would require, or even force, cognitive science to move beyond the problem-solving framework:</p>
<disp-quote><p>The assumption in CS [Cognitive Science] has all along been that the world can be divided into regions of discrete elements and tasks to which the cognitive system addresses itself, acting within a given &#x0201C;domain&#x0201D; of problems: vision, language, movement. Although it is relatively easy to define all possible states in the &#x0201C;domain&#x0201D; of the game of chess, it has proven less productive to carry this approach over into, say, the &#x0201C;domain&#x0201D; of mobile robots. Of course, here too one can single out discrete items (such as steel frames, wheels and windows in a car assembly). But it is also clear that while the chess world ends neatly at some point, the world of movement amongst objects does not. It requires our continuous use of common sense to configure our world of objects. (Varela, <xref ref-type="bibr" rid="B50">1992</xref>, p.251)</p></disp-quote>
<p>Yet, moving away from the problem-solving paradigm has taken a long path, most of which, rather than abandoning problem-solving has deeply transformed the way we understand how nature solves those problems. In a sense, biologically inspired robotics has mostly followed the problem-solving approach and biological inspiration has focused on picking up biological mechanisms to solve problems: from the internal neuronal inspiration of artificial neural networks since its early conception (Rosenblatt, <xref ref-type="bibr" rid="B44">1958</xref>) to their later development (Rumelhart et al., <xref ref-type="bibr" rid="B46">1988</xref>) to the embodied strategies that either transform the problems to be solved by their &#x0201C;brains&#x0201D; or have outsourced the computational load of the problem solving to body and world (Pfeifer and Scheier, <xref ref-type="bibr" rid="B40">2001</xref>). What radically distinguished biologically inspired robotics from GOFAI (Good Old Fashioned Artificial Intelligence) was a change of focus from the abstract to the concrete, from the symbolic to the sensorimotor and from the rational to the practical know-how of situated action. Despite the emphasis on self-organization, agent-environment emergence of behavioral functioning (Steels, <xref ref-type="bibr" rid="B49">1990</xref>) embodiment and situated action (Maes, <xref ref-type="bibr" rid="B28">1990</xref>), etc. the main goal was still to build robots capable of solving behavioral problems. After all, to put it with biologically-inspired roboticist Barbara Webb: &#x0201C;The sensorimotor problems faced by animals and by robots have much in common&#x0201D; (Webb, <xref ref-type="bibr" rid="B54">1995</xref>, p. 117) and, not only can animals help us devise robots that solve problems in a biologically inspired manner, but also solving a sensorimotor problem with the robot could help us understand how the animal solves it; like &#x0201C;[d]etecting which ear is closer to the sound&#x0201D; which &#x0201C;is a non-trivial problem for the cricket&#x0201D; (Webb, <xref ref-type="bibr" rid="B54">1995</xref>, p. 120).</p>
<p>Other trends of biological inspiration have built upon evolutionary theory itself and artificially evolved brains or brains and bodies to solve the problems (encoded as fitness function) in what is commonly known as evolutionary robotics (Cliff et al., <xref ref-type="bibr" rid="B12">1993</xref>; Nolfi et al., <xref ref-type="bibr" rid="B37">2016</xref>). Random variations to the parameters of robotic brain&#x00027;s and bodies are selected against a fitness function that operates as the benchmark of the problem to be solved. Despite the problem-solving focus that is almost inherent in artificial evolutionary optimization techniques, evolutionary robotics served to disclose a number of principles of behavioral self-organization that non-linear, fine grained agent-environment coupling display when artificial evolution can freely explore the solution space without the prejudices inherent to the human design (Harvey et al., <xref ref-type="bibr" rid="B25">1997</xref>).</p>
<p>Some of these approaches entail radical departures from core assumptions of the computational functionalist theory of the mind: cognitive processing does not only occur in the head and the body must be integrated as a key feature of cognitive problem solving (not simply as an executioner of the solution representation worked out in the head or a sensory transmitter of the problem into it); agent-environment interactions can self-organize with little if any representations; material bodily and interactive constitution are not mere implementation details of abstract capacities but intrinsic part of the problems and solutions that cut across them.</p>
<p>But <italic>enactive robotics</italic> moves yet further into biological inspiration. On the one hand there are enactive approaches that have attempted to introduce more of the living body of natural intelligence into robotics by including self-organized mechanical, soft bodies (Man and Damasio, <xref ref-type="bibr" rid="B29">2019</xref>) or even chemical bodies (Damiano and Stano, <xref ref-type="bibr" rid="B13">2021</xref>). But also, and perhaps most relevant for enactive theory transferring to the robot what metabolism has to offer to anchor intrinsic needs or emotional feedback (Ziemke and Lowe, <xref ref-type="bibr" rid="B57">2009</xref>). There is however another enactive path that brings into robotics some principles of living organization and, more specifically, the autonomy of behavior, a <italic>way of life</italic> for robots, not as somehow transferred from the biological body, but enacted at the scale of brain-body-world dynamics (Barandiaran and Moreno, <xref ref-type="bibr" rid="B3">2006</xref>). Some forerunners of this inspiration are no doubt Ross Ashby on the organism-centered inspiration of adaptive controllers as machines capable to remain homeostatic in the face of perturbations (Ashby, <xref ref-type="bibr" rid="B2">1952</xref>) and Grey Walter&#x00027;s &#x0201C;life imitating&#x0201D; robots &#x0201C;designed to illustrate the uncertainty, randomness, free will or independence so strikingly absent in most well-designed machines&#x0201D; (Grey Walter, <xref ref-type="bibr" rid="B23">1950</xref>, p. 44).</p>
<p>More recent development of this line of inspiration on natural and biological principles for the design of robots came hand in hand with the development of a theory of autonomous behavior and agency (Smithers, <xref ref-type="bibr" rid="B48">1997</xref>), organismically inspired robotics (Di Paolo, <xref ref-type="bibr" rid="B14">2003</xref>), and habit-centered enactive robotics (Egbert and Barandiaran, <xref ref-type="bibr" rid="B16">2014</xref>).</p></sec>
<sec>
<title>2.3. Enactivism and the autonomy of sensorimotor life</title>
<p>Varela, Thompson and Rosch opened up a new way of thinking in 1991. Their enactive approach conceives that &#x0201C;cognitive structures emerge from the recurrent sensorimotor patterns that enable action to be perceptually guided&#x0201D; (Varela et al., <xref ref-type="bibr" rid="B51">1991</xref>, p.173). They later stated that: &#x0201C;[C]ognition is no longer seen as problem solving on the basis of representations; instead, cognition in its most encompassing sense consists in the enactment or a bringing forth of a world by a viable history of structural coupling&#x0201D; (Varela et al., <xref ref-type="bibr" rid="B51">1991</xref>, p.205). The enactive approach thus emphasizes sensorimotor coupling and the recurrent patterns that emerge from agent-environment interactions.</p>
<p>This way Varela overcame the operational (en)closure of the nervous system that served as his main analogy with the organization of the living, captured (together with Humberto Maturana) within the theory of autopoiesis. Ever since, the relationship between the self-organizing nature of nervous activity and that of behavior became to some extent problematic (see Barandiaran, <xref ref-type="bibr" rid="B6">2017</xref> for a discussion).</p>
<p>Inspired by Maturana and Varela, Tim Smithers re-states the need for biologically inspired autonomy in robotics, in the context of the impossibility to design robot agents from a problem-solving stand point:</p>
<disp-quote><p>Designing and building autonomous agents thus becomes the problem of designing and building processes that can support and maintain this kind of identity formation through interaction: processes that, through interaction, are continuously forming the laws of interaction that can sustain and maintain the interaction needed to form them. In other words, we need interaction processes that can support the self-construction and maintenance of interaction processes through interaction, in essentially the same way that the material and energy interaction processes of single cells can be understood as being involved in the continual forming of the mechanisms that support this interaction. Such systems will thus be self-law making as well as self-regulating, in essentially the same way as we can understand biological systems and autonomous city states. (Smithers, <xref ref-type="bibr" rid="B48">1997</xref>, p.102)</p></disp-quote>
<p>This analogy between metabolic autonomy and the autonomy of behavior was further explored in Di Paolo (<xref ref-type="bibr" rid="B14">2003</xref>). According to this view, enactivism needs not be committed to build bio-chemically living robots (provided that this is possible or even desirable) but to endow a robot with a <italic>way of life</italic>. This intuition was further explored in Barandiaran (<xref ref-type="bibr" rid="B4">2007</xref>, <xref ref-type="bibr" rid="B5">2008</xref>).</p>
<p>Sensorimotor Contingency Theory can further enrich this approach. It uses regularities in the ways that motor activity affect sensory activity (sensorimotor contingencies) to explain the qualities of perceptual experience (O&#x00027;Regan and No&#x000EB;, <xref ref-type="bibr" rid="B39">2001</xref>). Empirical research involving sensory substitution, sensory modification, psychophysics has informed the development of sensorimotor contingency theory (SMCT), which attempts to explain diverse aspects of perceptual experience, including why certain sensory modalities have a particular &#x0201C;feel&#x0201D; to them; how it is possible to make one experience one sensory modality (e.g., touch) in a way that feels more like another (e.g., sight); and the conditions in which subjects are (or are not) capable of adapting to major transformations to their sensorium. The key idea in SMCT is the role that action plays in perception: a classical enactive theme (No&#x000EB;, <xref ref-type="bibr" rid="B36">2004</xref>).</p>
<p>Using variations of the basic theme of how motor activity modifies sensory input, a set of robotic architectures where made using sensorimotor contingencies as building blocks for robotic design (Maye and Engel, <xref ref-type="bibr" rid="B31">2013</xref>; Jacquey et al., <xref ref-type="bibr" rid="B26">2019</xref>). But these are hardly enactive in the sense of the deep biological inspiration that the enactive approach can offer.</p>
<p>Perhaps the best way to explore such potential is to bring forth the concept of sensorimotor autonomy (updated and refined from a previous proposal of Mental Life and also explored in more detail on the concept of Sensorimotor Life): the capacity of an agent to sustain and regulate the structures that generate behavior. This definition echoes the metabolism-based definitions of life as far-from-thermodynamic equilibrium chemical systems capable of maintaining the network of chemical reactions that constitute it (G&#x000E1;nti, <xref ref-type="bibr" rid="B21">1975</xref>; Maturana and Varela, <xref ref-type="bibr" rid="B30">1980</xref>; Rosen, <xref ref-type="bibr" rid="B43">1991</xref>; Ruiz-Mirazo et al., <xref ref-type="bibr" rid="B45">2004</xref>; Luisi, <xref ref-type="bibr" rid="B27">2006</xref>).</p>
<p>The basic constituent of sensorimotor autonomy is a sensorimotor structure (a behavioral scheme or habit) made possible by both a set embodied-neural pathways and a set of sensorimotor contingency relationships. Think of it as a coordination pattern that emerges out of environmental, sensorimotor and neural (or behavior generating) mechanisms. Now, if this structure is far-from-equilibrium or, said differently, if let alone it tends to extinguish or vanish, and if the very enactment of the sensorimotor scheme reinforces itself by repetition or by satisfying certain conditions that feed-back into its supporting structure (e.g., reinforcement of synaptic connections by Hebbian learning or reward reinforcement), then we have first sense of self-maintenance that is characteristic of habits. The more the habit is enacted the more it is strenghthened, the stronger it is the more likely it is to be enacted. That is the virtuous (or vicious!) self-sustaining nature of the habit.</p>
<p>We can now go back to the original, albeit obscure and self-referential, intuition of roboticist Tim Smithers and his idea of the autonomy of behavior based on the &#x0201C;processes that, through interaction, are continuously forming the laws of interaction that can sustain and maintain the interaction needed to form them.&#x0201D; These laws or norms are nothing other than the very conditions under which the habit is viable, that it can persist and sustain itself. This is a strong analogy with (metabolic) life that opens up the very possibility of <italic>having a problem of your own</italic> and <italic>having to solve it</italic>. The problem for the precarious, self-maintaining autonomous cell is persistence, avoiding decay and disintegration (see Barandiaran and Egbert, <xref ref-type="bibr" rid="B7">2013</xref> for a more detailed analysis). The same goes for the precarious, self-maintaining nature of a habit, and ultimately of autonomous sensorimotor life (see Barandiaran, <xref ref-type="bibr" rid="B5">2008</xref>; Di Paolo et al., <xref ref-type="bibr" rid="B15">2017</xref>). This approach also opens up a new mode of explanation that is characteristic of biological thinking and can be applied to sensorimotor dynamics: focusing on the nature and structure of constraints rather than the problems they are suppose to be adapted to solve. We return to these themes in the discussion section.</p>
<p>In what follows we introduce habit-based enactive robotics to illustrate and further develop the points we have briefly outlined. Inspired by SMC and enactive principles we build robots that are capable of generating spontaneously a complete ecology of habits that display structural constraints within the sensorimotor space. The results will help us discuss how enactive robotics can contribute to a new understanding of mind and cognition with a deeper biological inspiration than what problem-solving can provide.</p></sec></sec>
<sec id="s3">
<title>3. Model</title>
<sec>
<title>3.1. Overview</title>
<p>The computational model simulates a two-wheeled robot that moves around a two-dimensional environment. The robot has two independently controlled motors; one for each of its wheels, allowing it to move forwards or backwards in a straight line or to turn in a variety of arcs, or on the spot. The robot&#x00027;s motors are controlled by an iterant deformable sensorimotor medium (IDSM) (Egbert and Barandiaran, <xref ref-type="bibr" rid="B16">2014</xref>)&#x02014;a habit-forming controller that is described in detail below.</p>
<p>The robot&#x00027;s environment is periodic in the sense that when the robot moves off one side of the environment, it appears on the opposite side. A stimulus source is also located in the environment. This source, which moves around the environment, can be thought of as simultaneously emitting a sound tone and a source of light, but in any given trial the robot is only capable of perceiving one of those sensory modalities (light or sound). We now present each element of the simulation in detail.</p>
<p>The robot and its environment are simulated using Euler forward integration with a time step of &#x00394;<italic>t</italic> &#x0003D; 0.01. Thus, a single time-unit consists of 100 iterations, and the IDSM is updated every iteration.</p></sec>
<sec>
<title>3.2. Stimulus</title>
<p>The stimulus source moves around the environment in a circle. Each rotation, it slows to a stop at its left-most position, before accelerating again to complete another rotation. This trajectory is specified by the following equations which describe the stimulus&#x00027;s position (<italic>s</italic><sub><italic>x</italic></sub>, <italic>s</italic><sub><italic>y</italic></sub>) as a function of time (<italic>t</italic>):
<disp-formula id="E1"><label>(1)</label><mml:math id="M1"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>x</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:mfrac><mml:mo>&#x0002B;</mml:mo><mml:mo class="qopname">cos</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mn>10</mml:mn></mml:mrow></mml:mfrac><mml:mo>&#x0002B;</mml:mo><mml:mo class="qopname">sin</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mn>10</mml:mn></mml:mrow></mml:mfrac></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E2"><label>(2)</label><mml:math id="M2"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>y</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:mfrac><mml:mo>&#x0002B;</mml:mo><mml:mo class="qopname">sin</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mn>10</mml:mn></mml:mrow></mml:mfrac><mml:mo>&#x0002B;</mml:mo><mml:mo class="qopname">sin</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mn>10</mml:mn></mml:mrow></mml:mfrac></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p></sec>
<sec>
<title>3.3. Robot</title>
<p>The simulated robot (<xref ref-type="fig" rid="F1">Figure 1</xref>) has two independently controlled motorized wheels. It&#x00027;s position changes according to the following differential equations,
<disp-formula id="E3"><label>(3)</label><mml:math id="M3"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mfrac><mml:mrow><mml:mtext>d</mml:mtext><mml:mi>x</mml:mi></mml:mrow><mml:mrow><mml:mtext>d</mml:mtext><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:mo>=</mml:mo><mml:msub><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>s</mml:mi></mml:mrow></mml:msub><mml:mo class="qopname">cos</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>&#x003B1;</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>m</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi></mml:mrow></mml:msub><mml:mo>&#x0002B;</mml:mo><mml:msub><mml:mrow><mml:mi>m</mml:mi></mml:mrow><mml:mrow><mml:mi>r</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E4"><label>(4)</label><mml:math id="M4"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mfrac><mml:mrow><mml:mtext>d</mml:mtext><mml:mi>y</mml:mi></mml:mrow><mml:mrow><mml:mtext>d</mml:mtext><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:mo>=</mml:mo><mml:msub><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>s</mml:mi></mml:mrow></mml:msub><mml:mo class="qopname">sin</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>&#x003B1;</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>m</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi></mml:mrow></mml:msub><mml:mo>&#x0002B;</mml:mo><mml:msub><mml:mrow><mml:mi>m</mml:mi></mml:mrow><mml:mrow><mml:mi>r</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E5"><label>(5)</label><mml:math id="M5"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mfrac><mml:mrow><mml:mtext>d</mml:mtext><mml:mi>&#x003B1;</mml:mi></mml:mrow><mml:mrow><mml:mtext>d</mml:mtext><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:mo>=</mml:mo><mml:msub><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>s</mml:mi></mml:mrow></mml:msub><mml:mfrac><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>m</mml:mi></mml:mrow><mml:mrow><mml:mi>r</mml:mi></mml:mrow></mml:msub><mml:mo>-</mml:mo><mml:msub><mml:mrow><mml:mi>m</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:mn>2</mml:mn><mml:mi>R</mml:mi></mml:mrow></mml:mfrac><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
where <italic>x</italic> and <italic>y</italic> are the robot&#x00027;s position in the environment; &#x003B1; is its heading; <italic>k</italic><sub><italic>s</italic></sub> &#x0003D; 0.25 scales the speed of the motors; <italic>R</italic> &#x0003D; 0.05 is the robot&#x00027;s radius; and the variables <italic>m</italic><sub><italic>l</italic></sub> and <italic>m</italic><sub><italic>r</italic></sub> represent the velocities of the robot&#x00027;s left and right wheel motors.</p>
<fig id="F1" position="float">
<label>Figure 1</label>
<caption><p>The simulated two-wheeled robot. The variable &#x003B1; specifies the orientation of the robot (direction of forward travel). The robot&#x00027;s sensors are located on its periphery with the parameter &#x003B2; specifying the offset of the sensors from the direction the robot is facing.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0001.tif"/>
</fig>
<p>We consider two robot sensor configurations. &#x0201C;Visual&#x0201D; robots have two directional sensors. The excitation of each, <italic>V</italic>, is the product of an attenuation factor due to distance from that sensor to the stimulus, and an attenuation factor due to misalignment between the orientation of the sensor and the relative direction of the stimulus. This second attenuation factor is calculated by taking the scalar product of a unit vector that points from the sensor to the stimulus and <inline-formula><mml:math id="M6"><mml:mover accent="true"><mml:mrow><mml:mi>o</mml:mi></mml:mrow><mml:mo>&#x02192;</mml:mo></mml:mover></mml:math></inline-formula>, a unit vector that specifies the direction that the sensor is facing.</p>
<p>Formally,
<disp-formula id="E6"><label>(6)</label><mml:math id="M7"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>V</mml:mi><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:munder class="msub"><mml:mrow><mml:mstyle displaystyle="true"><mml:munder accentunder="false"><mml:mrow><mml:mfrac><mml:mrow><mml:mn>0</mml:mn><mml:mo>.</mml:mo><mml:mn>25</mml:mn></mml:mrow><mml:mrow><mml:mn>0</mml:mn><mml:mo>.</mml:mo><mml:mn>25</mml:mn><mml:mo>&#x0002B;</mml:mo><mml:mo>&#x02225;</mml:mo><mml:mover accent="true"><mml:mrow><mml:mi>r</mml:mi></mml:mrow><mml:mo>&#x02192;</mml:mo></mml:mover><mml:msup><mml:mrow><mml:mo>&#x02225;</mml:mo></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msup></mml:mrow></mml:mfrac></mml:mrow><mml:mo>&#x0FE38;</mml:mo></mml:munder></mml:mstyle></mml:mrow><mml:mrow><mml:mtext class="textrm" mathvariant="normal">distance</mml:mtext></mml:mrow></mml:munder></mml:mstyle><mml:mstyle displaystyle="true"><mml:munder class="msub"><mml:mrow><mml:mstyle displaystyle="true"><mml:munder accentunder="false"><mml:mrow><mml:msup><mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mover accent="true"><mml:mrow><mml:mi>r</mml:mi></mml:mrow><mml:mo>&#x02192;</mml:mo></mml:mover></mml:mrow><mml:mrow><mml:mo>&#x02225;</mml:mo><mml:mover accent="true"><mml:mrow><mml:mi>r</mml:mi></mml:mrow><mml:mo>&#x02192;</mml:mo></mml:mover><mml:mo>&#x02225;</mml:mo></mml:mrow></mml:mfrac><mml:mo>&#x000B7;</mml:mo><mml:mover accent="true"><mml:mrow><mml:mi>o</mml:mi></mml:mrow><mml:mo>&#x02192;</mml:mo></mml:mover></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msup></mml:mrow><mml:mo>&#x0FE38;</mml:mo></mml:munder></mml:mstyle></mml:mrow><mml:mrow><mml:mtext class="textrm" mathvariant="normal">orientation</mml:mtext></mml:mrow></mml:munder></mml:mstyle><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
where <inline-formula><mml:math id="M8"><mml:mover accent="true"><mml:mrow><mml:mi>r</mml:mi></mml:mrow><mml:mo>&#x02192;</mml:mo></mml:mover></mml:math></inline-formula> is a vector that describes the position of the stimulus relative to the sensor; &#x02225;<italic>r</italic>&#x02225; is the magnitude of that vector; and the &#x0002B; superscript indicates that negative values within the parentheses are truncated to zero. The excitation of these sensors is thus highest when the sensor is close to the stimulus and directly facing it.</p>
<p>&#x0201C;Auditory&#x0201D; robots have two sensors that respond to the rate at which the sensor is approaching or moving away from the stimulus source. This is analogous to the Doppler effect whereby the perceived frequency of a sound when approaching is higher than when moving away from the listener. The excitation of an auditory sensor, <italic>A</italic>, is given by:
<disp-formula id="E7"><label>(7)</label><mml:math id="M9"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>A</mml:mi><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:mfrac><mml:mo>&#x0002B;</mml:mo><mml:mi>k</mml:mi><mml:mfrac><mml:mrow><mml:mtext>d</mml:mtext><mml:mo>|</mml:mo><mml:mo>|</mml:mo><mml:mi>r</mml:mi><mml:mo>|</mml:mo><mml:mo>|</mml:mo></mml:mrow><mml:mrow><mml:mtext>d</mml:mtext><mml:mi>t</mml:mi></mml:mrow></mml:mfrac></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
where the first term can be thought of as the tones natural pitch which is offset by the relative speed of the sensor and the stimulus scaled by <inline-formula><mml:math id="M10"><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mn>3</mml:mn></mml:mrow><mml:mrow><mml:mn>4</mml:mn></mml:mrow></mml:mfrac></mml:math></inline-formula> to keep the magnitude of the sensor (given the relative speeds of the robot and the stimulus) within a similar range of excitation values as simulated for the visual sensor.</p>
<p>To address the periodic boundaries of the environment, auditory sensors always use the nearest stimulus source as defined by the minimum image convention and visual sensors calculate the combined effect of 5 stimuli: one in the simulated space and four virtual copies of the sensor offset north, south, west and east of the simulated space arena by one arena width. This means that if, for example, a visual robot is close to the north boundary of the arena and facing north it can still see stimuli.</p>
<p>Visual sensors are offset from the direction the robot is facing by &#x003B2;<sub><italic>v</italic></sub> &#x0003D; &#x003C0;/5 (see <xref ref-type="fig" rid="F1">Figure 1</xref>). Auditory sensors are offset by &#x003B2;<sub><italic>a</italic></sub> &#x0003D; &#x003C0;/2. The orientation of visual sensors is &#x003B1; &#x000B1; &#x003B2;<sub><italic>v</italic></sub>, i.e., perpendicular to the tangent of the robot&#x00027;s circular body at that position, facing outwards. Auditory sensors have no orientation.</p></sec>
<sec>
<title>3.4. IDSM</title>
<sec>
<title>3.4.1. Overview</title>
<p>The IDSM is a robot controller intended to capture the idea of a self-maintaining pattern of sensorimotor activity. Inspired by the habitual behavior of people and by the enactivist concept of &#x0201C;<italic>autonomous&#x0201D;</italic> self-sustaining sensorimotor systems (see e.g., Di Paolo et al., <xref ref-type="bibr" rid="B15">2017</xref>), the IDSM was designed such that patterns of sensorimotor activity reinforce the mechanisms that produce them (Egbert and Barandiaran, <xref ref-type="bibr" rid="B16">2014</xref>; Egbert and Ca&#x000F1;amero, <xref ref-type="bibr" rid="B18">2014</xref>; Egbert, <xref ref-type="bibr" rid="B19">2018</xref>). The IDSM has been used to explore how a habit-based individual can be trained to perform different tasks (Egbert and Barandiaran, <xref ref-type="bibr" rid="B16">2014</xref>); how different forms of motor babbling can bias the subsequent formation of habits (Zarco and Egbert, <xref ref-type="bibr" rid="B56">2019</xref>); how the essential variables of a biological autonomous system can be shared with the essential variables of a sensorimotor autonomous system (Egbert and Ca&#x000F1;amero, <xref ref-type="bibr" rid="B18">2014</xref>) and the extent to which IDSM-based sensorimotor autonomous systems can be considered to be adaptive (Egbert, <xref ref-type="bibr" rid="B19">2018</xref>). In the present paper, we use the IDSM in a new way: to show, in a formal model, how sensorimotor-contingencies can play an essential role of sculpting the form of habits without themselves being explicitly internalized or represented by the &#x0201C;brain&#x0201D; or &#x0201C;controller&#x0201D; of an embodied agent.</p>
<p>The IDSM works by recording trajectories taken through sensorimotor space, i.e., how the sensorimotor state changes for various experienced sensorimotor states. When a sensorimotor state is experienced that is similar to one that has been experienced in the past, the motors of the robot are actuated in a way similar to how they were actuated in that previous experience. Memories of previous trajectories are gradually forgotten, unless they are reinforced by re-enactment and so the only patterns of behavior that can persist for long periods of time are those that are re-enacted. When the sensorimotor state is in an unfamiliar (or forgotten) state, motor activity is random. The IDSM used in this paper is very similar to that described in Egbert (<xref ref-type="bibr" rid="B19">2018</xref>). Any differences between the model here and that in Egbert (<xref ref-type="bibr" rid="B19">2018</xref>) are explicitly highlighted below.</p>
<p>A useful metaphor for understanding how the IDSM works is the paths that form on university campuses, where paths taken by students crossing a grassy field between academic buildings trample and kill the grass. The emergent dirt paths influence the trajectories taken by subsequent students, but the grass also regrows, so only emergent paths that are regularly traveled can persist in the long-term. This is essentially how the IDSM operates, but the trajectories taken and the paths that form are in sensorimotor space, rather than on a university campus. The dynamic also relates to the self-reinforcing nature of habitual behavior, where repeated performance of patterns of behavior (e.g., the direction you look when crossing the street; smoking a cigarette; or a tendency to worry) increases the likelihood of similar behavior being performed in the future. And to reiterate: the IDSM was designed to capture the enactivist concept of autonomy (a precarious self-maintaining system) in a sensorimotor system&#x02014;see Di Paolo et al. (<xref ref-type="bibr" rid="B15">2017</xref>).</p>
<p>More formally, the IDSM can be thought of as a function, <italic>f</italic>, that transforms the robot&#x00027;s current sensorimotor state into an &#x0201C;output,&#x0201D; i.e., the next moment&#x00027;s motor state: <italic>f</italic><sub><italic>t</italic></sub>(<italic>S</italic><sub><italic>t</italic></sub>, <italic>M</italic><sub><italic>t</italic></sub>) &#x02192; <italic>M</italic><sub><italic>t</italic>&#x0002B;1</sub>. As this function is applied, the function itself also changes as a function of the current state of sensors and motors and the current state of the function: <inline-formula><mml:math id="M11"><mml:mfrac><mml:mrow><mml:mo>&#x00394;</mml:mo><mml:mstyle class="text"><mml:mtext class="textrm" mathvariant="normal">f</mml:mtext></mml:mstyle></mml:mrow><mml:mrow><mml:mo>&#x00394;</mml:mo><mml:mstyle class="text"><mml:mtext class="textrm" mathvariant="normal">t</mml:mtext></mml:mstyle></mml:mrow></mml:mfrac><mml:mo>=</mml:mo><mml:mi>g</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>f</mml:mi><mml:mo>,</mml:mo><mml:mi>S</mml:mi><mml:mo>,</mml:mo><mml:mi>M</mml:mi><mml:mo>,</mml:mo><mml:mfrac><mml:mrow><mml:mo>&#x00394;</mml:mo><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x00394;</mml:mo><mml:mstyle class="text"><mml:mtext class="textrm" mathvariant="normal">t</mml:mtext></mml:mstyle></mml:mrow></mml:mfrac><mml:mo>,</mml:mo><mml:mfrac><mml:mrow><mml:mo>&#x00394;</mml:mo><mml:mi>M</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x00394;</mml:mo><mml:mstyle class="text"><mml:mtext class="textrm" mathvariant="normal">t</mml:mtext></mml:mstyle></mml:mrow></mml:mfrac></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:math></inline-formula>. This change, which we shall now describe, was engineered so that sensorimotor state trajectories would bias the system to increase the likelihood that similar sensorimotor trajectories will be repeated in the future.</p></sec>
<sec>
<title>3.4.2. Tracking sensorimotor trajectories</title>
<p>As the robot&#x00027;s sensorimotor state changes, the IDSM maintains a set of records called &#x0201C;<italic>nodes.&#x0201D;</italic> Each node describes the sensorimotor-velocity (i.e., the rate of change in all sensors and motors) for a particular sensorimotor-state at the moment that the node was created. Each node, <italic>N</italic>, is a tuple, <italic>N</italic> &#x0003D; &#x02329;<bold><italic>p</italic></bold>, <bold><italic>v</italic></bold>, <italic>w</italic>&#x0232A;, where <bold><italic>p</italic></bold> represents the sensorimotor state associated with the node (referred to as the node&#x00027;s &#x02018;position in sensorimotor space&#x02019;); <bold><italic>v</italic></bold> indicates the sensorimotor velocity when the node was created; and <italic>w</italic> indicates the weight of the node, a value that changes dynamically and is used to scale the overall influence of the node. We shall refer to these components using a subscript notation, where the position, SM-velocity vector, and weight of node <italic>N</italic> are written as <italic>N</italic><sub><bold><italic>p</italic></bold></sub> and <italic>N</italic><sub><bold><italic>v</italic></bold></sub> and <italic>N</italic><sub><italic>w</italic></sub>, respectively.</p>
<p>As a robot controlled by the IDSM moves through sensorimotor states, new nodes are created recording the sensorimotor velocities experienced at different sensorimotor states. Specifically, when a new node is created, its <italic>N</italic><sub><bold><italic>p</italic></bold></sub> is set to the current sensorimotor state; its <italic>N</italic><sub><bold><italic>v</italic></bold></sub> is set to the current rate of change in each sensorimotor dimension, and its initial weight, <inline-formula><mml:math id="M12"><mml:msubsup><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn></mml:mrow></mml:msubsup></mml:math></inline-formula>.</p>
<p>The two vector terms (<italic>N</italic><sub><bold><italic>p</italic></bold></sub> and <italic>N</italic><sub><bold><italic>v</italic></bold></sub>) can be thought of as existing within an abstract sensorimotor space (ASMS). A many to one mapping transforms any given position in the ASMS to a specific sensorimotor state, according to:
<disp-formula id="E8"><label>(8)</label><mml:math id="M13"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mrow><mml:mo>[</mml:mo><mml:mrow><mml:mtable style="text-align:axis;" equalrows="false" columnlines="none none none none none none none none none" equalcolumns="false" class="array"><mml:mtr><mml:mtd><mml:mo>&#x0039B;</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>&#x003BC;</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mo>&#x0039B;</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>&#x003BC;</mml:mi></mml:mrow><mml:mrow><mml:mi>r</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>&#x003C3;</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi></mml:mrow></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>&#x003C3;</mml:mi></mml:mrow><mml:mrow><mml:mi>r</mml:mi></mml:mrow></mml:msub></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>]</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:mrow><mml:mtable style="text-align:axis;" equalrows="false" columnlines="none none none none none none none none none" equalcolumns="false" class="array"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>m</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi></mml:mrow></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>m</mml:mi></mml:mrow><mml:mrow><mml:mi>r</mml:mi></mml:mrow></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi></mml:mrow></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>r</mml:mi></mml:mrow></mml:msub></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>]</mml:mo></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
where &#x003BC;<sub><italic>l</italic></sub>, &#x003BC;<sub><italic>r</italic></sub>, &#x003C3;<sub><italic>l</italic></sub>, and &#x003C3;<sub><italic>r</italic></sub> indicate a position in the four dimensional ASMS; the vector on the right indicates the sensorimotor state (i.e., the actual state of left-motor, right-motor, left-sensor, and right-sensor) associated with that ASMS position; and &#x0039B;(<italic>x</italic>) &#x0003D; sin[(4&#x0002B;<italic>x</italic>)2&#x003C0;<italic>x</italic><sup>3</sup>] is the non-linear function plotted in <xref ref-type="fig" rid="F2">Figure 2</xref>. The purpose of this mapping is to avoid prescribing a characteristic rate of motor change, and instead to allow the IDSM to autonomously find habits with rates of motor change that are neither too fast nor too slow. Different regions of ASMS correspond to different rates of motor change. For instance, when abstract motor state <italic>x</italic> &#x02A85; 0.5, a small change in that state variable corresponds to a small change in &#x0039B;(<italic>x</italic>), the actual motor state (see the difference between the red circle and red X in <xref ref-type="fig" rid="F2">Figure 2</xref>), where an equivalent change in <italic>x</italic> when <italic>x</italic> &#x02A86; 0.7 corresponds to a greater change in the actual motor state (blue circle and X in <xref ref-type="fig" rid="F2">Figure 2</xref>). When the sensorimotor state of the robot is unfamiliar, changes in motor activity are driven randomly (as explained below) and this &#x0039B; mapping allows these random changes in sensorimotor space to correspond to slow or fast changes in motor state. By exploring different parts of the abstract sensorimotor state, the IDSM can experiment with sensorimotor patterns with different rates of change until ones that are self-reinforcing emerge. The ASMS is also treated as periodic so as to avoid the IDSM getting stuck at motor boundaries (as discussed in Egbert, <xref ref-type="bibr" rid="B19">2018</xref>). ASMS state variables, &#x003BC;<sub><italic>l</italic></sub>, &#x003BC;<sub><italic>r</italic></sub>, &#x003C3;<sub><italic>l</italic></sub>, &#x003C3;<sub><italic>r</italic></sub> all &#x02208; [0, 1] and the ASMS distance functions (described below) adhere to the minimum-image convention.</p>
<fig id="F2" position="float">
<label>Figure 2</label>
<caption><p>Abstract motor-state to motor-state mapping. Changes made to low abstract motor state [e.g., red circle and (X)] produce less change in actual motor state (&#x0039B;(<italic>x</italic>)) than when the abstract motor state is high (e.g., blue circle and X).</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0002.tif"/>
</fig>
<p>New nodes are added when the density of nodes near the current sensorimotor state is less than a threshold value, i.e., when &#x003C8;(<bold><italic>x</italic></bold>) &#x0003C; <italic>k</italic><sub><italic>t</italic></sub>. Loosely speaking, &#x003C8; is a measure of how &#x02018;familiar&#x02019; the current sensorimotor state is, as estimated by summing a non-linear function of the distance (Equation 11) from every node with a positive weight to the current sensorimotor state. Formally,
<disp-formula id="E9"><label>(9)</label><mml:math id="M14"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>&#x003C8;</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:munder><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munder></mml:mstyle><mml:mi>&#x003C9;</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mi>w</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mi>d</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>p</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mo>,</mml:mo><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E10"><label>(10)</label><mml:math id="M15"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>&#x003C9;</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mi>w</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:mrow><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mtext>if&#x000A0;</mml:mtext><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mi>w</mml:mi></mml:mrow></mml:msub><mml:mo>&#x0003E;</mml:mo><mml:mn>0</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>0</mml:mn></mml:mtd><mml:mtd><mml:mtext class="textrm" mathvariant="normal">otherwise</mml:mtext></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>where <bold><italic>x</italic></bold> represents the current ASMS position, <italic>k</italic><sub><italic>t</italic></sub> is a threshold parameter describing maximum node-density at which new nodes will be created and <italic>d</italic>() is the following non-linear ASMS-distance function.</p>
<disp-formula id="E11"><label>(11)</label><mml:math id="M16"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>d</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>p</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mo>,</mml:mo><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mfrac><mml:mrow><mml:mn>2</mml:mn></mml:mrow><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x0002B;</mml:mo><mml:mtext>exp</mml:mtext><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>d</mml:mi></mml:mrow></mml:msub><mml:mo>|</mml:mo><mml:mo>|</mml:mo><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>p</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mo>-</mml:mo><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle><mml:mo>|</mml:mo><mml:msup><mml:mrow><mml:mo>|</mml:mo></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:mfrac></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;<inline-graphic xlink:href="fnbot-16-847054-i0001.tif"/></p></sec>
<sec>
<title>3.4.3. Nodes influence the sensorimotor state</title>
<p>One time unit after a node has been created, it is added to the set of &#x02018;activated&#x02019; nodes that influence the sensorimotor state according to:
<disp-formula id="E13"><label>(12)</label><mml:math id="M18"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle mathvariant="bold-italic"><mml:mi>m</mml:mi></mml:mstyle><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>&#x003D5;</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:mfrac><mml:mstyle displaystyle="true"><mml:munder class="msub"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munder></mml:mstyle><mml:mi>&#x003C9;</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mi>w</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mi>d</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>p</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mo>,</mml:mo><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:msup><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mstyle displaystyle="true"><mml:munder class="msub"><mml:mrow><mml:mstyle displaystyle="true"><mml:munder accentunder="false"><mml:mrow><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>v</mml:mi></mml:mstyle></mml:mrow></mml:msub></mml:mrow><mml:mo>&#x0FE38;</mml:mo></mml:munder></mml:mstyle></mml:mrow><mml:mrow><mml:mtext class="textrm" mathvariant="normal">vel.</mml:mtext></mml:mrow></mml:munder></mml:mstyle><mml:mo>&#x0002B;</mml:mo><mml:mstyle displaystyle="true"><mml:munder class="msub"><mml:mrow><mml:mstyle displaystyle="true"><mml:munder accentunder="false"><mml:mrow><mml:mi>A</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>p</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mo>-</mml:mo><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle><mml:mo>,</mml:mo><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>v</mml:mi></mml:mstyle></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow><mml:mo>&#x0FE38;</mml:mo></mml:munder></mml:mstyle></mml:mrow><mml:mrow><mml:mtext class="textrm" mathvariant="normal">attraction</mml:mtext></mml:mrow></mml:munder></mml:mstyle></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>&#x003BC;</mml:mi></mml:mstyle></mml:mrow></mml:msup></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
This equation describes a weighted average of the influence of all of the nodes. The influence of each individual node is the sum of its &#x0201C;velocity&#x0201D; factor and its &#x0201C;attraction&#x0201D; factor. The velocity factor is simply the <italic>N</italic><sub><bold><italic>v</italic></bold></sub> vector (i.e., the sensorimotor velocity recorded when the node was created). The attraction factor is defined by
<disp-formula id="E14"><label>(13)</label><mml:math id="M19"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>A</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>a</mml:mi></mml:mstyle><mml:mo>,</mml:mo><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>v</mml:mi></mml:mstyle></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mstyle mathvariant="bold-italic"><mml:mi>a</mml:mi></mml:mstyle><mml:mo>-</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>a</mml:mi></mml:mstyle><mml:mo>&#x000B7;</mml:mo><mml:mfrac><mml:mrow><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>v</mml:mi></mml:mstyle></mml:mrow></mml:msub></mml:mrow><mml:mrow><mml:mo>|</mml:mo><mml:mo>|</mml:mo><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>v</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mo>|</mml:mo><mml:mo>|</mml:mo></mml:mrow></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mfrac><mml:mrow><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>v</mml:mi></mml:mstyle></mml:mrow></mml:msub></mml:mrow><mml:mrow><mml:mo>|</mml:mo><mml:mo>|</mml:mo><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>v</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mo>|</mml:mo><mml:mo>|</mml:mo></mml:mrow></mml:mfrac></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
and it causes the sensorimotor state to move toward the node. The attraction term is included to cause the system to move toward more familiar regions of sensorimotor space and to help stabilize patterns of repeated behavior (see Egbert and Barandiaran, <xref ref-type="bibr" rid="B16">2014</xref>, <xref ref-type="bibr" rid="B17">2015</xref>). The &#x003BC; superscript in Equation (12) expresses that the IDSM only (directly) controls the motor components of the sensorimotor state. The sensory components are the result of the robot&#x00027;s relation to its environment and so are not directly controlled by the IDSM, but are, of course, <italic>influenced</italic> by the motor dynamics, indirectly through the sensorimotor contingencies determined by the robot&#x00027;s environment and body.</p>
<p>The influence of each node is attenuated by a non-linear function of the distance between the node and the current sensorimotor state. This attenuation is expressed by the term <italic>d</italic>(<italic>N</italic><sub><bold><italic>p</italic></bold></sub>, <bold><italic>x</italic></bold>) and it means that nearby nodes affect the sensorimotor state and farther away nodes have little influence. The influence is also attenuated by a threshold function of weight, [&#x003C9;(<italic>N</italic><sub><italic>w</italic></sub>)] such that only positively weighted nodes affect the motor state. Previous versions of the IDSM had a more complicated sigmoidal function in place of the simpler threshold function used here (Equation 10). Note that the degradation of the nodes and the threshold function of Equation (10) mean that when nodes are not visited for a long period of time they cease to have any influence whatsoever. Nodes that have degraded to this point essentially cease to exist.</p>
<p>After a node is created, its weight changes according to:
<disp-formula id="E15"><label>(14)</label><mml:math id="M20"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mfrac><mml:mrow><mml:mtext>d</mml:mtext><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mi>w</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mrow><mml:mtext>d</mml:mtext><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:mo>=</mml:mo><mml:mo>-</mml:mo><mml:msub><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x02193;</mml:mi></mml:mrow></mml:msub><mml:mo>&#x0002B;</mml:mo><mml:msub><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x02191;</mml:mi></mml:mrow></mml:msub><mml:mi>d</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>p</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mo>,</mml:mo><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
In this equation, the first term represents a steady degradation of the node&#x00027;s influence and the second term represents a strengthening of the node that occurs when the current sensorimotor state is close to the node&#x00027;s position.</p>
<p>The influence of all nodes is summed and then scaled by the local density of nodes,
<disp-formula id="E16"><label>(15)</label><mml:math id="M21"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>&#x003D5;</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:munder class="msub"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munder></mml:mstyle><mml:mi>&#x003C9;</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mi>w</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mi>d</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>p</mml:mi></mml:mstyle></mml:mrow></mml:msub><mml:mo>,</mml:mo><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:msub><mml:mrow><mml:mo>|</mml:mo></mml:mrow><mml:mrow><mml:mtext class="textrm" mathvariant="normal">activated nodes</mml:mtext></mml:mrow></mml:msub></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
This equation looks similar to that used to calculate &#x003C8; (Equation 9), but is different in that &#x003C8; describes the local density of all nodes, where as &#x003D5; describes the local density of activated nodes <italic>only</italic>.</p></sec>
<sec>
<title>3.4.4. Random motor activity</title>
<p>When the local density of activated nodes is low, motor behavior is random. This is accomplished by defining of a &#x0201C;switch&#x0201D; value, <italic>s</italic>, which specifies when the behavior is to be random and when it is to be controlled by the influence of the IDSM&#x00027;s nodes. The following equation expresses that <italic>s</italic> is 1 when &#x003D5; is low and 0 when &#x003D5; is high; and that it moves between these values in a smooth, sigmoidal manner:</p>
<disp-formula id="E17"><label>(16)</label><mml:math id="M22"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>s</mml:mi><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mfrac><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x0002B;</mml:mo><mml:mtext class="textrm" mathvariant="normal">exp</mml:mtext><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>R</mml:mi></mml:mrow><mml:mrow><mml:mi>g</mml:mi></mml:mrow></mml:msub><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>&#x003D5;</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>x</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo><mml:msub><mml:mrow><mml:mi>R</mml:mi></mml:mrow><mml:mrow><mml:mi>t</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:mfrac></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;<inline-graphic xlink:href="fnbot-16-847054-i0002.tif"/></p>
<p>Here, <italic>R</italic><sub><italic>g</italic></sub> and <italic>R</italic><sub><italic>t</italic></sub> are parameters that specify the threshold of familiarity and the discreteness of the transition between random and non-random motor activity. This value is then used to switch between random motor activity (<bold><italic>r</italic></bold>), and IDSM&#x00027;s output (<bold><italic>m</italic></bold>), thus:
<disp-formula id="E19"><label>(17)</label><mml:math id="M24"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mfrac><mml:mrow><mml:mtext>d</mml:mtext><mml:mstyle mathvariant="bold-italic"><mml:mi>&#x003BC;</mml:mi></mml:mstyle></mml:mrow><mml:mrow><mml:mtext>d</mml:mtext><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:mo>=</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>-</mml:mo><mml:mi>s</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mstyle mathvariant="bold-italic"><mml:mi>m</mml:mi></mml:mstyle><mml:mo>&#x0002B;</mml:mo><mml:mi>s</mml:mi><mml:mstyle mathvariant="bold-italic"><mml:mi>r</mml:mi></mml:mstyle><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
where the random motor activity vector, <bold><italic>r</italic></bold> is varied over time to produce a random walk in motor space as follows: every iteration, there is a <italic>R</italic><sub><italic>p</italic></sub> chance that the components of <bold><italic>r</italic></bold> will be assigned random values selected from a normal distribution with a standard deviation of <italic>R</italic><sub>&#x003C3;</sub>. The value of the <italic>R</italic><sub>&#x003C3;</sub> and other parameters can be found in <xref ref-type="table" rid="T1">Table 1</xref>.</p>
<table-wrap position="float" id="T1">
<label>Table 1</label>
<caption><p>Parameters.</p></caption>
<table frame="box" rules="all">
<thead><tr style="background-color:#919498;color:#ffffff">
<th valign="top" align="left"><bold>Parameter</bold></th>
<th valign="top" align="left"><bold>Value</bold></th>
<th valign="top" align="left"><bold>Description</bold></th>
</tr>
</thead>
<tbody>
<tr style="background-color:#e0e1e2">
<td valign="top" align="left" colspan="3"><bold>ROBOT</bold></td>
</tr> <tr>
<td valign="top" align="left"><italic>R</italic><sub><italic>s</italic></sub></td>
<td valign="top" align="left">0.25</td>
<td valign="top" align="left">Robot speed</td>
</tr> <tr>
<td valign="top" align="left"><italic>R</italic><sub><italic>d</italic></sub></td>
<td valign="top" align="left">0.1</td>
<td valign="top" align="left">Robot diameter</td>
</tr> <tr>
<td valign="top" align="left"><italic>R</italic><sub><italic>o</italic></sub></td>
<td valign="top" align="left">&#x003C0;/5</td>
<td valign="top" align="left">Offset of sensors from center line of robot</td>
</tr> <tr>
<td valign="top" align="left"><italic>R</italic><sub><italic>f</italic></sub></td>
<td valign="top" align="left">0.25</td>
<td valign="top" align="left">Falloff of sensor excitation with distance</td>
</tr> <tr style="background-color:#e0e1e2">
<td valign="top" align="left" colspan="3"><bold>LIGHT</bold></td>
</tr> <tr>
<td valign="top" align="left"><italic>L</italic><sub><italic>v</italic></sub></td>
<td valign="top" align="left">0.1</td>
<td valign="top" align="left">Light velocity scale constant</td>
</tr> <tr>
<td valign="top" align="left"><italic>L</italic><sub><italic>r</italic></sub></td>
<td valign="top" align="left">1/3</td>
<td valign="top" align="left">The radius of the light&#x00027;s circling motion</td>
</tr> <tr style="background-color:#e0e1e2">
<td valign="top" align="left" colspan="3"><bold>IDSM</bold></td>
</tr> <tr>
<td valign="top" align="left"><italic>k</italic><sub><italic>d</italic></sub></td>
<td valign="top" align="left">500</td>
<td valign="top" align="left">Fall-off of non-linear ASMS distance measure</td>
</tr> <tr>
<td valign="top" align="left"><italic>k</italic><sub>&#x02193;</sub></td>
<td valign="top" align="left">5</td>
<td valign="top" align="left">Node weight decay rate</td>
</tr> <tr>
<td valign="top" align="left"><italic>k</italic><sub>&#x02191;</sub></td>
<td valign="top" align="left">2000</td>
<td valign="top" align="left">Node-reinforcement rate constant</td>
</tr> <tr>
<td valign="top" align="left"><italic>k</italic><sub><italic>t</italic></sub></td>
<td valign="top" align="left">1</td>
<td valign="top" align="left">Node-density threshold parameter, influencing when new nodes are to be added</td>
</tr> <tr>
<td valign="top" align="left"><inline-formula><mml:math id="M25"><mml:msubsup><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn></mml:mrow></mml:msubsup></mml:math></inline-formula></td>
<td valign="top" align="left">1000</td>
<td valign="top" align="left">Node weight upon creation</td>
</tr> <tr style="background-color:#e0e1e2">
<td valign="top" align="left" colspan="3"><bold>MOTOR BABBLING</bold></td>
</tr> <tr>
<td valign="top" align="left"><italic>R</italic><sub><italic>p</italic></sub></td>
<td valign="top" align="left">0.1</td>
<td valign="top" align="left">Chance of random reassignment of random-motor walk direction and velocity</td>
</tr> <tr>
<td valign="top" align="left"><italic>R</italic><sub>&#x003C3;</sub></td>
<td valign="top" align="left">0.25</td>
<td valign="top" align="left">Standard deviation of random motor components</td>
</tr> <tr>
<td valign="top" align="left"><italic>R</italic><sub><italic>t</italic></sub></td>
<td valign="top" align="left">1</td>
<td valign="top" align="left">Node-density threshold parameter, influencing when motor activity is random</td>
</tr> <tr>
<td valign="top" align="left"><italic>R</italic><sub><italic>g</italic></sub></td>
<td valign="top" align="left">20</td>
<td valign="top" align="left">Steepness of transition between random and IDSM driven motor activity</td>
</tr>
</tbody>
</table>
</table-wrap></sec></sec></sec>
<sec id="s4">
<title>4. Experiments and results</title>
<p>We now present two computational simulations of this model where we vary the sensory modality of the robot to explore how sensorimotor contingencies constrain the forms of the sensorimotor habits that can emerge and self-stabilize. The simulated robots and their environments are identical except that one robot&#x00027;s sensors are visual (as described above) and the other&#x00027;s are auditory. Formally, the only difference between these simulations is the equations that describe the stimulation of the robot&#x00027;s left and right sensors (Equations 6, 7).</p>
<p>To generate the data presented below, we ran 10 trials of each condition (i.e., we simulated 10 auditory robots and 10 visual robots) and for each condition, we selected a trial that displayed a wide variety of habits, and for which the simulated agent returned to one or more habits that it performed earlier but had stopped performing for some period of time. Not all trials did this&#x02014;some instead rapidly fell into a pattern of behavior that was stable for the duration of the simulation. Data for all of the simulations is available at [DATA STORE LOCATION]. The analysis below covers visual simulation &#x00023;9 and auditory simulation &#x00023;0.</p>
<p>We now present an overview of the behaviors demonstrated by these two robots.</p>
<sec>
<title>4.1. Visual sensors</title>
<p>The path taken by the visual robot through its environment is displayed in <xref ref-type="fig" rid="F3">Figure 3A</xref>. This is a complicated and difficult to visualize trajectory, but it in fact involves several distinct repeated patterns of sensorimotor behavior (<xref ref-type="fig" rid="F3">Figures 3B</xref>&#x02013;<xref ref-type="fig" rid="F3">H</xref>). To identify these, we first plotted the proximity of each of the IDSM&#x00027;s nodes to the robot&#x00027;s sensorimotor state over the course of the simulation (<xref ref-type="fig" rid="F4">Figure 4</xref>). In this time series, it is easy to observe segments of time in which a particular set of nodes is repeatedly visited. For instance, when 750 &#x02A85; <italic>t</italic> &#x02A85; 1,200, there are a few nodes with indices close to 5,000 that are repeatedly visited (tan horizontal sequence of dots), and the same set of nodes are briefly revisited three times in the final 500 time units of the simulation. We manually identified these repeated patterns of sensorimotor activity, and assigned each pattern a label and a color (the example just provided is labeled &#x0201C;D&#x0201D; and colored tan). Times when the robots behavior is not clearly repeated were not assigned a label and are marked with a light gray color. To be clear, repetitions were identified in sensorimotor space (<italic>via</italic> the identification of repeatedly visited nodes), not in physical space. Returning to <xref ref-type="fig" rid="F3">Figure 3</xref>, we can see that when the robot is performing the D pattern, it is moving around the environment, regularly turning in loops with squarish corners. <xref ref-type="fig" rid="F5">Figure 5</xref> shows how the state of the sensors, motors and the distance between the stimulus and the robot as the trial progresses. We can see that each of the colored regions tend to occupy particular regions of the sensorimotor space (the sensorimotor &#x02018;habitat&#x02019; of the habit&#x02014;see Buhrmann et al., <xref ref-type="bibr" rid="B9">2013</xref>) and certain patterns of behavior involve the robot pursuing the moving light (e.g., F-behavior) while others avoid it (e.g., D-behavior).</p>
<fig id="F3" position="float">
<label>Figure 3</label>
<caption><p>The trajectory taken by the visual robot <bold>(A)</bold>, broken down into 8 patterns of repeated sensorimotor activity, i.e., habits <bold>(B&#x02013;H)</bold>. The portions of the trajectory where the robot&#x00027;s behavior is not clearly repetitious are included in plot <bold>(I)</bold>. Each of the nine plots shows the full 1 &#x000D7; 1 arena. Video showing this trial is available at <ext-link ext-link-type="uri" xlink:href="https://www.youtube.com/watch?v=2v1TyvKz9qw">https://www.youtube.com/watch?v=2v1TyvKz9qw</ext-link>.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0003.tif"/>
</fig>
<fig id="F4" position="float">
<label>Figure 4</label>
<caption><p>Node proximity during the course of the visual robot&#x00027;s simulation. The darker the point in this plot, the closer that node is to the robot&#x00027;s sensorimotor state at that point in time. The larger colored points indicate the index of the node that was closest (in sensorimotor state) to the robot&#x00027;s sensorimotor state at that time. The color of the closest-node points indicates the habit that those nodes are associated with. These colors match those in <xref ref-type="fig" rid="F3">Figure 3</xref>.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0004.tif"/>
</fig>
<fig id="F5" position="float">
<label>Figure 5</label>
<caption><p>State of sensors and motors for the simulation of the visual robot.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0005.tif"/>
</fig>
<p>Each of patterns B&#x02013;H can be construed as a potential habit, i.e., a pattern of autonomous, self-maintaining sensorimotor activity. To justify this claim, we observe that</p>
<list list-type="order">
<list-item><p>A node can persist for an extended time if and only if it is regularly &#x0201C;visited,&#x0201D; i.e., if and only if the robot&#x00027;s sensorimotor state regularly comes sufficiently close to the node such that the node&#x00027;s weight always kept above zero.</p></list-item>
<list-item><p>The robot&#x00027;s sensorimotor trajectory depends upon the nodes in that (a) the nodes directly control how the motor components of the sensorimotor state change, and (b) the nodes indirectly constrain and influence the sensory components <italic>via</italic> their effect upon the motors.</p></list-item>
</list>
<p>From these observations it follows that the nodes enable the very thing that they depend upon to persist and reflexively, the enactment of the pattern of behavior enables the very thing that it depends upon (the nodes) for its persistence. Similar to a model of biological autopoiesis (Varela et al., <xref ref-type="bibr" rid="B52">1974</xref>), where inherently unstable components such as metabolism and membrane are mutually enabling, the components of these patterns of behavior in this model are in a relationship of mutual interdependence and support, and thus can be considered autonomous under some readings of the enactivist literature. The behavior itself is operationally closed entity (a &#x0201C;unity&#x0201D;), constituted by inherently unstable components (the nodes and the sensorimotor trajectory) and yet persists thanks to its own enactment or performance.</p>
<p>More detailed analysis of the operational closure of these systems is outside the scope of the present paper (but see Egbert, <xref ref-type="bibr" rid="B19">2018</xref> for initial analysis of the precarious autonomy of a simple IDSM-based habit). We can see the basic idea however, when we consider how the weight of the nodes changes as time passes in the simulation (see <xref ref-type="fig" rid="F6">Figure 6</xref>). Recalling that when the weight of a node reaches zero, that node ceases to exist, it is clear that only nodes that are regularly visited can persist in the long term. For a collection of nodes to be visited, the sensorimotor trajectory must move in a particular trajectory and the sensorimotor trajectory is largely determined by the activity of nodes. It follows from this that the only way that a pattern of behavior can persist is if it is one that causes the repeated revisitation of its constituent nodes. We can also note that this self-reinforcement of a habit need not be constant or contiguous. For example, the B-habit (red) is established early in the simulation [<italic>t</italic> &#x02248; 250)] and then is not visited again until <italic>t</italic> &#x02248; 1,250, where it is enacted a few times and the nodes are reinforced such that they survive until close to the end of the simulation. The parameters that prescribe the rates of node weight reinforcement (<italic>k</italic><sub>&#x02191;</sub>) and degradation (<italic>k</italic><sub>&#x02193;</sub>) determine how regularly patterns of behavior must be enacted if they are to persist.</p>
<fig id="F6" position="float">
<label>Figure 6</label>
<caption><p>Node weight during the course of the visual robot&#x00027;s simulation. The weight of each node (shades of gray) are shown as time progresses during the simulation. After most nodes appear, their weights decay (become lighter as time passes), but some nodes are regularly visited and their weights are reinforced (long gray horizontal lines). The closest node at any time is plotted in color, just as in <xref ref-type="fig" rid="F4">Figure 4</xref>.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0006.tif"/>
</fig></sec>
<sec>
<title>4.2. Auditory sensors</title>
<p>We performed a similar analysis on the auditory robot. <xref ref-type="fig" rid="F7">Figure 7A</xref> shows the complete path taken by the auditory robot in its environment, broken down by habit (<xref ref-type="fig" rid="F7">Figures 7B</xref>&#x02013;<xref ref-type="fig" rid="F7">H</xref>). <xref ref-type="fig" rid="F8">Figure 8</xref> shows the weight of each node which also provides an indication for how close each node is to the current sensorimotor state of the robot and the timeseries plots for the auditory robot are shown in <xref ref-type="fig" rid="F9">Figure 9</xref>. It is worth noting that though these patterns of behavior may seem random, they all (both visual and auditory) relate to the moving stimulus source in regular, non-random ways. The regularities in the interaction with the stimulus are much more easily seen in the animations linked to in the captions of <xref ref-type="fig" rid="F3">Figures 3</xref>, <xref ref-type="fig" rid="F7">7</xref>).</p>
<fig id="F7" position="float">
<label>Figure 7</label>
<caption><p>Spatial trajectory taken by the auditory robot <bold>(A)</bold>; broken down into &#x0201C;habits&#x0201D; i.e., self-reinforcing patterns of behavior <bold>(B&#x02013;H)</bold>; and portions of the trajectory not associated with any habits <bold>(I)</bold>. Each of the nine plots shows the full 1 &#x000D7; 1 arena. Video showing this trial is available at <ext-link ext-link-type="uri" xlink:href="https://www.youtube.com/watch?v=If_WeclEtCM">https://www.youtube.com/watch?v=If_WeclEtCM</ext-link>.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0007.tif"/>
</fig>
<fig id="F8" position="float">
<label>Figure 8</label>
<caption><p>Node weights with nodes closest to the current sensorimotor state of the auditory robot highlighted and marked by color of habit.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0008.tif"/>
</fig>
<fig id="F9" position="float">
<label>Figure 9</label>
<caption><p>State of sensors and motors for the simulation of the auditory robot.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0009.tif"/>
</fig></sec>
<sec>
<title>4.3. Comparison of sensorimotor structures</title>
<p>Even when two behavior&#x00027;s functions are the same (e.g., they both accomplish taxis) the forms of the underlying sensorimotor habit can be different. As a case in point, we can compare auditory habit E with visual habit F. In both cases, the robot circles around the stimulus, maintaining a similar approximate distance from the stimulus as the stimulus moves in spurts around the environment. When we look at the two patterns of sensorimotor activity (<xref ref-type="fig" rid="F10">Figure 10</xref>), we see two different pictures. Each row of this visualization shows the spatial trajectory of the robot (left) followed by four projections of the 4D sensorimotor trajectory of the robot and it is obvious that the two sensorimotor trajectories are qualitatively different. The auditory pattern involves less diversity in the range of states visited; the mean and other statistical properties of the sensorimotor trajectories are also clearly different, etc. But every habit in this model is unique&#x02014;how much of the difference between these habits is simply due to the fact that they are different habits, and how much is due to the difference in the robot&#x00027;s sensorimotor modalities?</p>
<fig id="F10" position="float">
<label>Figure 10</label>
<caption><p>Contrasting the sensorimotor dynamics of two &#x0201C;circling&#x0201D; habits. In both cases, the robot circles the stimulus, but they use different sensory modalities to do so. The patterns of sensorimotor activity are clearly different despite the external &#x0201C;functional&#x0201D; description of the behavior.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0010.tif"/>
</fig>
<p>To address this question, we can look at projections of the sensorimotor trajectories of all of the habits of the visual robot (<xref ref-type="fig" rid="F13">Figure 13</xref>) and observe that there are regularities in the forms of the habits of this robot. This is most readily seen in the left-sensor vs. right-sensor projection (upper right) which reveals elements of this robot&#x00027;s sensorimotor contingencies, including aspects of both its sensorimotor habitat and environment (Buhrmann et al., <xref ref-type="bibr" rid="B9">2013</xref>) of the robot. Specifically, the bilateral symmetry of the robot&#x00027;s embodiment result in a (statistical) mirror symmetry across the left sensor = right sensor diagonal. The directionality of the visual sensors and their different orientations mean that it is impossible (in the environment with a single stimulus) to maximally stimulate both left and right sensors concurrently, leading to the L-shape apparent in upper-right plot of <xref ref-type="fig" rid="F13">Figure 13</xref>, with arcs that pass between the stems of the L as the robot turns such that the stimulus moves from being in front of one sensor to in front of the other. These features are also found in the other nine visual robot trials.</p>
<p>Similar plots for the auditory robots (<xref ref-type="fig" rid="F14">Figure 14</xref>) also reveal regularities. Like the visual robot, the auditory robot&#x00027;s embodiment is bilaterally symmetric and so it has the same symmetry across the diagonal in left-sensor/right-sensor plot, but the non-directionality of the auditory sensors and the fact that their excitation does not fall off with distance make other aspects of the sensorimotor activity different. The auditory trajectories tend to repeat patterns of diverging from a center point in short arcs in a pattern of returning and diverging again, in a different direction. These dynamics can be seen in greater detail in <xref ref-type="app" rid="A1">Appendix A</xref>, which shows the projection in sensorimotor space of each habit individually.</p>
<p>Sensors and motors are involved in a recursive relationship of influence, where sensors influence motors (<italic>via</italic> the controller) and motors influence sensors (<italic>via</italic> the effect of actions upon the environment and the agent&#x00027;s relationship to the environment). It therefore would make sense that the constraints imposed upon sensory dynamics by different sensory modalities would constrain the motor dynamics. In other words, under one sensory modality, certain motor trajectories will be more readily repeated (and thus stabilized) than others, and which motor trajectories are more readily repeated would depend upon the sensory modality.</p>
<p>We do, in fact, see differences in the distributions of motor states between the two sensory conditions (<xref ref-type="fig" rid="F11">Figure 11</xref>). Visually, we can see that the distributions of motor (and abstract motor) states in the visual robots seem different than that of the auditory robots (<xref ref-type="fig" rid="F11">Figure 11</xref>); and the distributions of motor states within these groups in independent runs of the visual trial seem more similar to each other (<xref ref-type="fig" rid="F12">Figure 12</xref>) than they are to the other group. We confirmed the statistical significance of these differences by using Kullback&#x02013;Leibler divergence to assess the distributional conformity of 50 visual trials and 50 auditory trials. The within-group (intra-modal) distributions were significantly more similar than the between-group (inter-modal) distributions for <italic>m</italic> (<italic>t</italic> = &#x02013;4.67, <italic>p</italic> = 0.0114) and for &#x003BC; (<italic>t</italic> = &#x02013;8.507, <italic>p</italic> = 0.0215).</p>
<fig id="F11" position="float">
<label>Figure 11</label>
<caption><p>Histograms showing difference in distributions of sensorimotor states for 50 visual and 50 auditory trials. As expected, the different sensory modalities produce different distributions of the sensor state. It can also be seen that in visual trials, the abstract motor-state tended to be higher (corresponding to faster rates of motor state change&#x02014;see <xref ref-type="fig" rid="F2">Figure 2</xref>) and the motor state tended to be more extreme.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0011.tif"/>
</fig>
<fig id="F12" position="float">
<label>Figure 12</label>
<caption><p>Histograms showing in-group similarity in sensorimotor state for visual trials <bold>(left)</bold> and auditory trials <bold>(right)</bold>.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0012.tif"/>
</fig>
<p>In the absence of the self-reinforcing dynamic of the IDSM, habits, motor activity would be random, in which case we would expect the same distribution of motor states in both sensory modalities. The histograms in (new) (<xref ref-type="fig" rid="F11">Figures 11</xref>, <xref ref-type="fig" rid="F12">12</xref>) show that this is not the case, and thus provide support for the claim that the sensorimotor contingencies implicit in the different sensory modalities constrain not only the sensory dynamics, but also motor dynamics&#x02014;and in doing so, they play an important role of constraining the form of emergent autonomous sensorimotor dynamics.</p>
<p>The primary point that we wish to communicate using this model is that sensorimotor contingencies influence which patterns of autonomous sensorimotor behavior can emerge and persist. It is worth emphasizing that the plots in <xref ref-type="fig" rid="F10">Figures 10</xref>, <xref ref-type="fig" rid="F13">13</xref>, <xref ref-type="fig" rid="F14">14</xref> are not plots of the sensorimotor contingencies of the robot&#x02014;they are plots of the robots&#x00027; self-maintaining habits. Nevertheless, we see in these plots the <italic>influence</italic> of sensorimotor contingencies&#x02014;the way that the contingencies have constrained the set of habits that can form and self-stabilize, producing the regular patterns described in the paragraphs above.</p>
<fig id="F13" position="float">
<label>Figure 13</label>
<caption><p>Sensorimotor dynamics for the visual robot. These plots show motor trajectories in abstract sensorimotor space <bold>(upper left)</bold>; two projections of the actual sensorimotor trajectories of the robot <bold>(upper</bold> and <bold>lower right)</bold>; and the mapping function, &#x0039B;, that transforms the abstract motor state to an actual motor state (see Section 3.4.2). Colors indicate habits as in previous plots.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0013.tif"/>
</fig>
<fig id="F14" position="float">
<label>Figure 14</label>
<caption><p>Sensorimotor dynamics for the auditory robot. This figure is the same as <xref ref-type="fig" rid="F13">Figure 13</xref> but plotted for the auditory robot.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0014.tif"/>
</fig>
<p>The two different sensory modalities entail two different sets of sensorimotor contingencies. The different sensorimotor contingencies each imply a different set of possible self-stabilizing habits, and so the habits that form with one sensory modality are more similar to each other than they are to the habits that form with the other sensory modality. Two <italic>visual</italic> habits will be different from each other, but they will generally be more similar to each other than auditory habits. This is what we mean by the claim that the form of sensorimotor habits are constrained by their sensory modality and it is apparent in the regularities described above and seen in <xref ref-type="fig" rid="F13">Figures 13</xref>, <xref ref-type="fig" rid="F14">14</xref>. In this account, sensorimotor contingencies play an important role. They influence the form of the emergent autonomous sensorimotor strutures, but the rules themselves are not internalized or learned in any other way.</p></sec></sec>
<sec sec-type="discussion" id="s5">
<title>5. Discussion</title>
<sec>
<title>5.1. Solving somebody else&#x00027;s problems vs. having your own problems</title>
<p>The robots we have just presented display a number of interesting and challenging features for traditional problem-solving centered artificial intelligence (biologically inspired or otherwise). We see in this model how sensorimotor dynamics can have problems of their own. As we have seen AI and robots more specifically are generally built to solve problems. But whose problems? Perhaps the engineers&#x00027; problems or those of their clients, perhaps animal problems that require robotic models to be better understood, maybe societies problems but certainly not those of the robot itself. It is thus important to distinguish between having one&#x00027;s problem vs. being able to solve a problem. A bacterium that swims up a gradient toward the resources it needs to survive is doing both. It has a problem (it needs food to persist) and it solves that problem by navigating through its environment. AI, on the other hand, is capable only of the former. A &#x0201C;self-driving&#x0201D; car that takes me to a restaurant dinner is solving a problem (my need for food), but that problem is mine, and not the car&#x00027;s.</p>
<p>The enactive robots presented in this paper where not designed to do anything, to solve any problem. None of our analysis was described in terms of success or failure at the performance of a task. The architecture makes habits emerge and with them a basic sense of having a problem: the habit <italic>needs</italic> to enact itself in order to persist. The norms of behavior here are thus only in terms of the persistence of the behavior itself (and concurrently the mechanism that produces it). Surely, life is full of problems, we certainly have enough with our own and don&#x00027;t need to be concerned with creating new ones &#x02026;unless we want to understand what it is to have a problem and how life is, itself, a source of problems. Enactivism embraces a deep conception of life as the self maintenance of a network of precarious processes. In this sense life is inherently a source of problems. By making it possible for habits to emerge and sustain their own existence through recurrent sensorimotor interaction we have shown how this conception of life can be practically transferred to a &#x0201C;way of life&#x0201D; in sensorimotor robotics.</p>
<p>A research program on enactive robotics can and must address how sensorimotor life is a source of problems. We can move beyond the single habit and envision how the network of habits that constitutes the agents can display emergent problems of its own: that of keeping the whole network alive, coping with variations on the way the environment affords or precludes to enact them. This line of enquiry requires further work but the present model requires little development to start addressing it.</p></sec>
<sec>
<title>5.2. Constraints and form in biological and behavioral explanation</title>
<p>Biological inspiration in robotics can also move beyond the analogy between metabolic or physiological life and its sensorimotor counterpart. In particular the models presented and analyzed in this paper show how sensorimotor life and the problems that emerge within are constrained by sensory embodiment and how such constraints constitute an important part of what it means to explain life.</p>
<p>Whether it is understood as more fundamental-than, complementary-to or directly at odds-with (Newman, <xref ref-type="bibr" rid="B35">2018</xref>) natural selection, it is the repeated, regular, robust and often phylogenetically independent appearance of forms or structures in biology what becomes in itself an object of study that cannot be reduced-to or deduced-from life as a response to environmental problems. In this sense biologically inspired enactive robotics can learn from biological development and its organization and import into psychology some explanatory strategies followed by biologists. Darcy Thompson, Ren&#x000E9; Thom, Pere Albert, etc. all conceived biological forms to be both <italic>explananda</italic> and <italic>explanans</italic> on themselves. Homologies, analogies, inherency, are all concepts directed to capture the re-emergence of certain forms (and functions!) in biological organisms. The notion of constraint plays here a fundamental role. The way in which different layers of materiality and self-organization limit and channel the emergence of viable forms is essential to biological explanation.</p>
<p>If biological inspiration within the problem-solving paradigm concluded that the material embodiment of robots permits to offload and transform the problems to be solved, the biological inspiration of the enactive paradigm can conclude that the embodiment loads and informs the problems that constitute the sensorimotor agent. As we have seen, sensory modalities constraint motor trajectories, which in turn, shape how habits get stabilized. Thus, in a way that parallels the explanatory role of constrains in evolution, we can hypothesize that behavioral variability is not free (to explore potential solutions to cognitive problems) but is constrained and channeled by the embodiment of the agent.</p>
<p>Now, before the advent and widespread influence of evolutionary theory in biology, the latter synthesis with molecular biology and subsequent expansion to psychology, sociology and even epistemology, or philosophy more generally, the concept that was key for the continuity between life and mind was that of habit.</p></sec>
<sec>
<title>5.3. Enactive robotics revisited</title>
<p>There are different takes on how enactivism translates into robotics. As mentioned in the introduction it is possible to simply reject representational functionalism, or to claim for the importance of embodiment, or to demand that robot be endowed with living bodies, or to introduce some feedback mechanisms that parallel those provided by emotion bearing bodies. We have taken a different approach&#x02014;that envisioned by Smithers (<xref ref-type="bibr" rid="B48">1997</xref>) and latter developed by Di Paolo (<xref ref-type="bibr" rid="B14">2003</xref>), Barandiaran and Moreno (<xref ref-type="bibr" rid="B3">2006</xref>), and Barandiaran (<xref ref-type="bibr" rid="B5">2008</xref>). What a research programme in enactive robotics entails is the study of the organization of sensorimotor life: the form of viability of different habits, the topology of the network of habits that unfolds over development, the shape of the habitat that is thus constructed, the structure of the world that is experienced. Habit-based robots as designed here are capable of individuating habits and of creating an ecology of habits that can easily be understood as a form of sensorimotor self.</p>
<p>Not only has enactivism informed robotics (Ziemke and Lowe, <xref ref-type="bibr" rid="B57">2009</xref>; Vernon, <xref ref-type="bibr" rid="B53">2010</xref>) but robotics has often served enactivism (Beer, <xref ref-type="bibr" rid="B8">1995</xref>; Di Paolo, <xref ref-type="bibr" rid="B14">2003</xref>; Aguilera et al., <xref ref-type="bibr" rid="B1">2016</xref>) by clarifying its claims, pushing theoretical development, operationalizing its concepts or penetrating diverse problems. The model we developed here can be aligned with the latter. It can be used to clarify and make explicit the often obscure original formulation of enactivism that &#x0201C;cognition in its most encompassing sense consists in the enactment or bringing forth of a world by a viable history of structural coupling&#x0201D; (Varela et al., <xref ref-type="bibr" rid="B51">1991</xref>, p. 205). We have shown how a robot endowed with an IDSM can bring into being a number of habits, the world that it brings forth is the habitat, more specifically, the structured set of sensorimotor contingencies that the agent inhabits or enacts. This habitat must be viable in the sense that habits must be sustainable and results from a (developmental) history of sensorimotor (structural) coupling. And in so doing, we have demonstrated how sensorimotor contingencies can <italic>directly</italic> constrain or &#x0201C;sculpt&#x0201D; the form taken by sensorimotor habits without requiring any virtualization, i.e., without any internal model or representation of the sensorimotor contingency.</p>
<p>Perhaps it is worth clarifying why the nodes do not constitute an internal model or representation. The nodes do not stand-for something else other than the sensorimotor dynamics they partake in. It is not possible to operate upon the nodes in a decoupled offline mode that is not itself the enactment of a behavior, and there is not additional module or subsystem &#x0201C;consuming&#x0201D; such nodes to carry out any further operation. In these ways, the nodes do not represent a behavior or habit, but they embody it&#x02014;they constitute the habit <italic>together</italic> with elements of the robot&#x00027;s bodily and environmental dynamics.</p></sec></sec>
<sec sec-type="conclusions" id="s6">
<title>6. Conclusion</title>
<p>A central focus across the cognitive sciences is upon problem-solving ability and tremendous progress has been made in understanding how to mechanize problem solving. Much of AI and robotics research is validated by how well some artifact (neural network, human being, robot, etc.) performs at a problem-solving task (chess, maze navigation, bipedal walking, etc.).</p>
<p>However, the conflation of &#x0201C;problem-solving ability&#x0201D; with all of the phenomena associated with &#x02018;being a mindful body leaves out a number of features that demand be put at the center of (enactive) theorizing: historicity, embodiment, habitat, precariousness, identity, norm-establishing, etc. All these dimensions of mental life are worthy of study and remain outside of the problem-solving frame that scaffolded the development of Artificial Intelligence and Robotics.</p>
<p>We have here presented a set of robots with different sensory modalities that spontaneously develops a complex ecology of sensorimotor habits. These are constrained by the sensory modality of the robot and give rise to sensorimotor habitats of specific forms. The very nature of habits thus developed, understood as self-sustaining forms of sensorimotor activity, has shown how robots must first have their own problems instead of solving those posited by external observers; and, that, in doing so, they must assert a form of life whose structure and topology must be taken as the object of study. In particular we have seen how the form of sensory embodiment shapes potential sensorimotor contingencies and these constraint the shape and type of sensorimotor habits that emerge during development.</p>
<p>To be alive is not a computable function but the way in which materiality (implementation), behavior and function are deeply intertwined. What was once claimed as the triumph of functionalism as the clear conceptual separation between matter, behavior and machine state transitions is now its deepest weakness. The enactive approach brings all three together again. To be fair it is not the materiality of the robot&#x00027;s &#x0201C;body&#x0201D; that is a stake here (neither does the simulation itself possess any body beyond the computer in which the simulation was carried out, not the physical body, e.g. wheels and sensors, would be at stake was the robot to be implemented in real life), but the materiality of the sensorimotor mapping, its precarious existence, its fading structural stability.</p>
<p>Perhaps the &#x0201C;artificial sciences&#x0201D; (AI, artificial life, robotics, exploratory computational modeling, etc.) would benefit from similarly investing more time in targets other than problem-solving ability. Biologists are sometimes accused of suffering &#x0201C;physics envy&#x0201D;&#x02014;i.e., wishing that the objects of their study were more easily and completely summarized by simple, provable equations. Perhaps we enactivist and embodied researchers can be accused of &#x0201C;problem-solving envy&#x0201D; a desire for our artifacts and theories to be equally or more capable of solving problems as the expert systems or disembodied neural networks of other problem-solving focused approaches. And perhaps this envy is a distraction, and impeding our progress toward understanding minds. In fact, robotics cannot only reveal itself as an engineering practice directed at solving problems but also as a philosophical practice aimed at posing the right problems. This is a contribution that enactive robotics is ready to do.</p></sec>
<sec id="s7">
<title>7. Permission to reuse and copyright</title>
<p>Figures, tables, and images will be published under a Creative Commons CC-BY license and permission must be obtained for use of copyrighted material from other sources (including re-published/adapted/modified/partial figures and images from the internet). It is the responsibility of the authors to acquire the licenses, to follow any citation instructions requested by third-party rights holders, and cover any supplementary charges.</p></sec>
<sec sec-type="data-availability" id="s8">
<title>Data availability statement</title>
<p>The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.</p></sec>
<sec sec-type="author-contributions" id="s9">
<title>Author contributions</title>
<p>ME conceived, implemented, ran simulations and experiments, and authored the manuscript. XB contextualized research, interpreted results, and authored the manuscript. Both authors contributed to the article and approved the submitted version.</p></sec>
</body>
<back>
<sec sec-type="funding-information" id="s10">
<title>Funding</title>
<p>This work was supported in part <italic>via</italic> funding from the Digital Life Institute, University of Auckland. XB acknowledges funding from the Spanish Ministry of Science and Innovation for the research project Outonomy PID2019-104576GB-I00 and IAS-Research group funding IT1668-22 from Basque Government.</p>
</sec>
<sec sec-type="COI-statement" id="conf1">
<title>Conflict of interest</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec sec-type="disclaimer" id="s11">
<title>Publisher&#x00027;s note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
<ref-list>
<title>References</title>
<ref id="B1">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Aguilera</surname> <given-names>M.</given-names></name> <name><surname>Bedia</surname> <given-names>M. G.</given-names></name> <name><surname>Barandiaran</surname> <given-names>X. E.</given-names></name></person-group> (<year>2016</year>). <article-title>Extended neural metastability in an embodied model of sensorimotor coupling</article-title>. <source>Front. Syst. Neurosci</source>. <volume>10</volume>, <fpage>76</fpage>. <pub-id pub-id-type="doi">10.3389/fnsys.2016.00076</pub-id><pub-id pub-id-type="pmid">27721746</pub-id></citation></ref>
<ref id="B2">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Ashby</surname> <given-names>W. R.</given-names></name></person-group> (<year>1952</year>). <source>Design for a BRAIN: The Origin of Adaptative Behaviour, 2nd Edn</source>. <publisher-loc>London</publisher-loc>: <publisher-name>J. Wiley</publisher-name>.</citation></ref>
<ref id="B3">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Barandiaran</surname> <given-names>X.</given-names></name> <name><surname>Moreno</surname> <given-names>A.</given-names></name></person-group> (<year>2006</year>). <article-title>On what makes certain dynamical systems cognitive: a minimally cognitive organization program</article-title>. <source>Adapt. Behav</source>. <volume>14</volume>, <fpage>171</fpage>&#x02013;<lpage>185</lpage>. <pub-id pub-id-type="doi">10.1177/105971230601400208</pub-id></citation></ref>
<ref id="B4">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Barandiaran</surname> <given-names>X.</given-names></name></person-group> (<year>2007</year>). <article-title>Mental life: conceptual models and sythetic methodologies for a post-cognitivist psychology</article-title>, in <source>The World, the Mind and the Body: Psychology after Cognitivism</source>, eds <person-group person-group-type="editor"><name><surname>Wallace</surname> <given-names>B.</given-names></name> <name><surname>Ross</surname> <given-names>A.</given-names></name> <name><surname>Anderson</surname> <given-names>T.</given-names></name> <name><surname>Davies</surname> <given-names>J.</given-names></name></person-group> (<publisher-loc>Exeter</publisher-loc>: <publisher-name>Imprint Academic</publisher-name>), <fpage>49</fpage>&#x02013;<lpage>90</lpage>.</citation></ref>
<ref id="B5">
<citation citation-type="thesis"><person-group person-group-type="author"><name><surname>Barandiaran</surname> <given-names>X. E.</given-names></name></person-group> (<year>2008</year>). <source>Mental Life. A Naturalized Approach to the Autonomy of Cognitive Agents</source> (Ph.D. thesis). University of the Basque Country, Spain.</citation></ref>
<ref id="B6">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Barandiaran</surname> <given-names>X. E.</given-names></name></person-group> (<year>2017</year>). <article-title>Autonomy and enactivism: towards a theory of sensorimotor autonomous agency</article-title>. <source>Topoi</source> <volume>36</volume>, <fpage>409</fpage>&#x02013;<lpage>430</lpage>. <pub-id pub-id-type="doi">10.1007/s11245-016-9365-4</pub-id></citation></ref>
<ref id="B7">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Barandiaran</surname> <given-names>X. E.</given-names></name> <name><surname>Egbert</surname> <given-names>M. D.</given-names></name></person-group> (<year>2013</year>). <article-title>Norm-establishing and norm-following in autonomous agency</article-title>. <source>Artif. Life</source> <volume>20</volume>, <fpage>5</fpage>&#x02013;<lpage>28</lpage>. <pub-id pub-id-type="doi">10.1162/ARTL_a_00094</pub-id><pub-id pub-id-type="pmid">23373978</pub-id></citation></ref>
<ref id="B8">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Beer</surname> <given-names>R. D.</given-names></name></person-group> (<year>1995</year>). <article-title>A dynamical systems perspective on agent-environment interaction</article-title>. <source>Artif. Intell</source>. <volume>72</volume>, <fpage>173</fpage>&#x02013;<lpage>215</lpage>. <pub-id pub-id-type="doi">10.1016/0004-3702(94)00005-L</pub-id></citation></ref>
<ref id="B9">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Buhrmann</surname> <given-names>T.</given-names></name> <name><surname>Di Paolo</surname> <given-names>E. A.</given-names></name> <name><surname>Barandiaran</surname> <given-names>X.</given-names></name></person-group> (<year>2013</year>). <article-title>A dynamical systems account of sensorimotor contingencies</article-title>. <source>Front. Psychol</source>. <volume>4</volume>, <fpage>285</fpage>. <pub-id pub-id-type="doi">10.3389/fpsyg.2013.00285</pub-id><pub-id pub-id-type="pmid">23750143</pub-id></citation></ref>
<ref id="B10">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Buss</surname> <given-names>D. M.</given-names></name></person-group> (<year>1998</year>). <source>Evolutionary Psychology: The New Science of the Mind</source>. <publisher-loc>London</publisher-loc>: <publisher-name>Allyn and Bacon</publisher-name>.</citation></ref>
<ref id="B11">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Canaan</surname> <given-names>R.</given-names></name> <name><surname>Salge</surname> <given-names>C.</given-names></name> <name><surname>Togelius</surname> <given-names>J.</given-names></name> <name><surname>Nealen</surname> <given-names>A.</given-names></name></person-group> (<year>2019</year>). <article-title>Leveling the playing field: fairness in AI versus human game benchmarks</article-title>, in <source>Proceedings of the 14th International Conference on the Foundations of Digital Games, FDG &#x00027;19</source> (<publisher-loc>New York, NY</publisher-loc>: <publisher-name>Association for Computing Machinery</publisher-name>), <fpage>1</fpage>&#x02013;<lpage>8</lpage>.</citation></ref>
<ref id="B12">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Cliff</surname> <given-names>D.</given-names></name> <name><surname>Husbands</surname> <given-names>P.</given-names></name> <name><surname>Harvey</surname> <given-names>I.</given-names></name></person-group> (<year>1993</year>). <article-title>Explorations in evolutionary robotics</article-title>. <source>Adapt. Behav</source>. <volume>2</volume>, <fpage>73</fpage>&#x02013;<lpage>110</lpage>. <pub-id pub-id-type="doi">10.1177/105971239300200104</pub-id><pub-id pub-id-type="pmid">19619980</pub-id></citation></ref>
<ref id="B13">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Damiano</surname> <given-names>L.</given-names></name> <name><surname>Stano</surname> <given-names>P.</given-names></name></person-group> (<year>2021</year>). <article-title>A wetware embodied AI? Towards an autopoietic organizational approach grounded in synthetic biology</article-title>. <source>Front. Bioeng. Biotechnol</source>. <volume>9</volume>, <fpage>724023</fpage>. <pub-id pub-id-type="doi">10.3389/fbioe.2021.724023</pub-id><pub-id pub-id-type="pmid">34631678</pub-id></citation></ref>
<ref id="B14">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Di Paolo</surname> <given-names>E. A.</given-names></name></person-group> (<year>2003</year>). <article-title>Organismically-inspired robotics: homeostatic adaptation and teleology beyond the closed sensorimotor loop</article-title>, in <source>Dynamical Systems Approach to Embodiment and Sociality</source>, eds <person-group person-group-type="editor"><name><surname>Murase</surname> <given-names>K.</given-names></name> <name><surname>Asakura</surname> <given-names>T.</given-names></name></person-group> (<publisher-loc>Adelaide, SA</publisher-loc>: <publisher-name>Advanced Knowledge International</publisher-name>), <fpage>19</fpage>&#x02013;<lpage>42</lpage>.</citation></ref>
<ref id="B15">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Di Paolo</surname> <given-names>E. A.</given-names></name> <name><surname>Buhrmann</surname> <given-names>T.</given-names></name> <name><surname>Barandiaran</surname> <given-names>X.</given-names></name></person-group> (<year>2017</year>). <source>Sensorimotor Life: An Enactive Proposal, 1st Edn</source>. <publisher-loc>Oxford</publisher-loc>: <publisher-name>Oxford University Press</publisher-name>.</citation></ref>
<ref id="B16">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Egbert</surname> <given-names>M. D.</given-names></name> <name><surname>Barandiaran</surname> <given-names>X. E.</given-names></name></person-group> (<year>2014</year>). <article-title>Modeling habits as self-sustaining patterns of sensorimotor behavior</article-title>. <source>Front. Hum. Neurosci</source>. <volume>8</volume>, <fpage>590</fpage>. <pub-id pub-id-type="doi">10.3389/fnhum.2014.00590</pub-id><pub-id pub-id-type="pmid">25941482</pub-id></citation></ref>
<ref id="B17">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Egbert</surname> <given-names>M. D.</given-names></name> <name><surname>Barandiaran</surname> <given-names>X. E.</given-names></name></person-group> (<year>2015</year>). <article-title>Corrigendum: Modeling habits as self sustaining patterns of sensorimotor behavior</article-title>. <source>Front. Hum. Neurosci.</source> <volume>9</volume>, <fpage>209</fpage>. <pub-id pub-id-type="doi">10.3389/fnhum.2015.00209</pub-id><pub-id pub-id-type="pmid">25941482</pub-id></citation></ref>
<ref id="B18">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Egbert</surname> <given-names>M. D.</given-names></name> <name><surname>Ca&#x000F1;amero</surname> <given-names>L.</given-names></name></person-group> (<year>2014</year>). <article-title>Habit-based regulation of essential variables</article-title>, in <source>Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems</source>, eds <person-group person-group-type="editor"><name><surname>Sayama</surname> <given-names>H.</given-names></name> <name><surname>Rieffel</surname> <given-names>J.</given-names></name> <name><surname>Risi</surname> <given-names>S.</given-names></name> <name><surname>Doursat</surname> <given-names>R.</given-names></name> <name><surname>Lipson</surname> <given-names>H.</given-names></name></person-group> (<publisher-loc>New York, NY</publisher-loc>: <publisher-name>MIT Press</publisher-name>), <fpage>168</fpage>&#x02013;<lpage>175</lpage>.</citation></ref>
<ref id="B19">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Egbert</surname> <given-names>M. D.</given-names></name></person-group> (<year>2018</year>). <article-title>Investigations of an adaptive and autonomous sensorimotor individual</article-title>, in <source>The 2018 Conference on Artificial Life: A Hybrid of the European Conference on Artificial Life (ECAL) and the International Conference on the Synthesis and Simulation of Living Systems (ALIFE)</source>, eds <person-group person-group-type="editor"><name><surname>Ikegami</surname> <given-names>T.</given-names></name> <name><surname>Virgo</surname> <given-names>N.</given-names></name> <name><surname>Witkowski</surname> <given-names>O.</given-names></name> <name><surname>Oka</surname> <given-names>M.</given-names></name> <name><surname>Suzuki</surname> <given-names>R.</given-names></name> <name><surname>Iizuka</surname> <given-names>H.</given-names></name></person-group> (<publisher-loc>New York, NY</publisher-loc>: <publisher-name>MIT Press</publisher-name>), <fpage>343</fpage>&#x02013;<lpage>350</lpage>.</citation></ref>
<ref id="B20">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Fodor</surname> <given-names>J. A.</given-names></name></person-group> (<year>1968</year>). <source>Psychological Explanation; An Introduction to the Philosophy of Psychology</source>. <publisher-loc>New York, NY</publisher-loc>: <publisher-name>Random House Inc</publisher-name>.</citation></ref>
<ref id="B21">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>G&#x000E1;nti</surname> <given-names>T.</given-names></name></person-group> (<year>1975</year>). <article-title>Organization of chemical reactions into dividing and metabolizing units: the chemotons</article-title>. <source>Biosystems</source> <volume>7</volume>, <fpage>15</fpage>&#x02013;<lpage>21</lpage>. <pub-id pub-id-type="doi">10.1016/0303-2647(75)90038-6</pub-id><pub-id pub-id-type="pmid">1156666</pub-id></citation></ref>
<ref id="B22">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Graves</surname> <given-names>A.</given-names></name> <name><surname>Schmidhuber</surname> <given-names>J.</given-names></name></person-group> (<year>2009</year>). <article-title>Offline handwriting recognition with multidimensional recurrent neural networks</article-title>, in <source>Advances in Neural Information Processing Systems, Vol. 21</source>, eds <person-group person-group-type="editor"><name><surname>Koller</surname> <given-names>D.</given-names></name> <name><surname>Schuurmans</surname> <given-names>D.</given-names></name> <name><surname>Bengio</surname> <given-names>Y.</given-names></name> <name><surname>Bottou</surname> <given-names>L.</given-names></name></person-group> (<publisher-loc>New York, NY</publisher-loc>: <publisher-name>Curran Associates, Inc.</publisher-name>)</citation></ref>
<ref id="B23">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Grey Walter</surname> <given-names>W.</given-names></name></person-group> (<year>1950</year>). <article-title>An imitation of life</article-title>. <source>Sci. Am</source>. <volume>182</volume>, <fpage>42</fpage>&#x02013;<lpage>45</lpage>. <pub-id pub-id-type="doi">10.1038/scientificamerican0550-42</pub-id></citation></ref>
<ref id="B24">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Hambrick</surname> <given-names>D. Z.</given-names></name> <name><surname>Burgoyne</surname> <given-names>A. P.</given-names></name> <name><surname>Altmann</surname> <given-names>E. M.</given-names></name></person-group> (<year>2020</year>). <article-title>Problem-solving and intelligence</article-title>, in <source>The Cambridge Handbook of Intelligence, Cambridge Handbooks in Psychology, 2nd Edn</source>, ed <person-group person-group-type="editor"><name><surname>Sternberg</surname> <given-names>R. J.</given-names></name></person-group> (<publisher-loc>Cambridge</publisher-loc>: <publisher-name>Cambridge University Press</publisher-name>), <fpage>553</fpage>&#x02013;<lpage>579</lpage>.</citation></ref>
<ref id="B25">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Harvey</surname> <given-names>I.</given-names></name> <name><surname>Husbands</surname> <given-names>P.</given-names></name> <name><surname>Cliff</surname> <given-names>D.</given-names></name> <name><surname>Thompson</surname> <given-names>A.</given-names></name> <name><surname>Jakobi</surname> <given-names>N.</given-names></name></person-group> (<year>1997</year>). <article-title>Evolutionary robotics: the sussex approach</article-title>. <source>Rob. Auton. Syst</source>. <volume>20</volume>, <fpage>205</fpage>&#x02013;<lpage>224</lpage>. <pub-id pub-id-type="doi">10.1016/S0921-8890(96)00067-X</pub-id></citation></ref>
<ref id="B26">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Jacquey</surname> <given-names>L.</given-names></name> <name><surname>Baldassarre</surname> <given-names>G.</given-names></name> <name><surname>Santucci</surname> <given-names>V. G.</given-names></name> <name><surname>O&#x00027;Regan</surname> <given-names>J. K.</given-names></name></person-group> (<year>2019</year>). <article-title>Sensorimotor contingencies as a key drive of development: from babies to robots</article-title>. <source>Front. Neurorobot</source>. <volume>13</volume>, <fpage>98</fpage>. <pub-id pub-id-type="doi">10.3389/fnbot.2019.00098</pub-id><pub-id pub-id-type="pmid">31866848</pub-id></citation></ref>
<ref id="B27">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Luisi</surname> <given-names>P. L.</given-names></name></person-group> (<year>2006</year>). <article-title>The emergence of life</article-title>, in <source>From Chemical Origins to Synthetic Biology</source> (<publisher-loc>Cambridge</publisher-loc>: <publisher-name>Cambridge University Press</publisher-name>).</citation></ref>
<ref id="B28">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Maes</surname> <given-names>P.</given-names></name></person-group> (<year>1990</year>). <source>Designing Autonomous Agents: Theory and Practice from Biology to Engineering and Back</source>. <publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>MIT Press</publisher-name>.</citation></ref>
<ref id="B29">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Man</surname> <given-names>K.</given-names></name> <name><surname>Damasio</surname> <given-names>A.</given-names></name></person-group> (<year>2019</year>). <article-title>Homeostasis and soft robotics in the design of feeling machines</article-title>. <source>Nat. Mach. Intell</source>. <volume>1</volume>, <fpage>446</fpage>&#x02013;<lpage>452</lpage>. <pub-id pub-id-type="doi">10.1038/s42256-019-0103-7</pub-id></citation></ref>
<ref id="B30">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Maturana</surname> <given-names>H. R.</given-names></name> <name><surname>Varela</surname> <given-names>F. J.</given-names></name></person-group> (<year>1980</year>). <source>Autopoiesis and Cognition: The Realization of the Living</source>. <publisher-loc>Dordrecht</publisher-loc>: <publisher-name>Springer</publisher-name>.</citation></ref>
<ref id="B31">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Maye</surname> <given-names>A.</given-names></name> <name><surname>Engel</surname> <given-names>A. K.</given-names></name></person-group> (<year>2013</year>). <article-title>Extending sensorimotor contingency theory: prediction, planning, and action generation</article-title>. <source>Adapt. Behav</source>. <volume>21</volume>, <fpage>423</fpage>&#x02013;<lpage>436</lpage>. <pub-id pub-id-type="doi">10.1177/1059712313497975</pub-id></citation></ref>
<ref id="B32">
<citation citation-type="journal"><person-group person-group-type="author"><collab>MLPerf</collab></person-group> (<year>2021</year>). <source>MLCommons</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://mlcommons.org/">https://mlcommons.org/</ext-link></citation></ref>
<ref id="B33">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Newell</surname> <given-names>A.</given-names></name> <name><surname>Shaw</surname> <given-names>J. C.</given-names></name> <name><surname>Simon</surname> <given-names>H. A.</given-names></name></person-group> (<year>1958</year>). <article-title>Elements of a theory of human problem solving</article-title>. <source>Psychol. Rev</source>. <volume>65</volume>, <fpage>151</fpage>&#x02013;<lpage>166</lpage>. <pub-id pub-id-type="doi">10.1037/h0048495</pub-id></citation></ref>
<ref id="B34">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Newell</surname> <given-names>A.</given-names></name> <name><surname>Simon</surname> <given-names>H. A.</given-names></name></person-group> (<year>1976</year>). <article-title>Computer science as empirical inquiry: symbols and search</article-title>. <source>Commun. ACM</source> <volume>19</volume>, <fpage>113</fpage>&#x02013;<lpage>126</lpage>. <pub-id pub-id-type="doi">10.1145/360018.360022</pub-id></citation></ref>
<ref id="B35">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Newman</surname> <given-names>S. A.</given-names></name></person-group> (<year>2018</year>). <article-title>Inherency</article-title>, in <source>Evolutionary Developmental Biology</source>, eds <person-group person-group-type="editor"><name><surname>Nuno de la Rosa</surname> <given-names>L.</given-names></name> <name><surname>M&#x000FC;ller</surname> <given-names>G.</given-names></name></person-group> (<publisher-loc>Cham</publisher-loc>: <publisher-name>Springer International Publishing</publisher-name>), <fpage>1</fpage>&#x02013;<lpage>12</lpage>.</citation></ref>
<ref id="B36">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>No&#x000EB;</surname> <given-names>A.</given-names></name></person-group> (<year>2004</year>). <source>Action in Perception</source>. <publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>The MIT Press</publisher-name>.</citation></ref>
<ref id="B37">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Nolfi</surname> <given-names>S.</given-names></name> <name><surname>Bongard</surname> <given-names>J.</given-names></name> <name><surname>Husbands</surname> <given-names>P.</given-names></name> <name><surname>Floreano</surname> <given-names>D.</given-names></name></person-group> (<year>2016</year>). <article-title>Evolutionary robotics</article-title>, in <source>Springer Handbook of Robotics</source> (<publisher-loc>Berlin</publisher-loc>: <publisher-name>Springer</publisher-name>), <fpage>2035</fpage>&#x02013;<lpage>2068</lpage>.</citation></ref>
<ref id="B38">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Norvig</surname> <given-names>P.</given-names></name> <name><surname>Russell</surname> <given-names>S.</given-names></name></person-group> (<year>2021</year>). <source>Artificial Intelligence: A Modern Approach, Global Edition</source>. <publisher-loc>Harlow</publisher-loc>: <publisher-name>Pearson</publisher-name>.</citation></ref>
<ref id="B39">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>O&#x00027;Regan</surname> <given-names>J. K.</given-names></name> <name><surname>No&#x000EB;</surname> <given-names>A.</given-names></name></person-group> (<year>2001</year>). <article-title>A sensorimotor account of vision and visual consciousness</article-title>. <source>Behav. Brain Sci</source>. <volume>24</volume>, <fpage>939</fpage>&#x02013;<lpage>973</lpage>; discussion 973-1031. <pub-id pub-id-type="doi">10.1017/S0140525X01000115</pub-id><pub-id pub-id-type="pmid">12239892</pub-id></citation></ref>
<ref id="B40">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Pfeifer</surname> <given-names>R.</given-names></name> <name><surname>Scheier</surname> <given-names>C.</given-names></name></person-group> (<year>2001</year>). <source>Understanding Intelligence</source>. <publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>MIT Press</publisher-name>.</citation></ref>
<ref id="B41">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Pinker</surname> <given-names>S.</given-names></name></person-group> (<year>1997</year>). <source>How the Mind Works</source>. <publisher-loc>New York, NY</publisher-loc>: <publisher-name>W. W. Norton &#x00026; Company</publisher-name>.</citation></ref>
<ref id="B42">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Putnam</surname> <given-names>H.</given-names></name></person-group> (<year>1965</year>). <article-title>The mental life of some machines</article-title>, in <source>Philosophical Papers: Volume 2, Mind, Language and Reality</source> (<publisher-loc>Cambridge</publisher-loc>: <publisher-name>Cambridge University Press</publisher-name>), <fpage>408</fpage>&#x02013;<lpage>428</lpage>.</citation></ref>
<ref id="B43">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Rosen</surname> <given-names>R.</given-names></name></person-group> (<year>1991</year>). <source>Life Itself, 1st Edn</source>. <publisher-loc>New York, NY</publisher-loc>: <publisher-name>Columbia University Press</publisher-name>.</citation></ref>
<ref id="B44">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rosenblatt</surname> <given-names>F.</given-names></name></person-group> (<year>1958</year>). <article-title>The perceptron: a probabilistic model for information storage and organization in the brain</article-title>. <source>Psychol. Rev</source>. <volume>65</volume>, <fpage>386</fpage>. <pub-id pub-id-type="doi">10.1037/h0042519</pub-id><pub-id pub-id-type="pmid">13602029</pub-id></citation></ref>
<ref id="B45">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ruiz-Mirazo</surname> <given-names>K.</given-names></name> <name><surname>Pereto</surname> <given-names>J.</given-names></name> <name><surname>Moreno</surname> <given-names>A.</given-names></name></person-group> (<year>2004</year>). <article-title>A universal definition of life: autonomy and open-ended evolution</article-title>. <source>Origins Life Evolut. Biospheres</source> <volume>34</volume>, <fpage>323</fpage>&#x02013;<lpage>346</lpage>. <pub-id pub-id-type="doi">10.1023/B:ORIG.0000016440.53346.dc</pub-id><pub-id pub-id-type="pmid">15068038</pub-id></citation></ref>
<ref id="B46">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Rumelhart</surname> <given-names>D. E.</given-names></name> <name><surname>McClelland</surname> <given-names>J. L.</given-names></name> <name><surname>Group</surname> <given-names>P. R.</given-names></name></person-group> (<year>1988</year>). <source>Parallel Distributed Processing, Vol. 1</source>. <publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>MIT Press</publisher-name>.</citation></ref>
<ref id="B47">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Simon</surname> <given-names>J. R.</given-names></name> <name><surname>Wolf</surname> <given-names>J. D.</given-names></name></person-group> (<year>1963</year>). <article-title>Choice reaction time as a function of angular stimulus-response correspondence and age</article-title>. <source>Ergonomics</source> <volume>6</volume>, <fpage>99</fpage>&#x02013;<lpage>105</lpage>. <pub-id pub-id-type="doi">10.1080/00140136308930679</pub-id></citation></ref>
<ref id="B48">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Smithers</surname> <given-names>T.</given-names></name></person-group> (<year>1997</year>). <article-title>Autonomy in robots and other agents</article-title>. <source>Brain Cogn</source>. <volume>34</volume>, <fpage>88</fpage>&#x02013;<lpage>106</lpage>. <pub-id pub-id-type="doi">10.1006/brcg.1997.0908</pub-id><pub-id pub-id-type="pmid">9209757</pub-id></citation></ref>
<ref id="B49">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Steels</surname> <given-names>L.</given-names></name></person-group> (<year>1990</year>). <article-title>Towards a theory of emergent functionality</article-title>, in <source>Proceedings of the First International Conference on Simulation of Adaptive Behavior on From Animals to Animats</source> (<publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>MIT Press</publisher-name>), <fpage>451</fpage>&#x02013;<lpage>461</lpage>.</citation></ref>
<ref id="B50">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Varela</surname> <given-names>F. J.</given-names></name></person-group> (<year>1992</year>). <article-title>Whence perceptual meaning? A cartography of current ideas</article-title>, in <source>Understanding Origins: Contemporary Views on the Origin of Life, Mind and Society, Boston Studies in the Philosophy and History of Science 130, 1st Edn</source>, eds <person-group person-group-type="editor"><name><surname>Dupuy</surname> <given-names>J.-P.</given-names></name> <name><surname>Varela</surname> <given-names>F. J.</given-names></name></person-group> (<publisher-loc>Dordrecht</publisher-loc>: <publisher-name>Springer Netherlands</publisher-name>), <fpage>235</fpage>&#x02013;<lpage>263</lpage>.</citation></ref>
<ref id="B51">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Varela</surname> <given-names>F. J.</given-names></name> <name><surname>Thompson</surname> <given-names>E.</given-names></name> <name><surname>Rosch</surname> <given-names>E.</given-names></name></person-group> (<year>1991</year>). <source>The Embodied Mind : Cognitive Science and Human Experience</source>. <publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>MIT Press</publisher-name>.</citation></ref>
<ref id="B52">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Varela</surname> <given-names>F.</given-names></name> <name><surname>Maturana</surname> <given-names>H.</given-names></name> <name><surname>Uribe</surname> <given-names>R.</given-names></name></person-group> (<year>1974</year>). <article-title>Autopoiesis: The organization of living systems, its characterization and a model</article-title>. <source>Biosystems</source> (<publisher-loc>Cambridge, MA</publisher-loc>). <volume>5</volume>, <fpage>187</fpage>&#x02013;<lpage>196</lpage>. <pub-id pub-id-type="doi">10.1016/0303-2647(74)90031-8</pub-id><pub-id pub-id-type="pmid">4407425</pub-id></citation></ref>
<ref id="B53">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Vernon</surname> <given-names>D.</given-names></name></person-group> (<year>2010</year>). <article-title>Enaction as a conceptual framework for developmental cognitive robotics</article-title>. <source>Paladyn J. Behav. Rob</source>. <volume>1</volume>, <fpage>89</fpage>&#x02013;<lpage>98</lpage>. <pub-id pub-id-type="doi">10.2478/s13230-010-0016-y</pub-id></citation></ref>
<ref id="B54">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Webb</surname> <given-names>B.</given-names></name></person-group> (<year>1995</year>). <article-title>Using robots to model animals: a cricket test</article-title>. <source>Rob. Auton. Syst</source>. <volume>16</volume>, <fpage>117</fpage>&#x02013;<lpage>134</lpage>. <pub-id pub-id-type="doi">10.1016/0921-8890(95)00044-5</pub-id><pub-id pub-id-type="pmid">22315581</pub-id></citation></ref>
<ref id="B55">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Winograd</surname> <given-names>T.</given-names></name> <name><surname>Flores</surname> <given-names>F.</given-names></name></person-group> (<year>1987</year>). <source>Understanding Computers and Cognition</source>. <publisher-loc>Boston, MA</publisher-loc>: <publisher-name>Addison-Wesley Professional</publisher-name>.</citation></ref>
<ref id="B56">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Zarco</surname> <given-names>M.</given-names></name> <name><surname>Egbert</surname> <given-names>M.</given-names></name></person-group> (<year>2019</year>). <article-title>Different forms of random motor activity scaffold the formation of different habits in a simulated robot</article-title>, in <source>Artificial Life Conference Proceedings</source> (<publisher-loc>New York, NY</publisher-loc>: <publisher-name>MIT Press</publisher-name>).</citation></ref>
<ref id="B57">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ziemke</surname> <given-names>T.</given-names></name> <name><surname>Lowe</surname> <given-names>R.</given-names></name></person-group> (<year>2009</year>). <article-title>On the role of emotion in embodied cognitive architectures: from organisms to robots</article-title>. <source>Cogn. Comput</source>. <volume>1</volume>, <fpage>104</fpage>&#x02013;<lpage>117</lpage>. <pub-id pub-id-type="doi">10.1007/s12559-009-9012-0</pub-id></citation></ref>
</ref-list>
<app-group>
<app id="A1">
<title>Appendix A</title>
<fig id="F15" position="float">
<label>Figure A1</label>
<caption><p>Projections of the sensorimotor trajectories of the visual robot. Colors and letters <bold>(B&#x02013;H)</bold> are included to facilitate comparison with <xref ref-type="fig" rid="F3">Figure 3</xref>. <bold>(A)</bold> Shows the superposition of trials <bold>(B&#x02013;H)</bold>.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0015.tif"/>
</fig>
<fig id="F16" position="float">
<label>Figure A2</label>
<caption><p>Projections of the sensorimotor trajectories of the visual robot. Colors and letters <bold>(B&#x02013;H)</bold> are included to facilitate comparison with <xref ref-type="fig" rid="F7">Figure 7</xref>. <bold>(A)</bold> Shows the superposition of trials <bold>(B&#x02013;H)</bold>.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnbot-16-847054-g0016.tif"/>
</fig>
</app></app-group>
</back>
</article>