Front. Ecol. Evol. Frontiers in Ecology and Evolution Front. Ecol. Evol. 2296-701X Frontiers Media S.A. 10.3389/fevo.2021.681704 Ecology and Evolution Review Learning and Animal Movement Lewis Mark A. 1 2 * † Fagan William F. 3 † Auger-Méthé Marie 4 5 Frair Jacqueline 6 Fryxell John M. 7 Gros Claudius 8 Gurarie Eliezer 3 Healy Susan D. 9 Merkle Jerod A. 10 1Department of Mathematical and Statistical Sciences, University of Alberta, Edmonton, AB, Canada 2Department of Biological Sciences, University of Alberta, Edmonton, AB, Canada 3Department of Biology, University of Maryland, College Park, College Park, MD, United States 4Department of Statistics, University of British Columbia, Vancouver, BC, Canada 5Institute for the Oceans and Fisheries, University of British Columbia, Vancouver, BC, Canada 6Department of Environmental and Forest Biology, State University of New York, Syracuse, NY, United States 7Department of Integrative Biology, University of Guelph, Guelph, ON, Canada 8Physics Department, Goethe University, Frankfurt, Germany 9School of Biology, University of St Andrews, St Andrews, United Kingdom 10Department of Zoology and Physiology, University of Wyoming, Laramie, WY, United States

Edited by: Louise Riotte-Lambert, University of Glasgow, United Kingdom

Reviewed by: Mathieu Lihoreau, Centre National de la Recherche Scientifique (CNRS), France; John Fieberg, University of Minnesota System, United States

 *Correspondence: Mark A. Lewis, mark.lewis@ualberta.ca

†These authors share first authorship

This article was submitted to Behavioral and Evolutionary Ecology, a section of the journal Frontiers in Ecology and Evolution

 09 07 2021 2021 9 681704 17 03 2021 16 06 2021 Copyright © 2021 Lewis, Fagan, Auger-Méthé, Frair, Fryxell, Gros, Gurarie, Healy and Merkle. 2021 Lewis, Fagan, Auger-Méthé, Frair, Fryxell, Gros, Gurarie, Healy and Merkle

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

Integrating diverse concepts from animal behavior, movement ecology, and machine learning, we develop an overview of the ecology of learning and animal movement. Learning-based movement is clearly relevant to ecological problems, but the subject is rooted firmly in psychology, including a distinct terminology. We contrast this psychological origin of learning with the task-oriented perspective on learning that has emerged from the field of machine learning. We review conceptual frameworks that characterize the role of learning in movement, discuss emerging trends, and summarize recent developments in the analysis of movement data. We also discuss the relative advantages of different modeling approaches for exploring the learning-movement interface. We explore in depth how individual and social modalities of learning can matter to the ecology of animal movement, and highlight how diverse kinds of field studies, ranging from translocation efforts to manipulative experiments, can provide critical insight into the learning process in animal movement.

 animal cognition decision-making migration reinforcement statistical learning translocation Natural Sciences and Engineering Research Council of Canada10.13039/501100000038 Canada Research Chairs10.13039/501100001804 Introduction

Animal movement, in the form of translocation from one locale to another, takes many forms and is critical to ecological processes. This understanding has given rise to the rapidly growing discipline called movement ecology (Nathan, 2008). Concurrently, the subject of learning has been studied from the perspective of animal behavior, both in the context of ecological interactions and in the context of movement itself (Box 1 and Table 1). Animal behavior has a well-established and celebrated history of understanding learning and there has been recent growth in connecting learning and memory to animal movement behavior (e.g., Fagan et al., 2013). At the same time, a recent explosion of ideas about machine learning is now creating new perspectives on understanding animal movement based on algorithms.

Box 1. Definitions of terms associated with learning.

This box defines terms central to a synthesis of concepts from animal behavior, ecology, psychology, and certain quantitative methods.

Foundational Concepts

Learning:

Psychology-based definition: the cause-effect process leading to information acquisition that occurs as a result of an individual’s experience.

Task-based definition: improved performance for a specific task, based on experience.

Memory: The storage, retention and retrieval of information.

Spatial memory: The memory for where objects/resources/places are in space. Representation of space. Encodes spatial relationships or configurations.

Supervised machine learning: The process by which the machine is trained to perform a task where some input data are already labeled with the correct output. It can be compared to learning in the presence of a supervisor or teacher.

Statistical learning theory: An unsupervised framework for machine learning that deals with the problem of extracting statistically relevant correlations from data.

Modes of Learning

Associative learning: When an animal makes an association between a stimulus and an outcome. Two forms are:

Classical (Pavlovian) conditioning: an animal associates a biologically relevant stimulus (e.g., food) with a previously irrelevant stimulus. For example, a dog presented the sound of a bell rung alongside the presentation of food, will come to salivate at the sound of the bell in the absence of food. Another example would be that a raccoon learns that garbage cans contain food.

Operant (instrumental) conditioning: the behavior of an animal is controlled by the consequences of that behavior. Typically, this behavior develops through sequential reinforcement (e.g., a raccoon learns how to open the garbage can to get food and is rewarded).

Positive reinforcement: Behavior is rewarded and then increases.

Negative reinforcement: Behavior is increased through avoidance of an unpleasant stimulus (also known as instrumental conditioning).

Punishment or Inhibitory learning: Behavior is decreased through avoidance of an unpleasant stimulus. This contrasts with negative reinforcement, where the behavior increases.

Reinforcement learning: From machine learning: The learner is not told which actions to take, but instead must discover which actions yield the most reward by trying them. This is synonymous with trial and error learning. As in optimal foraging in ecology, the focus is on the balance between exploration (of unfamiliar objects/places) and exploitation (of current knowledge).

Online learning: From machine learning: A technique for implementing machine learning based on data becoming available in a sequential order and then being used to update the best predictor for future data at each step.

Habituation: after repeated exposure, an animal decreasingly responds to a stimulus. The stable end state is the animal’s level of tolerance of a stimulus and the outcome is higher tolerance.

Sensitization: after repeated exposure, an animal increasingly responds to a stimulus. The stable end state is the animal’s level of tolerance of a stimulus, and the outcome is decreased tolerance.

Latent learning: an animal learns by gathering and storing information, without immediate reward.

Pathways of Learning

Social learning: Also called “transmission,” this is an umbrella term that includes transfer of skills, concepts, rules and strategies that occur in social contexts and can affect individual behavior. These include:

Social facilitation: An animal has an increased probability of performing a behavior in the presence of a conspecific.

Local enhancement: An individual’s interest in an object or location is mediated by the interest or movement of others.

Imitation: Novel copying of a model behavior through observation that results in a reliably similar outcome.

Cultural transmission: Social transmission leading to the development of traditions that are passed down from generation to generation.

Vertical vs. horizontal learning: Sometimes referred to as parent vs. peer learning, this dichotomy characterizes the generational source of social information.

Information center: Particular locations or events that provide opportunity for information exchange. For example, a community roost may enable individuals to follow well-fed peers to new foraging locations.

Direct information exchange: An animal is provided sender-based, actively communicated information by another individual. For example, honeybees tell their sisters the locations of rewarding flowers.

Optimization-related Terms

Genetic algorithm: A population of candidate solutions to an optimization problem that evolve toward better solutions.

Policy: In machine learning, the mapping of states to actions (e.g., a hungry animal begins to hunt).

Utility function: In machine learning, the assignment of weights or values to agent states. Actions are selected by comparing the values of the predicted states that derive from particular action. For example, a policy involving search vs. sit-and-wait strategies will yield different outcomes for a hungry animal.

Adaptive movement: When animals modify their movement in response to a change. In models, adaptive implies movement behaviors that confer fitness/performance benefits.

Case studies of learning and animal movement.

References Species Spatial processes involved Individual learning vs. social learning Learning in novel or familiar contexts? Learning linked to memory? Simple elapsed time? Juvenile vs. adult compariso?
Barry et al. (2020) Wolves 1. Natal dispersal Individual Novel Y Y N
2. Territory formation
de Grissac et al. (2017) Wandering Albatross Foraging Individual Novel N Y Y
Grecian et al. (2018) Gannets 1. Foraging Elements of both Novel Y N Y
2. Exploration
Leadbeater and Chittka (2009) Bumblebees Foraging Social Novel N Y N
Lihoreau et al. (2012) Bumblebees Foraging Individual Familiar Y Y N
Papastamatiou et al. (2011) Sharks 1. Orientation Individual Familiar Y N Partly
2. Patch use
Scott et al. (2014) Sea turtles 1. Foraging Individual Novel Y Y Partly
2. Migration
Sigaud et al. (2017) Bison 1. Foraging Social Novel Y Y N
2. Patch Use
Teitelbaum et al. (2016) Whooping Cranes 1. Migration Elements of both Familiar Y Y Y
2. Shortstopping
Votier et al. (2017) Gannets 1. Foraging Elements of both Novel Y Y Y
2. Exploration

Along with these recent developments, the ability of ecologists to track animal movements and behaviors remotely in the wild has been steadily increasing. The collection of massive amounts of data on animal movement, primarily via satellite tracking, is now possible at a scale and level of detail previously unimaginable and can be linked with similarly improving remotely sensed or modeled environmental data (e.g., vegetation, anthropogenic disturbance, terrain, NDVI, snow depth) (Kays et al., 2015). Furthermore, more recent advances in bio-logging (e.g., accelerometers, proximity measures, audio-, and video-recording devices) provide direct information on some of the physiological (e.g., jaw movement, heart rate, cortisol, stable isotopes, reproductive status), and social (e.g., interactions with conspecifics), contexts of movements (Wilmers et al., 2015). This coupling of movement patterns with the movement context has created opportunities to infer learning mechanisms and meld ideas from animal behavior, movement ecology, and remote sensing in the context of ecology of learning and animal movement. We develop such a synthesis here.

We start with a focus on learning as a means for acquiring information and making decisions. Employing two related definitions of learning, one from psychology and the other related to computer science, we evaluate the benefits, costs and limitations of learning in the context of animal movement. Next, we address the modality of learning in animal movement, ranging from individual to social. We then develop links to related disciplines: psychology, animal cognition, and machine learning. We close by reviewing approaches to studying the process of learning and animal movement, whether from experimental or observational studies, discussing the role that models can play in this endeavor, and suggesting areas for future developments.

 Individual Information Acquisition and Decision Making Definition of Learning

We start with a psychology-based definition of learning, which states that learning is the information acquisition that occurs via an individual’s experience that results in a detectable and consistent change in neurophysiology and/or behavior (Box 1). Movement intersects with this definition of learning in several key ways. First, movement will give rise to learning if the movement facilitates information acquisition by introducing an animal to a new environment (e.g., information on forage availability) or state (e.g., information from increased vigilance). Second, the learned information can give rise to new movement decisions if the information acquired is used to change movement patterns (e.g., switching to area-restricted search in regions of high forage availability). Lastly, learning can be about movement itself, for example, when an animal learns where and when to migrate by imitating conspecifics (e.g., crane migration). Figure 1 depicts these connections among movement, information processing, the environment, and the internal states of the animal.

A conceptualization of learning in the context of animal movement. An individual’s environment (green, including social context) and its internal state (gray) can both influence the onset of information gathering via the attention that an individual pays to landscape features (arrows 1 and 2, respectively). As currently understood by psychologists, the information gathering pathway involving attention, perception, learning, and memory appears inside the animal’s brain (pink, unlabeled arrows) ultimately providing input to a movement decision (arrow 3). Both the individual’s environment (arrow 4) and its internal state (arrow 5) can then shape and modify the link between memory and movement. The movement decision has ramifications for the environment (arrow 6) and for the internal state (arrow 7). Lastly, the environment can alter an individual’s internal state directly (arrow 8) without invoking information gathering and memory, often via social interactions.

Laboratory studies of learning can be used to seek out direct cellular evidence for neurophysiological changes arising from information acquisition and storage via functional magnetic resonance imaging (Marsh et al., 2010). However, these approaches are impractical in studies of wild animals, for which most ecologically relevant evidence for learning comes from observing changes in behavior as a result of experience. Thus, although the psychology-based definition of learning above does not strictly involve decision-making, the ecological implications of learning are often intimately tied to experience and the decision-making process. This emphasis on process means that movement-related learning is more similar to how machine learning is defined: improved performance for a specific task as a result of prior experience. This definition, which we refer to as the task-based definition, differs from the psychological definition because it is directly tied to experience-based improvements in performance for a specific task (Box 1).

 The Learning Process

The process of learning includes all the steps needed for information acquisition based on experiences encountered. Broadly, these steps include attention to relevant information, perception of the information, acquisition of that information, and, finally, storage, retention, and retrieval (memory) of that information. At this point, the information can be acted upon, for example, to make a movement decision (Figure 1).

Diverse factors may impede or enhance an animal’s attention to information from its environment or from other individuals. For example, animals in unfamiliar environments may be more (or less) observant of environmental cues (Wolfe, 1969) and certain types of social interaction may increase or decrease attentiveness, leading to social learning (Heyes, 1994). Other factors, such as the internal state of an animal (Dorrance and Zentall, 2001) or its risk sensitivity (Bacon et al., 2010) may also play a role in determining attentiveness (Figure 1).

The perception and acquisition of information depend on an animal’s sensory capacities. For most animals, certain sensory cues will be easier to detect than others, which can lead to different hierarchies of inputs, which may be altered contextually. For example, many aural and olfactory cues may be more important than visual information at night (Zollner and Lima, 1999). Once acquired, information must be committed to memory as part of the learning process. Spatially distributed information may be stored as a cognitive map, sometimes in a network-based non-Euclidean format (Noser and Byrne, 2014). Storage and retrieval of learned information is essential for decision making, which can be based on recent events or information from long ago (Polansky et al., 2015; Abrahms et al., 2019).

A test of successful learning is the ability to make a decision using information from past experiences that discriminates among alternative strategies. For example, in laboratory studies, exposure to spatially distributed food rewards in mazes can affect the movement choices of rats (Leonard and McNaughton, 1990). Similarly, for wolves, memory-related statistical metrics like “time since last visit” to a location may form the basis for movement decision discrimination (Schlägel et al., 2017). Of course, this link between experiences and decision making is both complex and context-dependent, being modulated by layers of complexity regarding habitats, social status, and internal states (Figure 1). The so-called diffusion theory for learning posits that the brain does not solve decision-making problems exactly but uses algorithms that optimize the speed and accuracy of choices (Bogacz, 2007).

 Benefits and Costs of Learning

All mobile organisms face a wide variety of spatial challenges that influence individual fitness and present opportunities for decision making shaped by learning. Foraging opportunities and energetic constraints are patchy in space and time, in large part because the underlying physical and biotic processes are also patchy. Optimal foraging theory (McNamara and Houston, 1985; Stephens and Krebs, 1987; Mangel and Clark, 1988) provides a framework for understanding how benefits accrue from foraging in patches that offer the highest returns of energy or nutrient intake per unit time relative to time or energetic costs. Lost opportunities for social interaction, breeding, reproductive care, or shelter, and the risks of mortality due to predation, parasitism, or disease can then be considered.

When the rate of environmental change varies across time and space, as is common along elevation or rainfall gradients, theory suggests an animal may be able to improve its fitness through appropriate patterns of nomadic or migratory movement (e.g., Fryxell and Sinclair, 1988). Field studies support this theory. For example, migratory ungulates can choose patches at a landscape scale that yield appreciable improvement in rates of energy gain, even when such gains are transitory and require continual nomadic repositioning (Fryxell et al., 2004; Holdo et al., 2009). Memory can also influence the choice of movement patterns, such as the balance between range residency and migration (e.g., Shaw and Couzin, 2013). For example, when undergoing seasonal transitions between ranges, migratory ungulates can obtain fitness benefits by remembering previous trajectories (Bracis and Mueller, 2017; Jesmer et al., 2018; Merkle et al., 2019).

Researchers have investigated how learning can influence and confer advantages to moving organisms. Agent-based models of foragers with spatial memory have shown how fitness accrues from moving to acquire reliable information, even when that movement samples sub-optimal patches (Bracis et al., 2015). This is particularly clear when naïve animals are presented with an unfamiliar environment and movement is exploratory. However, even experienced individuals can benefit by spatially sampling a dynamic environment, in particular when resources can be depleted (Boyer and Walsh, 2010) or predation risk can change (Bracis et al., 2018). In this case, movement keeps current the information needed for appropriate decision making.

Given that foraging often results in resource depletion, fitness may also be improved through informed departure criteria based on marginal value leaving rules (Charnov, 1976; Arditi and Dacorogna, 1988; Brown, 1988). The field of “sampling behavior” (Stephens, 1987) extends ideas originally developed within the optimal foraging theory framework, which traditionally assumed that animals are omniscient (Krebs and Inman, 1992; Stephens et al., 2007). One sampling framework considers when animals should visit a patch to assess whether it has changed in value (Green, 1980), whereas another framework focuses on the benefit accrued by tracking a changing environment (Shettleworth et al., 1988). Foragers that sample patches or track changing conditions are learning about the current state of the environment (Stephens, 1987). Informed decision making about which patches to feed in and how long to do so requires reliable expectations regarding resource availability, predation risk, and energetic costs across an individual’s home range, as well as the capacity to estimate these same variables at a given spatial location. For example, primates foraging on fruit track the productivity of different trees and possibly fruit ripeness (Janson and Byrne, 2007). Overall, environmental predictability appears to be essential for the origin and success of movements based on learning and the reshaping of movement strategies based on experience more generally (Mueller et al., 2011; Riotte-Lambert and Matthiopoulos, 2020).

Learning can also help improve fitness even when spatial movement processes are not directly tied to foraging (e.g., territorial defense, migration, reproduction) (Box 2). For example, learning can provide advantages in dominance interactions (Kokko et al., 2006), efficiency of movement (Stamps, 1995), effective escape from predators (Brown, 2001), and large-scale dispersal decisions (Barry et al., 2020), all of which can translate into fitness benefits (Brown et al., 2008; Patrick and Weimerskirch, 2017). For territorial species, learning can influence how conflicts drive pattern formation (Stamps and Krishnan, 1999, 2001; Sih and Mateo, 2001) and alter strategies for territorial defense (Potts and Lewis, 2014; Schlägel and Lewis, 2014; Schlägel et al., 2017). For migratory species, this includes determining least-cost migration corridors between seasonal ranges (Bischof et al., 2012; Poor et al., 2012).

Box 2. Learning and Movement Processes.

Movement is the spatial consequence of a number of different behaviors by animals. For example, a predator searching for predictable but mobile prey must change its location in space to increase the chances it will encounter a prey item. In many situations (e.g., predictable environments or regularly available prey), learning can reduce uncertainty and increase success in such spatial behaviors. We outline a selection of these below:

Search and attack in predation—When prey live in a complex and heterogeneous environment, predators may benefit by adjusting their search and attack behavior over time (Stephens et al., 2007). When predators detect their prey through visual, auditory, or olfactory cues, they can use associative learning to refine their “search image” and improve their ability to detect and attack prey (Ishii and Shimada, 2010). For instance, desert ants (Cataglyphis fortis) use associative learning to connect specific odors to food, and then use this food-odor memory to assist their next foraging journey (Huber and Knaden, 2018).

Escape from a predator—Spending time in familiar space allows animals to learn motor programs that enhance efficient movement within that space (Stamps, 1995). For instance, in response to a pursuing human, Eastern Chipmunks (Tamias striatus) within their home range (i.e., familiar space) take half as much time and travel half as far to reach a refuge compared to when outside their home range (Clarke et al., 1993).

Foraging bouts—An animal’s rate of energy gain while foraging can increase by collecting information about the environment (Stephens and Krebs, 1987), given the environment changes in a (at least somewhat) predictive way. In most of these cases, animals use associative learning to connect the reward of a food source with some aspect (e.g., color, nearby landmark) of that food source. For instance, Rufous Hummingbirds learned the location of flowers that they had emptied in a foraging trial, and in subsequent trials did not waste time visiting them again (Healy and Hurly, 1995).

Navigation and migration—Migratory movements notably occur at spatial scales that greatly exceed perceptual abilities of animals (mammals: Teitelbaum et al., 2015; birds: Alerstam et al., 2003). Thus, it is expected that animal migration is at least partly based on memory of past experience (though some migrations appear to be innate). When migration has a learned component, learning is likely used to improve migratory performance. For instance, social learning of migration helps ungulates improve energy gain (Jesmer et al., 2018) and helps birds reduce costs (Mueller et al., 2013).

Home range or territory selection—The decision process of choosing the size and location of home range or territories can be thought of as a learning process of integrating new information about the distribution of resources of a landscape (Mitchell and Powell, 2004). For instance, home range size is often larger in areas with fewer resources available (e.g., Morellet et al., 2013; Viana et al., 2018). Further, increased exploration events, presumably to sample new locations when others are unavailable, can result in still larger home ranges (Merkle et al., 2015).

While learning may have benefits, acquiring information based on experience does not come without costs. For example, information gathering can require substantial investment in time and/or energy, and may heighten risk (Eliassen et al., 2007) or come at the expense of lost opportunities for foraging, social interaction, or search for suitable breeding sites (Dall et al., 2005). The machinery for learning also exacts an energetic cost (Isler and Van Schaik, 2006; Niven, 2016). Furthermore, retained memories may negatively affect the acquisition of new information, and so there may be a trade-of between memory retention and acquiring new memories (Tello-Ramos et al., 2019).

 Limitations to Measuring Learning From Animal Movement Patterns

Typical methods for recognizing learning in animal movement patterns do not measure the acquisition of information directly but rather rely on the task-based definition of learning, which requires improved performance for a specific task, based on acquired experience (Box 1). There are limitations to such methods, which pose challenges to learning from uncontrolled field-derived data. Unambiguously explaining a particular movement is a general challenge in the study of wildlife, where context, perception, internal states, and particular environmental cues all determine an animal’s response, but are often unobserved. For example, the “time since last visit” behavior in wolves, mentioned above, may not require memory, but could be explained by information from decaying scent marks (Schlägel and Lewis, 2014).

Obvious and obscure alternative explanations to learning and memory must be carefully considered in uncontrolled field studies. Table 2 categorizes a number of movement studies according to the level of evidence for learning—from strong to simply consistent with learning. For each we provide other, non-learning interpretations of the data that cannot be definitively excluded (Table 2).

Mapping empirical examples of learning to machine learning concepts.

Machine learning example Empirical examples
Step AlphaGo Zero (Silver et al., 2017) Hummingbird traplining (Tello-Ramos et al., 2015) Crane migration (Mueller et al., 2013) Experimental elk translocation (Frair et al., 2007) Sheep and moose migration (Jesmer et al., 2018)
Task Win Forage efficiently Migrate efficiently Exploit environment optimally Exploit environment optimally
Experience Repeated play against self Movement within a controlled array of feeders Repeated migration journeys across years Movement away from initial capture/release location Movement and population persistence over decades
Performance measure Victories Path distance per bout Deviations from straight-line migratory path Settlement and survival rate 1. Proportion of green wave exploited 2. Percent of population migrating
Demonstrated improvement over time (or in comparison to benchmark) Increased competitive ranking Decreased length of movement path Decreased length of migratory journey Increased rate of residency Increased migratory tracking and universal migration
Plausible learning mechanisms Reinforcement learning Positive reinforcement 1. Spatial memory 1. Positive reinforcement (forage) 1. Vertical transmission
2. Social learning 2. Negative 2. Positive reinforcement
3. Positive reinforcement (individual moose
reinforcement (predation) foraging)
3. Horizontal social transmission 3. Positive reinforcement (social sheep foraging)
4. Cultural transmission
Alternative explanations Not necessary Controlled experiment Tested and rejected wind-mediated movement and ontogeny Mortality-mediated natural selection 1. Mortality-mediated natural selection
2. Population growth and expansion
3. Kinesis
Evidence of learning? Yes Yes Yes 1. Consistent with learning but not direct evidence.
2. Population-level rather than individual-level
metrics impede direct evidence for learning.
Pathways of Learning for Animal Movement

Individuals can experience or gain information about their environment via different pathways—individually (i.e., by direct interaction with the environment; Dall et al., 2005) or socially (i.e., by observing others; Bandura and Walters, 1963; Rendell et al., 2010)—with learning demonstrated by a change in an individual’s behavior due to its experience (Box 1).

Individual Learning

Much of an animal’s individual learning is associative; that is, the individual learns by making an association between a stimulus and an outcome. Associative learning may arise either from classical (Pavlovian) conditioning, where an animal associates a biologically relevant stimulus (e.g., food) with a previously irrelevant stimulus (e.g., railway tracks), or from operant (instrumental) conditioning, where the behavior of the animal is controlled by the consequences of that behavior (e.g., feeding on grain on tracks leads to a food reward) (Pearce and Bouton, 2001).

These learning processes can make a behavior more likely through positive reinforcement (via rewards) or negative reinforcement (via unpleasant stimuli), or less likely through punishment or inhibitory learning (again, via unpleasant stimuli). For example, a bear foraging on railway tracks (Murray et al., 2017) might be more likely to forage when it finds grain (positive reinforcement) but less likely to forage through negative interactions with moving trains (punishment or inhibitory learning). Additionally, it might increase its vigilance through negative interactions with moving trains (negative reinforcement).

One associative learning mode relevant to animal movement is discrimination learning, where an animal learns to respond differently to distinct stimuli. For example, because homing pigeons can discriminate between the presence and absence of anomalies in magnetic fields, magnetoreception could be used for navigation (Mora et al., 2004).

Two non-associative learning modes that are relevant to movement are habituation (decreased response to a stimulus after repeated exposure) and sensitization (increased response to a stimulus after repeated exposure). These modes depend on the strength of association between stimulus and outcome, rather than the association itself. For example, the sensory responsiveness of honey bees declines after bees receive low sucrose sugar solutions (habituation) and increases after offerings of high sugar solutions (sensitization) (Scheiner, 2004). In turn, the sensory responsiveness of honey bees constrains individual foraging plasticity and skews the collective foraging decisions of colonies (Scheiner, 2004).

Another mode of learning, latent learning, is relevant to animal movement (Franks et al., 2007). Latent learning involves the gathering and storing of information, without immediate reward, such as when animals learn their migration route away from breeding grounds after they are born (e.g., in autumn) and must use that information to return in springtime. Box 1 provides further details on these modes of learning.

 Social Learning

Social learning is an umbrella term for the learning pathway that includes transfer of skills, concepts, rules, and strategies that occur in social contexts and can affect individual behavior. Types of social learning include (i) social facilitation (increased probability of performing a behavior in the presence of a conspecific), (ii) local enhancement (an individual’s interest in an object or location mediates interest/movement by others), and (iii) imitation (novel copying of a model behavior through observation that results in a reliably similar outcome) (Visalberghi and Fragaszy, 1990). Note that these are distinct from the transfer of declarative or procedural information via direct information exchange, such as in bee dancing, to relay information concerning resource locations (Leadbeater and Chittka, 2007).

Each type of social learning is relevant to movement ecology. For example, social facilitation explains bison movement: individuals are more likely to travel to a given new location when in a group where another animal had knowledge of that location (Sigaud et al., 2017). Following behavior occurs in ants where leaders provide guidance to naïve individuals concerning the location of resources (Franks and Richardson, 2006), and in elephants where matriarchs lead herds to waterholes not known to the rest of the group (Fishlock et al., 2016). Imitation can be seen in fish, where translocation experiments demonstrate how naïve individuals learn migration routes through association with experienced individuals (Helfman and Schultz, 1984), as well as in replacement experiments where the long-term re-use of resting and mating sites can be socially learned rather than quality-based (Warner, 1988).

Individual learning can interact with social learning. For example, independent exploration allows ants to improve upon the paths they have learned via social learning through tandem running (Franklin and Franks, 2012). Here, independent exploration is the basis for improvement of route navigation, which can then be distributed within a colony via “information cascades.” More generally, individual learning may be modulated by associational acquisition, where options for individual learning are constrained by the individuals with which an animal associates (Fragaszy and Visalberghi, 2004).

Social learning is emphasized though existing social bonds, such as parent-offspring relationships. For example, elephants will learn resource locations in complex landscapes through both vertical and horizontal transmission (Bowell et al., 1996) and maternal-offspring pairs of whales may complete entire migrations together (Hamilton and Cooper, 2010), thus enhancing the potential for social learning.

However, social learning does not always confer a net benefit (Giraldeau et al., 2002), and may result in costly strategies of movement and resource use (Sigaud et al., 2017). For example, tested alone, adult female guppies that had shoaled with trained conspecifics as they swam to food used the same route used by their trained fellows, even if the route taken by the trained shoal was longer and more energetically costly than were alternative routes (Laland and Williams, 1997; Giraldeau et al., 2002).

 Learning and Space Use: Connections to Other Disciplines

We distinguish two fundamental constructs for learning in conjunction with animal movement: updating the world model and building a new world model. To understand the difference between these, it helps to assume that the animal has a cognitive model of the world (Q^) and a set of “policy rules” (β) for mapping conditions—including the snapshot of that cognitive model and the state or priorities of the animal—into outcomes, in particular movement decisions. The policy rules can be thought of as the coefficients of a function governing outcomes in terms of conditions. Within this construct, updating the world model refers to the process of movement through a world, acquiring and storing information about the world, updating the world model Q^, and acting upon that knowledge according to the fixed set of policy rules β. The learning process itself is limited to updating the world model. Note that this kind of learning is only meaningful if the world itself is dynamic, with resources or threats moving, regenerating, or depleting in a way that makes it necessary to update expectations. When confronted with a new world, either via dispersal, translocation, or a significant perturbation to the existing world, the very structure of the world model and the policy rules both require adjustment by building a new world model. These two fundamental kinds of learning are schematized in Figure 2 where an elk’s movement among three dynamic patches permits constant updating of information (updating the world model), a process with relies on moving between those patches. But when a patch is significantly perturbed, or becomes unusable in a novel way, the fundamental structure of the world needs to be altered (building a new world model), and novel policy rules to govern interaction with novel elements must be developed.

A schematic representation of a forager’s movement rules in a heterogeneous landscape, how a stable set of rules might be applied, and how landscape disturbance could force an update to the movement rules via learning. In a pre-disturbance world (left three columns), the forager (denoted by the white elk symbol) occupies a landscape with three depletable and renewable resource patches and a water body. The “real world” is represented in the top row, with all of its complexity. The second row represents the forager’s model of that world, which distils the complexity to the most relevant information. Shapes indicate different landscape elements, while colors reflect a quantitative score: darker greens are regenerated, paler greens are depleted. The forager has two movement rules in this landscape (bottom row): (1) move from depleted resource patch to a regenerated resource patch and (2) avoid the water body. The pre-disturbance movements rely on a dynamically updated spatial memory, as the forager learns about a changing environment. Post-disturbance, the forager’s world model changes after it gains information about the loss of a potential foraging area, e.g., a new oil well destroys one of the patches. Accordingly, the forager’s world model is refined to include a novel categorical element (orange triangle), with its own avoidance rule for movement (dynamic learning).

The main distinction between updating the world model and building a new world model appears in a slightly different form in the machine-learning literature, where the two kinds of learning are labeled as base-level and meta-level. Specifically, “The base-level learning problem is the problem of learning functions, just like regular supervised learning. The meta-level learning problem is the problem of learning properties of functions, i.e., learning entire function spaces” (Thrun and Pratt, 1998). The function spaces in our analogy comprise Q^, whereas the learning functions are the coefficients β. In the neurosciences, the terms model-based and model-free reinforcement learning are used in analogy with base-level and meta-level learning (Doll et al., 2012).

Cognitive ecologists typically have stringent experimental criteria for identifying learning. For example, experimentation plus control conditions sufficient to rule out alternate explanations are fundamental to confirming the existence of social learning (Reader and Biro, 2010). In this framework, experimentation could involve manipulation of physical aspects of the environment, individual animals via translocations or similar means, or the routes governing social transmission of information. Rare cases where a wild population can be experimentally manipulated provide the strongest cases for demonstrating and parameterizing memory-based movements (Ranc et al., 2020).

It is also interesting to note that complex behaviors that appear to involve decision-making can arise from other mechanisms of self-organized behavior. Self-organization occurs when simple rules lead to emergent behavior (Gros, 2015). A prominent theoretical example is cellular automata whereby a specific rule set, such as “the game of life,” gives rise to agent-like configurations that may travel, replicate, and combine. Self-organized robots (Box 3) can exhibit emergent behavior, such as autonomous direction reversal, which an external observer could mistakenly interpret as decision-making (Kubandt et al., 2019). Because self-organization is not purposeful, an agent solely based on self-organizational principles will not be able to improve, or to “learn” its score in a given task. However, complex, emergent behavior that appears to be adaptable can confound efforts to recognize signals of learning in movement data.

Box 3. Robotics: learning by mobile autonomous agents.

Robots that move and act autonomously, learning as they go, are confronted with tasks that parallel, in some ways, the life needs faced by moving animals. As in living animals, future decisions by a mobile autonomous robot hinge on what the learning robot experiences and encounters. Consequently, it is interesting to investigate how animal decision making about movement (Figure 1) may be understood using concepts commonly used in robotics and control theory (Jordan and Mitchell, 2015).

The basic model of an autonomous learner includes the following ingredients:

The external environment (e.g., spatial locations of forage).

An internal state representation, sometimes termed a world representation (e.g., an individual’s location, energy level and knowledge of forage locations).

A set of possible actions (e.g., foraging strategies).

A policy map that relates state representations to actions (e.g., anticipated energy gain from each foraging strategy).

Information acquisition, which is a consequence of actions interacting with the environment and the state representations (e.g., accumulated information on forage locations).

Value functions that quantify benefits and consequence of actions as represented by the internal states (e.g., benefits and consequences of choosing a foraging strategy, given an individual’s location, energy level and knowledge of forage locations).

A robot’s state representation simplifies all the information in the environment to a manageable (pruned and stylized) subset of relevant information that can eventually be linked to actions. Unsupervised state representations (Lesort et al., 2018) in which there are no performance measures, may be particularly relevant as constructs for how learning operates in animals. State representations allow the policy map to act on a dimensionally reduced decision space (the collection of states), which dramatically simplifies the task of learning individual policies.

A policy map structures the relationship of the robot’s state representation to possible actions. A policy map may be complete, mapping all possible states to actions, or calculated on the run. Monte Carlo tree search, as used in the Go program AlphaGo from Google Deepmind (Silver et al., 2017), determines the next move via an extensive stochastic search. As an additional complication, a robot may possess several policy maps and then select among the alternatives in a rule-based fashion.

Specified in this way, the basic details of a mobile autonomous robot map quite closely onto a formal conceptualization of the learning process in the context of animal movement (Figure 1).

 Machine Learning Approaches

Machine learning tasks involve an explicit goal, such as parameter estimation or classification, and require a clear objective function, such as minimizing a cost function or correctly classifying data. To the extent that animals also have clear objective functions (e.g., ultimately: increasing individual fitness; proximally: eating, avoiding being eaten, reproducing), and that these objectives might be satisfied by performing a specific movement-related task (e.g., selecting appropriate places to forage), it is useful to draw a general analogy between a machine-learning algorithm and an animal that learns. As described above, we use the term task-based learning when referring to this type of process.

Types of Machine Learning

Machine learning has three main learning paradigms: supervised, statistical (unsupervised) learning and reinforcement (Box 1). Training data for supervised learning is labeled with the correct output (Jordan and Mitchell, 2015). However, statistical and reinforcement machine learning do not require labeled training data and thus may be more directly applicable to animal learning. Statistical learning attempts to extract statistically relevant correlations from data (Hastie et al., 2009) whereas reinforcement learning attempts to maximize a cumulative reward through a balance between exploitation of current knowledge and exploration of new strategies (Sutton and Barto, 2017; Box 1).

A wide range of machine learning approaches emphasizes the importance of improvement through experience (Jordan and Mitchell, 2015), which is close to some definitions of animal learning. Good examples are artificial neural networks (ANN), a class of biologically inspired statistical learning algorithms. The input of an ANN, typically the sensory perception of the agent or animal, is propagated through a network of idealized neurons, which can be readjusted by experience-generated reward signals. The sophistication of the ANN can be increased via multiple layers (referred to as deep learning). The output of the ANN induces observable behavior, although it may suffer from overfitting the model to the particular data set at hand. Another way to incorporate the effects of improvement through experience is via evolutionary computing. This method mimics the trial-and-error process of natural evolution, with inheritance, mutation, and crossing over providing the material upon which selection, via reward signals, acts.

The Bayesian probabilistic model for inference provides another perspective on learning. While Bayesian reasoning is most often applied for statistical tasks such as parameter estimation and complex model fitting, it is also a central, probabilistic model for human cognition and learning (Chater et al., 2006; Tenenbaum et al., 2006). In the context of animal movement, prior information represents existing knowledge or existing preference sets (e.g., spatial memory and selection coefficients). Bayesian perspectives readily permit prior knowledge to be updated with new data (experiences) gained by an animal’s movement through the environment. For example, Michelot et al. (2019) draw an explicit analogy between stochastic rule-based animal movement and a Gibbs sampler performing Markov chain Monte Carlo sampling. The resulting posterior distributions accurately reflect the animal’s resource selection function (RSF).

As introduced above, reinforcement learning is a paradigm involving iterated remapping of situations to actions with the goal of maximizing a numerical reward (Sutton and Barto, 2017). Learners are not provided with rules, but must instead employ repeated trials to discover relationships between actions and rewards. This framework has strong parallels to experience-based frameworks for animal learning. Indeed, the temporal difference algorithm from machine learning calculates a reward-prediction error, reflecting how much better the world is than expected (Sutton and Barto, 2017). This algorithm closely resembles the Rescorla-Wagner learning rule (Rescorla and Wagner, 1972), a mainstay from animal learning theory, which posits the change in associative strength during learning is proportional to the difference between the reward received vs. predicted. By way of example, a schematic of the reinforcement optimizer for a computer learning to play the game Go is broadly similar to schematics of animal behavior and learning (Table 2). In both frameworks, an agent takes actions (movements) in the environment, and the outcomes of those actions are processed by an interpreter (cognitive model), which either “rewards” or “punishes” the agent, thereby modifying its internal state and modifying its subsequent actions. Additional aspects of realism are that rewards can be short term or delayed, and that the appropriateness of actions is not provided initially but must be learned via exploration.

Criteria of machine learning applied to animal learning

The machine learning literature provides concrete criteria for identifying if an algorithm has learned (Thrun and Pratt, 1998). Specifically, given (1) a task, (2) training experience, and (3) a performance measure, if performance at the task improves with experience, the algorithm is said to have learned. This is a useful framework for interpreting observational animal movement data. For example, for the sheep and moose in Jesmer et al. (2018) the task was maximizing energy intake and the training experience was several years of moving around the landscape. The performance measure was whether the animals adopted a migratory movement strategy to track variability in energy availability across space and time. Because of an increase in the proportion of migrants in the population over time (and, thereby, an increase in the proportion of individuals with increased energy intake), the animals likely had “learned”. Other instances of mapping empirical examples to machine learning concepts, given in Table 2, include hummingbird traplining, crane migration, and experimental elk translocation.

A major challenge to applying machine learning criteria to moving animals involves identifying the task and performance measure in meaningful ways, given the animals’ spatial context and scale of movement. Survival and reproduction are the ultimate tasks, but foraging, resting, finding a mate, and avoiding predation are all proximal tasks. Nonetheless, the framework helpfully and unambiguously associates movement in the environment with training experience. Table 2 cross-references a machine-learning example with field studies that provided experimental evidence of learning.

Machine learning may suggest new avenues for research in learning and animal movement. Active topics include feature extraction, in which derived values are intended to be informative and non-redundant (for example, preference for exploring as yet unvisited locations in mice or composition of feeding groups in jackdaws), and feature selection, which is the choice of a subset of goal-relevant features (for example, availability of resources for mice or foraging efficiency for jackdaws) (Valletta et al., 2017; Maekawa et al., 2020). These subjects must also play a role in the information processing associated with learning and animal movement; developing the connections may provide new insights.

A particularly interesting learning challenge involves updating the world (as described above) in a familiar rather than novel landscape. For example, in the foraging models of Bracis et al. (2015, 2018), the task is maximization of instantaneous energy intake, the training experience is the movement (together with the acquisition of information for updating the cognitive map), and the performance measure is the amount of forage obtained. This challenge can be connected to that of online statistical machine learning (Box 1), where data become available in a sequential order and are used to update the best predictor for future data at each step.

Could machine learning move beyond an analogy by providing specific hypotheses about the way animals learn to move? It has done so, but the cases are few. By way of example, foraging bumblebees were manipulated in a laboratory environment by presentation with artificial blue and yellow flowers dispensing sucrose solution according to probabilistic reward schedules, and their sampling strategy was compared to the results under the equivalent two-armed bandit reinforcement learning decision rules (Keasar et al., 2002). These decision rules describe optimal behavior of gamblers choosing repeatedly between options that differ in reward probability, without any prior information. In this case, the bees’ behaviors were generally consistent with the decision rule predictions.

 Learning About Learning: Methods and Approaches Experimental vs. Observational Frameworks for Gathering Evidence of Learning in Movement

Researchers have inferred connections between learning and animal movement via classical experiments, observational studies, and translocation/reintroduction efforts. These diverse data types provide distinct insights into how movement can be used to infer learning.

Experimental Studies

Informative experimental studies of learning and movement derive from both field and laboratory settings (Jacobs and Menzel, 2014). Many experimental studies involve insects. Indeed, study of insect navigation propelled much of the early understanding of animal behavior and movement and includes work by Nobel Prize winners Tinbergen and von Frisch. Examples range from moving landmarks to show the effects on navigation to food sources (Wystrach and Graham, 2012) to displacing individuals to show the effects on path integration when returning to an organizing center (Collett and Collett, 2000). Experimental resource manipulations have been used to demonstrate that hummingbirds can learn abstract concepts like spatial position (Henderson et al., 2006) and can encode spatial location on the basis of surrounding landmarks (Flores-Abreu et al., 2012). When applied to roe deer, experimental resource manipulation in a field environment demonstrates that memory, rather than perception, drives foraging decisions (Ranc et al., 2020). Elsewhere, Preisler et al. (2006) tracked elk movements in relation to experimental treatments involving all-terrain vehicles (ATV). They found that elk were more likely to respond to ATVs when on an ATV route, even if the ATV was far away. These data suggest that elk have learned to associate ATV presence with their routes.

In laboratory settings, radial mazes and water mazes (e.g., Leonard and McNaughton, 1990) have been used to study how quickly rodents can learn movement routes and improve their efficiency. Elsewhere, laboratory arenas built for insects have demonstrated that pesticide exposure can impair spatial learning of resource locations by bumblebees (Stanley et al., 2015).

Sometimes field and laboratory experiments can be combined with great benefit, including comparisons among three classic model systems (homing pigeons, bees, and rats; Jacobs and Menzel, 2014). For example, experimental lesioning studies of young homing pigeons, followed by release in unfamiliar areas, demonstrate that immature birds are very good at learning movement routes and that there is a consolidation phase during which experiences (e.g., encounters with landmarks) are neurally encoded (Bingman et al., 2005).

 Observational Studies

To assess learning in observational studies, researchers must analyze how an animal behaves at a given time based on local conditions and past experiences. Observational studies typically record the location of animals and thus their experiences over relatively long time-frames (e.g., multiple years, or entire lifetimes). Remotely sensed geographic and climatological data then provide the local conditions the animal is experiencing during movement. Additional information on the behavioral and physiological states of the animal may also be relevant. Fortunately, the ongoing evolution in remote animal tracking and sensing technology means that researchers are increasingly able to infer physiological and behavioral states over long periods of time (Kays et al., 2015).

Data on repeated movement patterns can help differentiate learning hypotheses. For example, data on repeated migration routes have helped distinguish whether animals follow resource gradients, rely on memory to navigate, or learn from experience to shape their movement decisions (Mueller et al., 2013; Merkle et al., 2019). However, long-term tracking data may also be sufficient for analysis. For example, wolf movement data have identified how animals follow resource levels, but that they may also rely on the memory of time since last visit to a location (Schlägel et al., 2017). Augmenting tracking data with information that the animals might gather, for example the location of kill sites (Gurarie et al., 2011) or profitable forage patches (Merkle et al., 2014), can further enhance our understanding of how animals monitor their environment (Gurarie et al., 2011).

Comparative studies can be useful for identifying instances of learning. For example, comparing the movement efficiency of juveniles and adults shows that seabirds start by exploring their landscape and then learn to identify the good foraging areas and cues as adults (de Grissac et al., 2017; Votier et al., 2017; Grecian et al., 2018; Wakefield et al., 2019). Effects of early-life experience can be identified by analyzing the site fidelity of animals to their breeding ground (Weinrich, 1998) and by comparing the migration patterns of offspring to those of their mother’s (Colbeck et al., 2013). Finally, comparing the movement of cultural groups, especially if sympatric, can help to assess the effect of culturally transmitted information on animals’ space use (Kendal et al., 2018; Owen et al., 2019).

 Translocations and Reintroductions

Some management actions involve human-aided displacements of animals, either from captivity (reintroductions) or from wild populations (translocations). Tracking the animals released in such manipulations can provide unique opportunities to understand how the animals adapt to their new environments (He et al., 2019). For example, recurring short displacements (such as when animals are repeatedly taken to the same sampling station for physiological samples), can be used to assess how quickly the animal learns the return route to its home range (Biro et al., 2007).

Translocations of animals into existing populations can aid understanding of learning when movement behaviors of individuals new to the environment can be compared to those of already-resident individuals. For example, quantifying the rate of convergence of movement metrics between new arrivals and residents could help estimate learning rates. In addition, if translocated animals, such as elk, are sourced from areas that differ in predation risk (or other factors) but released in a common space, comparison of the survival and movement patterns could be useful to understanding how previous experience shapes learning (Frair et al., 2007). Translocations of social animals may also create opportunities for newly arrived individuals to learn from resident conspecifics (Dolev et al., 2002).

Overall, comparing movements of animals in novel environments over years or even generations with historical populations can reveal learning and cultural transmission and identify the rate at which animals gain knowledge. For example, Jesmer et al. (2018) found that it took multiple decades for translocated bighorn sheep and moose to regain the capacity to identify and follow the optimal forage gradients that existed in their landscapes as they migrated. Likewise, tracking the movement of prey species before and after the introduction of predators into a landscape affords unique opportunities for investigating how animals learn to avoid predators (Ford et al., 2015).

 Uncontrolled Experiments

Beyond intentional displacements, other management actions can serve as uncontrolled experiments for learning. For example, aversive conditioning, which is routinely used in wildlife conflict management, could provide guidance on the mode of learning (Bejder et al., 2009) and may provide contrast the efficacy of different deterrence systems. For example, Ronconi and Clair (2006) showed that presence-activated deterrent systems were more useful than were randomly activated systems for limiting the landing of waterfowl on tailing ponds from oil extraction. Likewise, fences involving bee hives were more likely to turn away elephants than were bush fences (King et al., 2011) and problem elk repeatedly chased by humans and dogs stayed further from town (Kloppers et al., 2005).

Rapid changes in habitat can also serve as uncontrolled experiments. For example, because ungulates will select recently burned areas (Allred et al., 2011), monitoring animal movement in fire-prone systems could help understand how these animals learn about and navigate to novel habitats. Studying movement in the vicinity of new obstacles (e.g., pipelines and roads) and passageways (e.g., road-crossing structures) could help to understand how animals change their spatial patterns as they learn to circumvent barriers and make use of new structures (McDonald and Clair, 2004; Ford and Clevenger, 2018).

 Identifying and Characterizing Learning

Analytical and computational tools have a special role to play in the context of learning and animal movement. They can be used both to develop new theory, and in inference regarding actual movement behaviors.

Modeling Frameworks for Exploring How Learning Operates

Dynamical systems models are often used to investigate learning and animal movement in a purely theoretical context (Table 3). The most common purpose is to investigate possible emergent patterns, which arise from the inclusion of learning in movement models. Here spatial location and spatial memory are given by variables that change in time and space, and dynamical rules postulate how these variables could change through the interplay of movement and learning. The actual form of the dynamical systems ranges from difference equations used to analyze home ranges (van Moorter et al., 2009), to “record-keeping” models of cognitive maps based on incremental experiences (Spencer, 2012), to partial differential equations used to analyze searching ability (Berbert and Lewis, 2018) to stochastic processes used to investigate patrolling ability (Schlägel and Lewis, 2014). Agent-based simulations have also been used to track the development of complex spatial movement behaviors via learning (Tang and Bennett, 2010; Avgar et al., 2013). A review of the ways in which decisions can be integrated into agent-based models is given in DeAngelis and Diaz (2018). Often a balance is required between current perceptual information vs. memories of long-term averages and between random exploration vs. determinism when exploiting resources (see Boyer and Walsh, 2010; Bracis and Mueller, 2017). When it comes to the sharing of information between individuals, ephemeral public information about resource locations can lead to permanent aggregations of memory-based foragers that move via circuits (traplines) (Riotte-Lambert and Matthiopoulos, 2019), and sometimes the rules for near-optimal traplines can be developed based on simple heuristics (Lihoreau et al., 2013). Theoretical studies can investigate relationships or feedbacks between movement and learning that generate patterns similar those seen in nature. They can also be used to explore the environments in which learning might confer benefits. Intriguingly, in the face of an uncertain heterogeneous environment, it may be better for individuals to overestimate environmental quality, as optimistic animals can learn the true value of the environment faster, allowing for a higher rate of exploration (Berger-Tal and Avgar, 2012). Theoretical explorations are particularly useful for studying the updating the world model type of learning, where it is more difficult to make a clear distinction between precipitating events of experiences and movement outcomes in observational data.

Models for learning and animal movement.

Step Bracis et al., 2015 Merkle et al., 2017 Avgar et al., 2016 Schlägel et al., 2017
Task 1. Maximize consumption 2. Reduce predation Forage efficiently Forage efficiently and survive Patrol
Experience Movement Movement among patches Movement Movement
Model prediction Consumption and predator encounter rate Patch selection Redistribution kernel Entire movement path
Null model Context-dependent behavioral switching Connectivity, size, and quality of patch Forage quality, predation risk, competitors, and snow 1. Movement in response to prey density
2. Distance to territory boundary
Information updated Location and quality of forage and encounters 1. Location and quality of patches Location and quality of habitat Time since last visit to territorial locations
2. Memory of past patch quality
Improvement via learning Learning forager outperforms null model Learning forager is more efficient Yes Yes
Plausible connections to fitness 1. Foraging efficiency Past experience leads to foraging in higher quality patches Past experience leads to better habitat use Territorial maintenance and defense
2. Reducing encounters with predators
Plausible learning mechanism Sampling and trial-and-error plus reinforcement Positive reinforcement Positive reinforcement Positive reinforcement

Machine learning is emerging as a powerful paradigm for the analysis of many biological systems. In the context of learning and animal movement, these approaches can map environmental conditions to movement behavior outcomes without necessarily investigating the learning process itself (see, for example, Mueller et al., 2011; Wijeyakulasuriya et al., 2020). Furthermore, as described earlier, machine learning can serve as prototype models for the process of animal learning itself.

 Testing for Change Over Time in Key Movement Metrics

Across diverse data types, a key indicator of learning is a change quantified as a function of “time in the environment” (Figure 3). While not sufficient to say confidently that learning has occurred, a strong signal that an animal’s movement behavior has changed with experience suggests that it is learning. For example, the range occupied by a group of newly translocated animals would be expected to stay very close to their point of release as they focus on learning attributes of their new environment, but wander more widely as time since release increases as they start to exploit their new environment more widely (e.g., total daily displacement, He et al., 2019). It has been proposed that Lévy walks may arise from a learning process wherein animals attempt to learn optimally from their environment (Namboodiri et al., 2016). In this situation the change from simple random (Brownian) motion to a Lévy walk pattern of movement could be interpreted as learning (but see, for example, Benhamou and Collet, 2015 for a critique of this type of formalism).

Exemplar movement patterns associated with learning. We represent clusters of movement activity as squiggles and range displacement events as periods of directed motion. In each of the three examples, the process of learning alters the pattern of movement in a statistically detectable manner. Exploration becomes exploitation through repeated visitation (top row). Conditioned responses to habitat elements may manifest as before / after displacement events (middle row). Information gathering during a juvenile (or otherwise naïve) phase may yield improved efficiency of travel. In all three examples, one or more key metrics will exhibit time-dependence (right column).

Decreases in the rate of range expansion over time indicates that translocated individuals may have learned to favor certain parts of the landscape. In this case, exploration shifts to an exploitation phase (Berger-Tal et al., 2014) as translocated animals exhibit a greater probability of revisiting previously visited areas in a goal-directed manner (Figure 3, top row), and may ultimately establish home-ranges (Moorcroft and Lewis, 2006). Similarly, exposure to a hostile landscape element (e.g., human habitation) may condition wild animals to avoid such elements, altering their spatial distribution to favor locations far from habitation (Figure 3, middle row). This issue has been particularly well-investigated with elephants (Hoare and Du Toit, 1999; Cheptou et al., 2017).

Animals that “sample” different landscapes during exploratory movements may ultimately settle in landscapes featuring the kinds of elements they encountered and exploited during the exploration phase. This can occur during dispersal, during which animals effectively sample and make decisions in an environment about which they are completely naïve. Wolves have been shown to show less avoidance of human elements, in particular relatively little-used forest roads, in new territories after a greater level of exposure and use during a dispersal phase, suggesting that they might have learned that the benefits of using those human elements outweigh the risks (Barry et al., 2020). Translocation, which can be considered an artificial and more abrupt dispersal, also requires decision making in novel environments. Changes in movement behavior (and improved survival) were recorded following translocation of naïve elk from a savannah landscape in Alberta to a forested landscape in eastern Canada (Fryxell et al., 2008).

Migration can also feature time-dependence in characteristics of movement (Figure 3, bottom row). For example, both Mueller et al. (2013) and Jesmer et al. (2018) report changes in migration performance as a function of animals’ time in a landscape (Table 2). On smaller scales, foraging journeys from a central place and other kinds of daily activity patterns can show the same kind of performance gains (e.g., reduced tortuosity) as a function of experience or age (Franklin and Franks, 2012; de Grissac et al., 2017; Votier et al., 2017; Wakefield et al., 2019; Table 1). Resulting spatial patterns of movement can be complex, exhibiting increased speed and goal-directedness (Noser and Byrne, 2014) and even providing evidence of future-oriented cognitive mechanisms (Janmaat et al., 2014). Emerging patterns may include periodic recursions (Riotte-Lambert et al., 2013) as well as sequential movements, where locations are revisited in a regular order (Ayers et al., 2015, 2018; De Groeve et al., 2016; Riotte-Lambert et al., 2017).

 Statistical Inference to Identify Learning in Movement Processes

Analytical and computational tools may also be used to infer learning processes from data. For example, the step-selection function (SSF, Fortin et al., 2005) is of particular utility when it is connected to regular samples of location data and allows for inference of movement parameters that depend on different habitat types. Computationally efficient approaches such as integrated step selection analysis (iSSA) (Avgar et al., 2016), provide practitioners a straightforward way to evaluate movement decisions against actual observations. A generalized form of the SSF, termed the coupled SSF (Potts et al., 2014), allows for the inclusion of memory and past social interactions. Here memory and past interactions can be included into the model, as one or more spatio-temporal maps, sometimes referred to as cognitive maps. Although superficially similar to a changing habitat layer, the contents of the cognitive maps are particular to each individual as they are populated by information gleaned from the individual’s past experiences (Fagan et al., 2013). With such an SSF, one can test how the individual’s movement behavior is governed by cognitive maps whose contents arise from different types of memories or social interactions. Coupled SSFs have been used to test for evidence of memory (Polansky et al., 2015; Oliveira-Santos et al., 2016; Schlägel et al., 2017) and learning (Merkle et al., 2014) in animal movement patterns.

Analysis via SSF assumes that animals’ location data are known without error. If error is significant, as it can be for marine systems, a different class of model, known as state space models, are needed. State space models are hierarchical and feature separate models for the movement process and the measurement error process. These models can be modified to include a hidden Markov process, whose latent state is determined by physiological status (e.g., searching or traveling) or by learning (Avgar et al., 2016). Such models, while flexible, may suffer from parameter estimability issues (Auger-Méthé et al., 2016) and must be implemented with care.

 Conclusion and New Horizons

Traditionally, studies of animal learning and movement have taken place in controlled laboratory environments or small-scale field studies. Thanks to animal tracking technologies, increasingly detailed observations of how free-ranging animals move and interact are possible, leading to opportunities to formulate and test new ideas about learning and movement. We summarize a variety of outstanding new opportunities as grand challenges in Box 4. However, potential pitfalls accompany this exciting development. Alternative explanations to learning must be considered, and if these alternatives cannot be ruled out, then we can only infer that observations are consistent with learning (Table 2).

Box 4. Grand challenges in the study of animals learning to move.

How animals learn to move in novel environments. As a key form of experimental manipulation on animals in the wild, translocations and reintroductions have provided unique insights into the role of social learning of migration and the time-lags required to re-establish migration routes (Mueller et al., 2013; Jesmer et al., 2018). By designing efforts to collect pre-translocation movement that could be compared with post-release data would allow insight into the ways animals learn to move in novel environments.

Social learning. Social learning is particularly hard to study in the context of animal movement because it requires simultaneous information on the location of multiple individuals (Fragaszy and Visalberghi, 2004; Sigaud et al., 2017). One promising approach for studies in this area involves the deployment of animal tracking collars with proximity detectors that can be used to characterize and quantify how known individuals spend their time near or far from other known individuals.

Near-term prediction of movement. Successful prediction of movement, even over modest time horizons of one or a few days, requires a strong, probabilistic representation of animals’ decision-making process. With such a representation in hand it would become possible to gauge how novel experiences shape subsequent movements.

Understanding fitness consequences of learning on population interactions. Learning about movement affects interactions with other individuals (conspecifics, predator, prey and so forth), as well as with the environment. While much has been done to connect individual learning to the environment via optimal foraging (Stephens and Krebs, 1987) there is not yet a comprehensive theory for the influence of learning about movement on population level interactions and the subsequent impacts of these interactions on individual fitness. A natural place to start investigating these feedbacks would be social insects.

Machine learning as a source for new testable hypotheses regarding animal learning and movement. This contrasts with simply providing an interesting analogy for the learning process. While the multi-armed bandit problem has been applied as a model for insect foraging (e.g., Keasar et al., 2002), there are few other cases. However, ML algorithms (for example, K nearest neighbors, decision trees) provide intriguing hypotheses for how learning could proceed. A good place to start would be to build on connections between the theory of ML and the theory of learning, such as the similarity of the reward-prediction error rules in the temporal difference algorithm from machine learning calculates (Sutton and Barto, 2017) and the Rescorla-Wagner learning rule in cognitive science (Rescorla and Wagner, 1972). To date, little has been done on applying machine learning as a source for new testable hypotheses regarding animal learning and movement, but this is an intriguing area for future research.

There are two possible approaches to solving this problem. First, field observations can be transformed into controlled experiments via manipulations, as in the hummingbird example in Table 2. While allowing for incisive analysis, this approach limits the scientific questions to those where such experiments can be set up. A second possible solution is to collect more direct data on the individual experiences over a life-time, including the environmental features of locations animals visit, physiological measurements, and sensory data as made possible by daylight sensors and collar cameras.

Exciting approaches to studying learning and animal movement arise from “uncontrolled” experiments, specifically translocations, reintroductions, aversive conditioning, and rapid environmental change. Understanding learning in the context of relocations and environmental change may ultimately help with understanding how animals can adapt to an increasingly complex world, driven by elevated levels of anthropogenic impacts.

The emergence of machine learning as a dominant paradigm for solving human problems provides fertile ground for modeling and understanding learning from animal movement patterns. Here, processes such as reinforcement learning have close natural ties to animals learning to move to maximize fitness (e.g., optimal foraging). As machine learning algorithms are currently improving and evolving, we expect this field to shed light on further possible models for learning and animal movement. However, as described in the fifth Grand Challenge of Box 4, machine learning has yet to meet its full promise as a reliable source for new testable hypotheses regarding animal learning and movement. This is despite the recognition that animal cognition and communication can be closely tied to computational models (Ma, 2015) and that behavioral decisions can often be best formulated by simple algorithmic models (heuristics) (Hutchinson and Gigerenzer, 2005).

Overall, the subject of learning and animal movement is at a crucial point in development and a host of new possibilities are on the horizon. Our goal in this review has been to set the context for these new possibilities and point out some future directions.

 Author Contributions

ML and WF designed and organized the review, secured the funding, and led writing of the manuscript. MA-M, JF, JMF, CG, EG, SH, and JM contributed to the review and helped writing the manuscript. All authors contributed to the article and approved the submitted version.

 Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding. We acknowledge the following grants for supporting this research: NSERC Discovery (ML and MA-M), NSF DMS-1853465 (WF and EG), and Canada Research Chairs Program (ML and MA-M).

We thank Banff International Research Station (BIRS) for hosting a Focused Research Group on Learning and Animal Movement in the spring of 2019. BIRS is an inspiring environment for research, with foraging elk right outside the window, and is where the review manuscript was conceived and started. We thank the reviewers and Handling Editor for helpful comments and suggestions, which improved the manuscript. O. Couriot, T. Hoffman, A. Swain, Y. Salmaniw, P. Thompson, and H. Wang provided helpful feedback on the manuscript. We thank K. Budinski for help with the references and citations.

 References Abrahms B. Hazen E. L. Aikens E. O. Savoca M. S. Goldbogen J. A. Bograd S. J. (2019). Memory and resource tracking drive blue whale migrations. Proc. Natl. Acad. Sci.U.S.A. 116 5582–5587. 10.1073/pnas.1819031116 30804188 Alerstam T. Hedenstrom A. Akesson S. (2003). Long-distance migration: evolution and determinants. Oikos 103 247–260. 10.1034/j.1600-0706.2003.12559.x 11841302 Allred B. W. Fuhlendorf S. D. Engle D. M. Elmore R. D. (2011). Ungulate preference for burned patches reveals strength: of fire-grazing interaction. Ecol. Evol. 1 132–144. 10.1002/ece3.12 22393490 Arditi R. Dacorogna B. (1988). Optimal foraging on arbitrary food distributions and the definition of habitat patches. Am. Nat. 131 837–846. 10.1086/284825 Auger-Méthé M. Field C. Albertsen C. M. Derocher A. E. Lewis M. A. Jonsen I. D. (2016). State-space models’ dirty little secrets: even simple linear Gaussian models can have estimation problems. Sci. Rep. 6:26677. Avgar T. Deardon R. Fryxell J. M. (2013). An empirically parameterized individual based model of animal movement, perception, and memory. Ecol. Modell. 251 158–172. 10.1016/j.ecolmodel.2012.12.002 Avgar T. Potts J. R. Lewis M. A. Boyce M. S. (2016). Integrated step selection analysis: bridging the gap between resource selection and animal movement. Methods Ecol. Evol. 7 619–630. 10.1111/2041-210x.12528 Ayers C. A. Armsworth P. R. Brosi B. J. (2015). Determinism as a statistical metric for ecologically important recurrent behaviors with trapline foraging as a case study. Behav. Ecol. Sociobiol. 69 1395–1404. 10.1007/s00265-015-1948-3 Ayers C. A. Armsworth P. R. Brosi B. J. (2018). Statistically testing the role of individual learning and decision-making in trapline foraging. Behav. Ecol. 29 885–893. 10.1093/beheco/ary058 Bacon I. E. Hurly T. A. Healy S. (2010). Both the past and the present affect risk-sensitive decisions of foraging rufous hummingbirds. Behav. Ecol. 21 626–632. 10.1093/beheco/arq031 Bandura A. Walters R. H. (1963). Social Learning and Personality Development. New York NY: Holt Rinehart and Winston. Barry T. Gurarie E. Cheraghi F. Kojola I. Fagan W. F. (2020). Does dispersal make the heart grow bolder? Avoidance of anthropogenic habitat elements across wolf life history. Anim. Behav. 166 219–231. 10.1016/j.anbehav.2020.06.015 Bejder L. Samuels A. Whitehead H. Finn H. Allen S. (2009). Impact assessment research: use and misuse of habituation, sensitisation and tolerance in describing wildlife responses to anthropogenic stimuli. Mar. Ecol. Prog. Ser. 395 177–185. 10.3354/meps07979 Benhamou S. Collet J. (2015). Ultimate failure of the lévy foraging hypothesis: two-scale searching strategies outperform scale-free ones even when prey are scarce and cryptic. J. Theor. Biol. 387 221–227. 10.1016/j.jtbi.2015.09.034 26463680 Berbert J. M. Lewis M. A. (2018). Superdiffusivity due to resource depletion in random searches. Ecol. Complex. 33 41–48. 10.1016/j.ecocom.2017.11.005 Berger-Tal O. Nathan J. Meron E. Saltz D. (2014). The exploration-exploitation dilemma: a multidisciplinary framework. PLoS One 9:e0119116. 10.1371/journal.pone.0119116 25742565 Berger-Tal O. Avgar T. (2012). The glass is half-full: overestimating the quality of a novel environment is advantageous. PLoS One 7:e34578. 10.1371/journal.pone.0034578 22509326 Bingman V. P. Gagliardo A. Hough G. E. Ioalé P. Kahn M. C. Siegel J. J. (2005). The avian hippocampus, homing in pigeons and the memory representation of large-scale space. Integr. Comparat. Biol. 45 555–564. 10.1093/icb/45.3.555 21676801 Biro D. Freeman R. Meade J. Roberts S. Guilford T. (2007). Pigeons combine compass and landmark guidance in familiar route navigation. Proc. Natl. Acad. Sci.U.S.A. 104 7471–7476. 10.1073/pnas.0701575104 17452634 Bischof R. Loe L. E. Meisingset E. L. Zimmermann B. van Moorter B. Mysterud A. (2012). A migratory northern ungulate in the pursuit of spring: jumping or surfing the green wave? Am. Nat. 180 407–424. 10.1086/667590 22976006 Bogacz R. (2007). Optimal decision-making theories: linking neurobiology with behaviour. Trends Cogn. Sci. 11 118–125. 10.1016/j.tics.2006.12.006 17276130 Bowell R. J. Warren A. Redmond I. (1996). Formation of cave salts and utilization by elephants in the Mount Elgon region. Kenya. Geol. Soc. London, Spec. Publ. 113 63–79. 10.1144/gsl.sp.1996.113.01.06 Boyer D. Walsh P. D. (2010). Modelling the mobility of living organisms in heterogeneous landscapes: does memory improve foraging success? Philos. Trans. R. Soc. A 368 5645–5659. 10.1098/rsta.2010.0275 21078640 Bracis C. Gurarie E. Van Moorter B. Goodwin R. A. (2015). Memory effects on movement behavior in animal foraging. PLoS One 10:e0136057. 10.1371/journal.pone.0136057 26288228 Bracis C. Gurarie E. Rutter J. D. Goodwin R. A. (2018). Remembering the good and the bad: memory-based mediation of the food–safety trade-off in dynamic landscapes. Theor. Ecol. 11 305–319. 10.1007/s12080-018-0367-2 Bracis C. Mueller T. (2017). Memory, not just perception, plays an important role in terrestrial mammalian migration. Proc. R. Soc. B Biol. Sci. 284:20170449. 10.1098/rspb.2017.0449 28539516 Brown J. S. (1988). Patch use as an indicator of habitat preference, predation risk, and competition. Behav. Ecol. Sociobiol. 22 37–47. 10.1007/bf00395696 Brown C. (2001). Familiarity with the test environment improves escape responses in the crimson spotted rainbowfish, Melanotaenia duboulayi. Anim. Cogn. 4 109–113. 10.1007/s100710100105 Brown C. R. Brown M. B. Brazeal K. R. (2008). Familiarity with breeding habitat improves daily survival in colonial cliff swallows. Anim. Cogn. 76 1201–1210. 10.1016/j.anbehav.2008.03.028 19802326 Charnov E. L. (1976). Optimal foraging, the marginal value theorem. Theor. Popul. Biol. 9 129–136. 10.1016/0040-5809(76)90040-x Chater N. Tenenbaum J. B. Yuille A. (2006). Probabilistic models of cognition: conceptual foundations. Trends Cogn. Sci. 10 287–291. 10.1016/j.tics.2006.05.007 16807064 Cheptou P. O. Hargreaves A. L. Bonte D. Jacquemyn H. (2017). Adaptation to fragmentation: evolutionary dynamics driven by human influences. Proc. R. Soc. B Biol. Sci. B Biol. Sci. 372:20160037.10.1098/rstb.2016.0037 27920382 Clarke M. F. da Silva K. B. Lair H. Pocklington R. Kramer D. L. McLaughlin R. L. (1993). Site familiarity affects escape behaviour of the eastern chipmunk. Tamias Striatus. Oikos 66 533–537. 10.2307/3544949 Colbeck G. J. Duchesne P. Postma L. D. Lesage V. Hammill M. O. Turgeon J. (2013). Groups of related belugas (Delphinapterus leucas) travel together during their seasonal migrations in and around Hudson Bay. Proc. R. Soc. B Biol. Sci. 280:20122552. 10.1098/rspb.2012.2552 23222451 Collett M. Collett T. S. (2000). How do insects use path integration for their navigation? Biol. Cybern.s 83 245–259. 10.1007/s004220000168 11007299 Dall S. R. X. Giraldeau L. A. Olsson O. McNamara J. M. Stephens D. W. (2005). Information and its use by animals in evolutionary ecology. Trends Ecol. Evol. 20 187–193. 10.1016/j.tree.2005.01.010 16701367 DeAngelis D. L. Diaz S. G. (2018). Decision-making in agent-based modeling: a review and future prospectus. Front. Ecol. Evol. 6:237. De Groeve J. Van de Weghe N. Ranc N. Neutens T. Ometto L. Rota-Stabelli O. (2016). Extracting spatio-temporal patterns in animal trajectories: an ecological application of Sequence Analysis Methods (SAM). Methods Ecol. Evol. 7 360–379. Dolev A. Saltz D. Bar-David S. Yom-Tov Y. (2002). Impact of repeated releases on space-use patterns of Persian fallow deer. J. Wildl. Manage. 66:737. 10.2307/3803139 Doll B. B. Simon D. A. Daw N. D. (2012). The ubiquity of model-based reinforcement learning. Curr. Opin. Neurobiol. 22 1075–1081. 10.1016/j.conb.2012.08.003 22959354 Dorrance B. R. Zentall T. R. (2001). Imitative learning in Japanese quail (Coturnix japonica) depends on the motivational state of the observer quail at the time of observation. J. Comp. Psychol. 115 62–67. 10.1037/0735-7036.115.1.62 11334220 Eliassen S. Jørgensen C. Mangel M. Giske J. (2007). Exploration or exploitation: life expectancy changes the value of learning in foraging strategies. Oikos 116 513–523. 10.1111/j.2006.0030-1299.15462.x Fagan W. F. Lewis M. A. Auger-Méthé M. Avgar T. Benhamou S. Breed G. (2013). Spatial memory and animal movement. Ecol. Lett. 16 1316–1329. Fishlock V. Caldwell C. Lee P. C. (2016). Elephant resource-use traditions. Anim. Cogn. 19 429–433. 10.1007/s10071-015-0921-x 26359083 Flores-Abreu I. N. Hurly T. A. Healy S. D. (2012). One-trial spatial learning: wild hummingbirds relocate a reward after a single visit. Anim. Cogn. 15 631–637. 10.1007/s10071-012-0491-0 22526688 Ford A. T. Goheen J. R. Augustine D. J. Kinnaird M. F. O’Brien T. G. Palmer T. M. (2015). Recovery of African wild dogs suppresses prey but does not trigger a trophic cascade. Ecology 96 2705–2714. 10.1890/14-2056.1 Ford A. T. Clevenger A. P. (2018). Factors affecting the permeability of road mitigation measures to the movement of small mammals. Can. J. Zool. 97 379–384. 10.1139/cjz-2018-0165 33356898 Fortin D. Beyer H. L. Boyce M. S. Smith D. W. Duchesne T. Mao J. S. (2005). Wolves influence elk movements: behavior shapes a trophic cascade in Yellowstone National Park. Ecology 86 1320–1330. 10.1890/04-0953 Fragaszy D. Visalberghi E. (2004). Socially biased learning in monkeys. Learn. Behav. 32 24–35. 10.3758/bf03196004 15161138 Frair J. L. Merrill E. H. Allen J. R. Boyce M. S. (2007). Know thy enemy: experience affects elk translocation success in risky landscapes. J. Wildl. Manage. 71 541–554. 10.2193/2006-141 Franklin E. L. Franks N. R. (2012). Individual and social learning in tandem-running recruitment by ants. Anim. Behav. 84 361–368. 10.1016/j.anbehav.2012.05.002 Franks N. R. Hooper J. W. Dornhaus A. Aukett P. J. Hayward A. L. Berghoff S. M. (2007). Reconnaissance and latent learning in ants. Proc. R. Soc. B Biol. Sci. 274 1505–1509. 10.1098/rspb.2007.0138 17426016 Franks N. R. Richardson T. (2006). Teaching in tandem-running ants. Nature 439 153–153. 10.1038/439153a 16407943 Fryxell J. M. Sinclair A. R. E. (1988). Causes and consequences of migration by large herbivores. Trends Ecol. Evol. 3 237–241. 10.1016/0169-5347(88)90166-8 Fryxell J. M. Wilmshurst J. F. Sinclair A. R. E. (2004). Predictive models of movement by serengeti grazers. Ecology 85 2429–2435. 10.1890/04-0147 Fryxell J. M. Hazell M. Börger L. Dalziel B. D. Haydon D. T. Morales J. M. (2008). Multiple movement modes by large herbivores at multiple spatiotemporal scales. Proc. Natl. Acad. Sci.U.S.A. 105 19114–19119. 10.1073/pnas.0801737105 19060190 Giraldeau L. A. Valone T. J. Templeton J. J. (2002). Potential disadvantages of using socially acquired information. Philos. Trans. R. Soc. Lon. Ser. B 357 1559–1566. 10.1098/rstb.2002.1065 12495513 Grecian W. J. Lane J. V. Michelot T. Wade H. M. Hamer K. C. (2018). Understanding the ontogeny of foraging behaviour: Insights from combining marine predator bio-logging with satellite-derived oceanography in hidden Markov models. J. R. Soc. Interface 15:20180084. 10.1098/rsif.2018.0084 29875281 Green R. F. (1980). Bayesian birds: a simple example of Oaten’s stochastic model of optimal foraging. Theor. Popul. Biol. 18 244–256. 10.1016/0040-5809(80)90051-9 de Grissac S. Bartumeus F. Cox S. L. Weimerskirch H. (2017). Early-life foraging: behavioral responses of newly fledged albatrosses to environmental conditions. Ecol. Evol. 7 6766–6778. 10.1002/ece3.3210 28904758 Gros C. (2015). Complex and Adaptive Dynamical Systems. Berlin: Springer. Gurarie E. Suutarinen J. Kojola I. Ovaskainen O. (2011). Summer movements, predation and habitat use of wolves in human modified boreal forests. Oecologia 165 891–903. 10.1007/s00442-010-1883-y 21207232 Hamilton P. K. Cooper L. A. (2010). Changes in North Atlantic right whale (Eubalaena glacialis) cow-calf association times and use of the calving ground: 1993-2005. Mar. Mammal Sci. 26 896–916. 10.1111/j.1748-7692.2010.00378.x Hastie T. Tibshirani R. Friedman J. (2009). The Elements of Statistical Learning: Prediction, Inference and Data Mining. New York NY: Springer-Verlag. He K. Dai Q. Foss-Grant A. Gurarie E. Fagan W. F. Lewis M. A. (2019). Movement and activity of reintroduced giant pandas. Ursus 29:163. 10.2192/ursus-d-17-00030.1 Healy S. D. Hurly T. A. (1995). Spatial memory in rufous hummingbirds (Selasphorus rufus): a field test. Anim. Learn. Behav. 23 63–68. 10.3758/bf03198016 Helfman G. S. Schultz E. T. (1984). Social transmission of behavioural traditions in a coral reef fish. Anim. Behav. 32 379–384. 10.1016/s0003-3472(84)80272-9 Henderson J. Hurly T. A. Healy S. D. (2006). Spatial relational learning in rufous hummingbirds (Selasphorus rufus). Anim. Cogn. 9 201–205. 10.1007/s10071-006-0021-z 16767469 Heyes C. M. (1994). Social learning in animals: categories and mechanisms. Biol. Rev. Camb. Philos. Soc. 69 207–231. 10.1111/j.1469-185x.1994.tb01506.x 8054445 Hoare R. E. Du Toit J. T. (1999). Coexistence between people and elephants in African savannas.”. Conserv. Biol. 13 633–639. 10.1046/j.1523-1739.1999.98035.x Holdo R. M. Holt R. D. Fryxell J. M. (2009). Opposing rainfall and plant nutritional gradients best explain the wildebeest migration in the serengeti. Am. Nat. 173 431–445. 10.1086/597229 19243258 Huber R. Knaden M. (2018). Desert ants possess distinct memories for food and nest odors. Proc. Natl. Acad. Sci.U.S.A. 115 10470–10474. 10.1073/pnas.1809433115 30249656 Hutchinson J. M. Gigerenzer G. (2005). Simple heuristics and rules of thumb: where psychologists and behavioural biologists might meet. Behav. Process. 69 97–124. 10.1016/j.beproc.2005.02.019 15845293 Ishii Y. Shimada M. (2010). The effect of learning and search images on predator-prey interactions. Popul. Ecol. 52 27–35. 10.1007/s10144-009-0185-x Isler K. Van Schaik C. P. (2006). Metabolic costs of brain size evolution. Biol. Lett. 2 557–560. 10.1098/rsbl.2006.0538 17148287 Jacobs L. F. Menzel R. (2014). Navigation outside of the box: what the lab can learn from the field and what the field can learn from the lab. Mov. Ecol. 2 1–22. 10.1017/cbo9780511625428.002 Janmaat K. R. L. Polansky L. Dagui Ban S. Boesch C. (2014). Wild chimpanzees plan their breakfast time, type and location. PNAS 111 16343–16348. 10.1073/pnas.1407524111 25349399 Janson C. H. Byrne R. (2007). What wild primates know about resources: opening up the black box. Anim. Cogn. 10 357–367. 10.1007/s10071-007-0080-9 17393193 Jesmer B. R. Merkle J. A. Goheen J. R. Aikens E. O. Beck J. L. Courtemanch A. B. (2018). Is ungulate migration culturally transmitted? Evidence of social learning from translocated animals. Science 361 1023–1025. 10.1126/science.aat0985 30190405 Jordan M. I. Mitchell T. M. (2015). Machine learning: trends, perspectives, and prospects. Science 349 255–260. 10.1126/science.aaa8415 26185243 Kays R. Crofoot M. C. Jetz W. Wikelski M. (2015). Terrestrial animal tracking as an eye on life and planet. Science 348:aaa2478. 10.1126/science.aaa2478 26068858 Keasar T. Rashkovich E. Cohen D. Shmida A. (2002). Bees in two-armed bandit situations: foraging choices and possible decision mechanisms. Behav. Ecol. 13 757–765. 10.1093/beheco/13.6.757 Kendal R. L. Boogert N. J. Rendell L. Laland K. N. Webster M. Jones P. L. (2018). Social learning strategies: bridge-building between fields. Trends Cogn. Sci. 22 651–665. 10.1016/j.tics.2018.04.003 29759889 King L. E. Douglas-Hamilton I. Vollrath F. (2011). Beehive fences as effective deterrents for crop-raiding elephants: field trials in northern Kenya. Afr. J. Ecol. 49 431–439. 10.1111/j.1365-2028.2011.01275.x Kokko H. López-Sepulcre A. Morrell L. J. (2006). From hawks and doves to self-consistent games of territorial behavior. Am. Natural. 167 901–912. 10.2307/3844746 Kloppers E.L St Clair C. C. Hurd T. E. (2005). Predator-resembling aversive conditioning for managing habituated wildlife. Ecol. Soc. 10 3O1. Krebs J. R. Inman A. J. (1992). Learning and foraging: individuals, groups, and populations. Am. Nat. 140 S63–S84. Kubandt F. Nowak M. Koglin T. Gros C. Sándor B. (2019). Embodied robots driven by self-organized environmental feedback. Adaptive Behavior 27 285–294. 10.1177/1059712319855622 Laland K. N. Williams K. (1997). Shoaling generates social learning of foraging information in guppies. Anim. Behav. 53 1161–1169. 10.1006/anbe.1996.0318 9236013 Leadbeater E. Chittka L. (2007). Social learning in insects - From miniature brains to consensus building. Curr. Biol. 17 703–713. Leadbeater E. Chittka L. (2009). Bumble-bees learn the value of social cues through experience. Biol. Lett. 5 310–312. 10.1098/rsbl.2008.0692 19324653 Leonard B. McNaughton B. (1990). “Spatial representation in the rat: conceptual, behavioral, and neurophysiological perspectives,” in Neurobiology of Comparative Cognition, eds Kesner P. R. Olton D. S. (Hillsdale, NJ: Psychology Press), 488. Lesort T. Díaz-Rodríguez N. Goudou J. F. Filliat D. (2018). State representation learning for control: an overview. Neural Networks 108 379–392. 10.1016/j.neunet.2018.07.006 30268059 Lihoreau M. Raine N. E. Reynolds A. M. Stelzer R. J. Lim K. S. Smith A. D. (2012). Radar tracking and motion-sensitive cameras on flowers reveal the development of pollinator multi-destination routes over large spatial scales. PLoS Biol. 10:e1001392. 10.1371/journal.pbio.1001392 23049479 Lihoreau M. Raine N. E. Reynolds A. M. Stelzer R. J. Lim K. S. Smith A. D. (2013). Unravelling the mechanisms of trapline foraging in bees. Commun. Integr. Biol. 6:e22701. 10.4161/cib.22701 23750293 Maekawa T. Ohara K. Zhang Y. Fukutomi M. Matsumoto S. Matsumura K. (2020). Deep learning-assisted comparative analysis of animal trajectories with DeepHL. Nat. Commun. 11:5316. Ma Z. S. (2015). Towards computational models of animal communications, an introduction for computer scientists. Cogn. Syst. Res. 33 70–99. 10.1016/j.cogsys.2014.08.002 Mangel M. Clark C. W. (1988). Dynamic Modeling in Behavioral Ecology, Vol. 63. Princeton NJ: Princeton University Press. Marsh R. Hao X. Xu D. Wang Z. Duan Y. Liu J. (2010). A virtual reality-based FMRI study of reward-based spatial learning. Neuropsychologia 48 2912–2921. 10.1016/j.neuropsychologia.2010.05.033 20570684 McDonald W. R. Clair C C St (2004). The effects of artificial and natural barriers on the movement of small mammals in Banff National Park. Canada Oikos 105 397–407. 10.1111/j.0030-1299.2004.12640.x McNamara J. M. Houston A. I. (1985). Optimal foraging and learning. J. Theor. Biol. 117 231–249. 10.1016/s0022-5193(85)80219-8 Merkle J. A. Fortin D. Morales J. M. (2014). A memory-based foraging tactic reveals an adaptive mechanism for restricted space use. Ecol. Lett. 17 924–931. 10.1111/ele.12294 24811575 Merkle J. A. Cherry S. G. Fortin D. Kotler B. P. (2015). Bison distribution under conflicting Foraging strategies: site fidelity vs. Energy Maximization. Ecol. 96 1793–1801. 10.1890/14-0805.1 Merkle J. A. Potts J. R. Fortin D. (2017). Energy benefits and emergent space use patterns of an empirically parameterized model of memory-based patch selection. Oikos 126 185–195. Merkle J. A. Sawyer H. Monteith K. L. Dwinnell S. P. Fralick G. L. Kauffman M. J. (2019). Spatial memory shapes migration and its benefits: evidence from a large herbivore. Ecol. Lett. 22 1797–1805. 10.1111/ele.13362 31412429 Michelot T. Blackwell P. G. Matthiopoulos J. (2019). Linking resource selection and step selection models for habitat preferences in animals. Ecology 100 1–12. 10.1007/0-306-48151-0_1 Mitchell M. S. Powell R. A. (2004). A mechanistic home range model for optimal use of spatially distributed resources. Ecol. Modell. 177 209–232. 10.1016/j.ecolmodel.2004.01.015 Moorcroft P. R. Lewis M. A. (2006). Mechanistic Home Range Analysis. Princeton NJ: Princeton University Press. van Moorter B. Visscher D. Benhamou S. Börger L. Boyce M. S. Gaillard J. M. (2009). Memory keeps you at home: a mechanistic model for home range emergence. Oikos 118 641–652. 10.1111/j.1600-0706.2008.17003.x Mora C. V. Davison M. Martin Wild J. Walker M. M. (2004). Magnetoreception and its trigeminal mediation in the homing pigeon. Nature 432 508–511. 10.1038/nature03077 15565156 Morellet N. Bonenfant C. Börger L. Ossi F. Cagnacci F. Heurich M. (2013). Seasonality, weather and climate affect home range size in roe deer across a wide latitudinal. J Anim Ecol. 82 1326–1339. 10.1111/1365-2656.12105 23855883 Mueller T. Fagan W. F. Grimm V. (2011). Integrating individual search and navigation behaviors in mechanistic movement models. Theor. Ecol. 4 341–355. 10.1007/s12080-010-0081-1 Mueller T. O’Hara R. B. Converse S. J. Urbanek R. P. Fagan W. F. (2013). Social learning of migratory performance. Science 341 999–1002. 10.1126/science.1237139 23990559 Murray M. H. Fassina S. Hopkins J. B. Whittington J. Clair C. C. S. (2017). Seasonal and individual variation in the use of rail-associated food attractants by grizzly bears (Ursus arctos) in a national park. PLoS One 12:e0175658. 10.1371/journal.pone.0175658 28542218 Namboodiri V. M. K. Levy J. M. Mihalas S. Sims D. W. Shuler M. G. H. (2016). Rationalizing spatial patterns of wild animals and humans through a temporal discounting framework. PNAS 113 8747–8752. 10.1073/pnas.1601664113 27385831 Nathan R. (2008). An emerging movement ecology paradigm. Proc. Natl. Acad. Sci.U.S.A. 105 19050–19051. 10.1073/pnas.0808918105 19060197 Niven J. E. (2016). Neuronal energy consumption: biophysics, efficiency and evolution. Curr. Opin. Neurobiol. 41 129–135. 10.1016/j.conb.2016.09.004 27664945 Noser R. Byrne R. W. (2014). Change point analysis of travel routes reveals novel insights into foraging strategis and cognitive maps of wild baboons. Am. J. Primatol. 76 399–409. 10.1002/ajp.22181 23946104 Oliveira-Santos L. G. R. Forester J. D. Piovezan U. Tomas W. M. Fernandez F. A. S. (2016). Incorporating animal spatial memory in step selection functions. J. Anim. Ecol. 85 516–524. 10.1111/1365-2656.12485 26714244 Owen C. Rendell L. Constantine R. Noad M. J. Allen J. Andrews O. (2019). Migratory convergence facilitates cultural transmission of humpback whale song. R. Soc. Open Sci. 6:190337. 10.1098/rsos.190337 31598287 Papastamatiou Y. P. Cartamil D. P. Lowe C. G. Meyer C. G. Wetherbee B. M. Holland K. N. (2011). Scales of orientation, directed walks and movement path structure in sharks. J. Anim. Ecol. 80 864–874. 10.1111/j.1365-2656.2011.01815.x 21366562 Patrick S. C. Weimerskirch H. (2017). Reproductive success is driven by local site fidelity despite stronger specialisation by individuals for large-scale habitat preference. J. Anim. Ecol. 86 674–682. 10.1111/1365-2656.12636 28117897 Pearce J. M. Bouton M. E. (2001). Theories of associative learning in animals. Ann. Rev. Psychol. 52 111–139. 10.1146/annurev.psych.52.1.111 11148301 Polansky L. Kilian W. Wittemyer G. (2015). Elucidating the significance of spatial memory on movement decisions by African savannah elephants using state–space models. Proc. R. Soc.B 282:20143042. 10.1098/rspb.2014.3042 25808888 Poor E. E. Loucks C. Jakes A. Urban D. L. (2012). Comparing habitat suitability and connectivity modeling methods for conserving pronghorn migrations. PLoS One 7:e49390. 10.1371/journal.pone.0049390 23166656 Potts J. R. Lewis M. A. (2014). How do animal territories form and change? lessons from 20 years of mechanistic modelling. Proc. R. Soc. B Biol. Sci. 281:20140231. 10.1098/rspb.2014.0231 24741017 Potts J. R. Mokross K. Lewis M. A. (2014). A unifying framework for quantifying the nature of animal interactions. J. R. Soc. Interface 11:20140333. 10.1098/rsif.2014.0333 24829284 Preisler H. K. Ager A. A. Wisdom M. J. (2006). Statistical methods for analysing responses of wildlife to human disturbance. J. Appl. Ecol. 43 164–172. 10.1111/j.1365-2664.2005.01123.x Ranc N. Moorcroft P. R. Ossi F. Cagnacci F. (2020). Experimental evidence of memory-based foraging decisions in a large wild mammal. biorxiv[Preprint] 10.1101/2020.05.23.112912 Reader S. M. Biro D. (2010). Experimental identification of social learning in wild animals. Learn. Behav. 38 265–283. 10.3758/lb.38.3.265 20628165 Rendell L. Boyd R. Cownden D. Enquist M. Eriksson K. Feldman M. W. (2010). Why Copy Others? Science (80-.). 328 208–213. Rescorla R. A. Wagner A. R. (1972). “A theory of pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement,” in Classical Conditioning II, eds Black A. H. Prokasy W. F. (New York NY: Appleton-Century-Crofts), 64–99. Riotte-Lambert L. Matthiopoulos J. (2019). Communal and efficient movement routines can develop spontaneously through public information use. Behav. Ecol. 30 408–416. 10.1093/beheco/ary180 Riotte-Lambert L. Matthiopoulos J. (2020). Environmental predictability as a cause and consequence of animal movement. Trends Ecol. Evol. 35 163–174. 10.1016/j.tree.2019.09.009 31699411 Riotte-Lambert L. Benhamou S. Chamaillé-Jammes S. (2013). Periodicity analysis of movement recursions. J. Theor. Biol. 317 238–243. 10.1016/j.jtbi.2012.10.026 23108209 Riotte-Lambert L. Benhamou S. Chamaillé-Jammes S. (2017). From randomness to traplining: a framework for the study of routine movement behavior. Behav. Ecol. 28 280–287. 10.1093/beheco/arw154 Ronconi R. A. Clair C C St (2006). Efficacy of a radar-activated on-demand system for deterring waterfowl from oil sands tailings ponds. J. Appl. Ecol. 43 111–119. 10.1111/j.1365-2664.2005.01121.x Scheiner R. (2004). Responsiveness to sucrose and habituation of the proboscis extension response in honey bees. J. Comp. Physiol. A Neuroethol. Sensory Neural, Behav. Physiol. 190 727–733. Schlägel U. E. Lewis M. A. (2014). Detecting effects of spatial memory and dynamic information on animal movement decisions. Methods Ecol. Evol. 5 1236–1246. 10.1111/2041-210x.12284 Schlägel U. E. Merrill E. H. Lewis M. A. (2017). Territory surveillance and prey management: wolves keep track of space and time. Ecol. Evol. 7 8388–8405. 10.1002/ece3.3176 29075457 Scott R. Marsh R. Hays G. C. (2014). Ontogeny of long distance migration. Ecology 95 2851–5859. Shaw A. K. Couzin I. D. (2013). Migration or residency? The evolution of movement behavior and information usage in seasonal environments. Am. Nat. 181 114–124. 10.1086/668600 23234849 Shettleworth S. J. Krebs J. R. Stephens D. W. Gibbon J. (1988). Tracking a fluctuating environment: a study of sampling. Anim. Behav. 36 87–105. 10.1016/s0003-3472(88)80252-5 Sigaud M. Merkle J. A. Cherry S. G. Fryxell J. M. Berdahl A. Fortin D. (2017). Collective decision-making promotes fitness loss in a fusion-fission society. Ecol. Lett. 20 33–40. 10.1111/ele.12698 27873440 Sih A. Mateo J. (2001). Punishment and persistence pay: a new model of territory establishment and space use. Trends Ecol. Evol. 16 477–479. 10.1016/s0169-5347(01)02251-0 Silver D. Schrittwieser J. Simonyan K. Antonoglou I. Huang A. Guez A. (2017). Mastering the game of Go without human knowledge. Nature 550 354–359. 10.1038/nature24270 29052630 Spencer W. D. (2012). Home ranges and the value of spatial information. J. Mammal. 93 929–947. 10.1644/12-mamm-s-061.1 Stamps J. A. (1995). Motor learning and the value of familiar space. Am. Nat. 146 41–58. 10.1086/285786 Stamps J. A. Krishnan V. V. (1999). A learning-based model of territory establishment. Q. Rev. Biol. 74 291–318. 10.1086/393163 Stamps J. A. Krishnan V. V. (2001). How territorial animals compete for divisible space: a learning-based model with unequal competitors. Am. Nat. 157 154–169. 10.1086/318634 18707269 Stanley D. A. Smith K. E. Raine N. E. (2015). Bumblebee learning and memory is impaired by chronic exposure to a neonicotinoid pesticide. Sci. Rep. 5:16508. Stephens D. W. (1987). On economically tracking a variable environment. Theor. Popul. Biol. 32 15–25. 10.1016/0040-5809(87)90036-0 Stephens D. W. Brown J. S. Ydenberg R.C. (Eds.) (2007). Foraging: Behavior and Ecology. Chicago IL: University of Chicago Press. Stephens D. W. Krebs J. R. (1987). Foraging Theory. Princeton NJ: Princeton University Press. Sutton R. S. Barto A.G. (Eds.) (2017). Reinforcement Learning: An Introduction. A Bradford Book. Cambridge, MA: MIT Press. Tang W. Bennett D. A. (2010). Agent-based modeling of animal movement: a review. Geogr. Compass 4 682–700. 10.1111/j.1749-8198.2010.00337.x Teitelbaum C. S. Fagan W. F. Fleming C. H. Dressler G. Calabrese J. M. Leimgruber P. (2015). How far to go? Determinants of migration distance in land mammals. Ecol. Lett. 18 545–552. 10.1111/ele.12435 25865946 Teitelbaum C. S. Converse S. J. Fagan W. F. Böhning-Gaese K. O’Hara R. B. Lacy A. E. (2016). Experience drives innovation of new migration patterns of whooping cranes in response to global change. Nat. Commun. 7:12793. Tello-Ramos M. C. Hurly T. A. Higgott C. Healy S. D. (2015). Time-place learning in wild, free-living hummingbirds. Anim. Behav. 104 123–129. 10.1016/j.anbehav.2015.03.015 Tello-Ramos M. C. Branch C. L. Kozlovsky D. Y. Pitera A. M. Pravosudov V. V. (2019). Spatial memory and cognitive flexibility trade-offs: to be or not to be flexible, that is the question. Anim. Behav. 147 129–136. 10.1016/j.anbehav.2018.02.019 Tenenbaum J. B. Griffiths T. L. Kemp C. (2006). Theory-based bayesian models of inductive learning and reasoning. Trends Cogn. Sci. 10 309–318. 10.1016/j.tics.2006.05.009 16797219 Thrun S. Pratt L. (Eds.) (1998). Learning to Learn. Boston, MA: Springer US. Valletta J. J. Torney C. Kings M. Thornton A. Madden J. (2017). Applications of machine learning in animal behaviour studies. Anim. Behav. 124 203–220. 10.1016/j.anbehav.2016.12.005 Viana D. S. Granados J. E. Fandos P. Pérez J. M. Cano-Manuel F. J. Burón D. (2018). Linking seasonal home range size with habitat selection and movement in a mountain ungulate. Mov. Ecol. 6 1–11. Visalberghi E. Fragaszy D. M. (1990). “Do monkeys ape?,” in Language and Intelligence in Monkeys and Apes: Comparative Developmental Perspectives, eds Parker S. T. Gibson K. R. (Cambridge: CambridgeUniversity Press), 247–273. 10.1017/cbo9780511665486.011 Votier S. C. Fayet A. L. Bearhop S. Bodey T. W. Clark B. L. Grecian J. (2017). Effects of age and reproductive status on individual foraging site fidelity in a long-lived marine predator. Proc. R. Soc. B Biol. Sci. 284:20171068. 10.1098/rspb.2017.1068 28747480 Wakefield E. D. Furness R. W. Lane J. V. Jeglinski J. W. E. Pinder S. J. (2019). Immature gannets follow adults in commuting flocks providing a potential mechanism for social learning. J. Avian Biol. 50 1–13. Warner R. R. (1988). Traditionality of mating-site preferences in a coral reef fish. Nature 335 719–721. 10.1038/335719a0 Weinrich M. (1998). Early experience in habitat choice by humpback whales (Megaptera novaeangliae). J. Mammal. 79 163–170. 10.2307/1382851 Wilmers C. C. Nickel B. Bryce C. M. Smith J. A. Wheat R. E. Yovovich V. (2015). The golden age of bio-logging: how animal-borne sensors are advancing the frontiers of ecology. Ecology 96 1741–1753. 10.1890/14-1401.1 Wijeyakulasuriya D. A. Eisenhauer E. W. Shaby B. A. Hanks E. M. (2020). Machine learning for modeling animal movement. PLoS One 15:e0235750. 10.1371/journal.pone.0235750 32716917 Wolfe J. L. (1969). Observations on alertness and exploratory behavior in the eastern chipmunk. Am. Midl. Nat. 81:249. 10.2307/2423669 Wystrach A. Graham P. (2012). What can we learn from studies of insect navigation? Anim. Behav. 84 13–20. 10.1016/j.anbehav.2012.04.017 Zollner P. A. Lima S. L. (1999). Illumination and the perception of remote habitat patches by white-footed mice. Anim. Behav. 58 489–500. 10.1006/anbe.1999.1186 10479364