A key function of vision is to extract behaviorally relevant information about the outside world from the activity patterns evoked in the retina. Especially fast locomotion requires information about the spatial layout of the environment to allow for meaningful behavioral decisions. Spatial information can be obtained from the relative movements of the retinal image, the optic flow patterns that are generated on the eyes during locomotion.

Visual motion information is not only generated on the eyes when a moving object crosses the visual field, but also all the time while the animal moves around in the environment. Despite this ongoing movement on the retina, we usually perceive the outside world as static. Nevertheless, the retinal motion information is conventionally thought to be important to signal self-motion. One particular type of self-motion has been studied intensively, especially in tethered animals: confronted with a rotating environment, most animals generate eye- or body-movements following this rotation. These rotational responses of the eyes and/or the body to visual motion were monitored and interpreted to compensate for deviations from an intended course of locomotion or an intended gaze direction. In this context the retinal motion is regarded as a disturbance that needs to be compensated (reviews: Götz, 1972; Taylor and Krapp, 2008; Borst, 2014). Although this view may be correct in many behavioral situations, it misses one important point: retinal image motion induced by self-motion of the animal is not just a nuisance, but may also be a highly relevant source of environmental information. In particular, fast flying animals, such as many insects, heavily rely on environmental information derived from optic flow, for instance, to avoid collisions with obstacles, to find a landing site and control landing maneuvers or when learning the landmark constellation around a goal and when later navigating towards this previously learnt site. However, also sitting animals may induce specific body, head, and eye movements for estimating distances to objects in their environment (for review see Collett and Zeil, 1996; Kral, 2003; Srinivasan, 2011; Egelhaaf et al., 2012; Zeil, 2012).

The working hypothesis of much of our recent research on insects, such as flies and bees (Egelhaaf et al., 2012) and, thus, the assumption underlying this article is that the output of the motion vision system combines two highly relevant cues of environmental information: nearness and contrast borders. As a consequence it segments the time-dependent retinal images into potentially relevant nearby structures and—in many behavioral contexts–potentially less relevant distant objects. On the one hand, we will argue that all this is likely to be accomplished by simple computational principles that have been conceptually lumped into a well-known and well-established computational model, i.e., the correlation-type movement detector (often also termed Hassenstein-Reichardt detector or elementary motion detector, EMD; Reichardt, 1961; Borst and Egelhaaf, 1989; Egelhaaf and Borst, 1993; Borst, 2000). On the other hand, we will sketch the current knowledge about how local motion information is further processed to guide orientation behavior.

The correlation-type motion detection scheme has been derived originally as a computational model on the basis of behavioral and electrophysiological experiments on insects (Reichardt, 1961; Egelhaaf and Borst, 1993; Borst et al., 2003; Borst, 2004; Lindemann et al., 2005; Straw et al., 2008; Brinkworth and O’Carroll, 2010; Meyer et al., 2011). Only recently, the computational principles are being decomposed on the circuit level. The neural networks and synaptic interactions underlying motion detection are investigated mainly in the fruitfly Drosophila by employing the sophisticated repertoire of novel genetic tools (e.g., Freifeld et al., 2013; Joesch et al., 2013; Maisak et al., 2013; Reiser and Dickinson, 2013; Silies et al., 2013; Tuthill et al., 2013; Behnia et al., 2014; Hopp et al., 2014; Mauss et al., 2014; Meier et al., 2014; Strother et al., 2014). Since we are focusing here especially on the overall output of the motion detection system, rather than on the cellular details of its internal structure, our considerations are mainly based on model analyses of EMDs. Variants of this computational model can account for many features of motion detection, as they manifest themselves in the activity of output cells of the motion vision pathway and even in the behavioral performance of the entire animal.

In its simplest form, an EMD is composed of two mirror-symmetrical subunits (Figure 1A). In each subunit, the signals of adjacent light-sensitive cells receiving the filtered brightness signals from neighboring points in visual space are multiplied after one of them has been delayed. The final detector response is obtained by subtracting the outputs of two such subunits with opposite preferred directions, thereby considerably enhancing the direction selectivity of the motion detection circuit. Each motion detector reacts with a positive signal to motion in a given direction and with a negative signal to motion in the opposite direction (reviews: Reichardt, 1961; Borst and Egelhaaf, 1989, 1993). Various elaborations of this basic motion detection scheme have been proposed to account for the responses of insect motion-sensitive neurons under a wide range of stimulus conditions including even natural optic flow as experienced under free-flight conditions (e.g., Borst et al., 2003; Lindemann et al., 2005; Shoemaker et al., 2005; Brinkworth et al., 2009; Hennig et al., 2011; Hennig and Egelhaaf, 2012).

As a consequence of their computational structure, EMDs and their counterparts in the insect brain have a number of peculiar features that deviate in many respects from those of veridical velocity sensors. Therefore, they often have been interpreted as the consequence of a simple, but somehow deficient computational mechanism. The most relevant of these features are:

These response features of EMDs make their responses ambiguous with respect to a representation of the retinal velocity. Because these ambiguities, especially the contrast- and texture-dependent response modulations, deteriorate the quality of representing pattern velocity, they have often been discussed as “pattern noise” (Dror et al., 2001; Shoemaker et al., 2005; Rajesh et al., 2006; O’Carroll et al., 2011) and, thus, as a limitation of the biological motion detection mechanism. Here we want to take an alternative stance by proposing that these pattern-dependent modulations of the movement detector output do not reflect noise in the context of velocity coding. Rather, they can be interpreted as being relevant from a functional point of view, as they reflect potentially useful information about the environment and, thus, may be relevant for visually guided orientation behavior (Meyer et al., 2011; O’Carroll et al., 2011; Hennig and Egelhaaf, 2012; Schwegmann et al., 2014; Ullrich et al., 2014b).

It is indispensable that the animal is active and moves to be able to use the environmental information provided by EMDs. This is because movement detectors do not respond in a stationary world if the animal is also stationary. However, not every type of self-motion is equally suitable for the brain to extract useful information about the environment from the image flow and, thus, from the EMD responses. Especially, if spatial information is concerned only the optic flow component generated by translational self-motion is useful. During pure translational self-motion the retinal images of objects close to the observer move faster than those of more distant ones. More specifically, for a given translation velocity, retinal image velocity evoked by an environmental object at a given viewing angle increases linearly with its nearness, i.e., the inverse of its distance. However, the retinal velocity of an object even at a given distance also depends on its viewing angle relative to the direction of motion: the optic flow vectors are maximal at 90° relative to the direction of motion and decrease according to a sine function from here towards the direction of self-motion, where they are zero. Hence, at this singular point, i.e., the direction in which the agent is heading, it is not possible to obtain nearness information. The geometrical situation differs much for pure rotational self-movements of the agent. Then the retinal image displacements are independent of the distance to objects in the environment (Koenderink, 1986).

If locomotion is characterized by an arbitrary combination of translation and rotation, the optic flow field is more complex, and information about the spatial structure of the environment cannot readily be derived. Nevertheless, a segregation of the optic flow into its rotational and translational components can, at least in principle, be accomplished computationally for most realistic situations (Longuet-Higgins and Prazdny, 1980; Prazdny, 1980; Dahmen et al., 2000). However, such a computational strategy is demanding, and it is not clear whether it can be pursued by a nervous system. Several insect species with their tiny brains appear to employ other computationally much more parsimonious strategies.

Specific combinations of rotatory and translatory self-motion may generate an optic flow pattern that contains useful spatial information. For instance, when the animal circles around a pivot point while fixating it, the retinal images of objects before and behind the pivot point move in opposite directions and, thus, provide distance information relative to the pivot point, rather than to the moving observer (Collett and Zeil, 1996; Zeil et al., 1996). Other insects generate pure translational self-motion to obtain distance information relative to the animal. For instance, mantids, dragonflies, and locusts, perform lateral body and head translations and employ the resulting optic flow for gaining distance information, when sitting in ambush to catch a prey or preparing for a jump (Collett, 1978; Sobel, 1990; Collett and Paterson, 1991; Kral and Poteser, 1997; Olberg et al., 2005). During flight, flies, wasps and bees reveal a distinctive behavior that is characterized by sequences of rapid saccade-like turns of body and head interspersed with virtually pure translational, i.e., straight flight phases (Schilstra and Van Hateren, 1999; van Hateren and Schilstra, 1999; Mronz and Lehmann, 2008; Boeddeker et al., 2010, submitted; Braun et al., 2010, 2012; Geurten et al., 2010; Kern et al., 2012; van Breugel and Dickinson, 2012; Zeil, 2012). Saccadic gaze changes have a rather uniform time course and are shorter than 100 ms. Angular velocities of up to several thousand °/s can occur during saccades (Figures 2A, 3A). Rotational movements associated with body saccades are shortened for the visual system by coordinated head movements and roll rotations performed for steering purposes during sideways translations, are compensated by counter-directed head movements. As a consequence, the animal’s gaze direction is kept virtually constant during intersaccades (Schilstra and Van Hateren, 1999; van Hateren and Schilstra, 1999; Boeddeker and Hemmi, 2010; Boeddeker et al., 2010, submitted; Braun et al., 2010, 2012; Geurten et al., 2010, 2012). Hence, turns that are essential to reach behavioral goals are minimized in duration and separated from translational flight phases in which the direction of gaze is kept largely constant. This peculiar time structure of insect flight facilitates the processing of distance information from the translational intersaccadic optic flow. With regard to gathering information about the outside world, it is highly relevant from a functional perspective that the intersaccadic translational motion phases last for more than 80% of the entire flight time (van Hateren and Schilstra, 1999; Boeddeker and Hemmi, 2010; Boeddeker et al., 2010; Braun et al., 2012; van Breugel and Dickinson, 2012). Still, the individual intersaccadic time intervals are short and usually last for only some ten milliseconds; they are only rarely longer than 100 to 200 ms in blowflies, for example (Kern et al., 2012). This characteristic dynamic feature of the active flight and gaze strategy of insects, thus, constrains considerably the timescales on which spatial information can be extracted from the optic flow patterns during flight, a fact the underlying neuronal mechanisms have to cope with (Egelhaaf et al., 2012).

Although the translational intersaccadic flight phases are diverse with regard to the direction and velocity of motion they appear to be adjusted to the respective behavioral context (Figure 2B; Braun et al., 2010, 2012; Dittmar et al., 2010; Geurten et al., 2010). This is especially true for the overall velocity of translational self-motion, although it does not change much during individual intersaccadic intervals (Figure 3B; Schilstra and Van Hateren, 1999; van Hateren and Schilstra, 1999; Boeddeker et al., 2010; Kern et al., 2012). For instance, insects tend to decelerate when their flight path is obstructed, and flight speed is thought to be controlled by optic flow generated during flight (David, 1979, 1982; Farina et al., 1995; Srinivasan et al., 1996; Kern and Varjú, 1998; Baird et al., 2005, 2006, 2010; Frye and Dickinson, 2007; Fry et al., 2009; Dyhr and Higgins, 2010; Straw et al., 2010; Kern et al., 2012). Thereby, they appear to regulate their intersaccadic translational flight velocity to keep the retinal velocities in the frontolateral visual field largely constant at a “preset” level (Baird et al., 2010; Portelli et al., 2011; Kern et al., 2012). This level appears to lie within the part of the operating range of the motion detection system where the response amplitude still increases with increasing retinal velocity (Figure 3; See Section Insect Motion Detection Reflects the Properties of the Environment in Addition to Velocity). These features are likely to be of functional significance from the perspective of spatial vision, because they help to reduce the ambiguities in extracting nearness information from the EMD outputs that represent the optic flow in the visual system. On the other hand, since insects may adjust their translational velocity to the behavioral context (see above, but also Srinivasan et al., 2000), no absolute nearness cues can be obtained by any mechanism extracting spatial information from optic flow: this is because a given retinal velocity and, thus, response level of a motion detection system may be obtained for different combinations of translation velocity and nearness. Hence, nearness information can be extracted only in relative terms, unless translation velocity is known. This implies that translation velocity should be kept constant, if from the response modulations of EMDs (See Section Insect Motion Detection Reflects the Properties of the Environment in Addition to Velocity) nearness information needs to be determined. If also the translation velocity varies, the resulting response modulations are ambiguous with regard to their origin: they could be a consequence of either changes in self-motion or the spatial structure of the surroundings.

Insects provide the basis for representing computationally efficient environmental information from the optic flow generated during the intersaccadic intervals of largely translational self-motion. However, optic flow information is not explicitly given at the retinal input. Rather, it needs to be computed from the spatiotemporal brightness fluctuations that are sensed by the array of photoreceptors of the retina. This is accomplished by local neural circuits residing in the visual neuropils. As explained in Section Insect Motion Detection Reflects the Properties of the Environment in Addition to Velocity the overall performance of these circuits can be lumped together and explained by variants of the correlation-type EMD. Despite the detailed knowledge at the cellular and computational level, the functional significance of the information provided by these movement detectors has not been clearly unraveled yet. Since EMDs are sensitive to velocity, they may exploit the different speeds of objects at different nearnesses during translational self-motion and, thus, may represent information about the depth structure of the environment. However, EMDs are also sensitive to textural features of the environment (See Section Insect Motion Detection Reflects the Properties of the Environment in Addition to Velocity). Is this pattern dependence of the EMD output just an unwanted by-product of a simple computational mechanism, or could it have any functional significance?

Recent model simulations of arrays of EMDs provided evidence that their pattern dependence may make sense from a functional perspective during translatory self-motion in cluttered natural environments. Although several experimental and modeling studies probed the insect motion vision system already before with moving natural images, they only employed image sequences that did not contain any depth structure and, thus, differed much from what an animal experiences in natural environments (Straw et al., 2008; Wiederman et al., 2008; Brinkworth et al., 2009; Barnett et al., 2010; Meyer et al., 2011; O’Carroll et al., 2011). The potential significance of the combined velocity and pattern dependence of correlation-type EMDs became obvious by comparing the activity profiles of EMD arrays induced by image sequences that were obtained from constant-velocity translational movements through a variety of cluttered natural environments containing the full depth information and after the depth structure of the environment was removed. For both types of situations, sample activity profiles of EMD arrays are shown in Figure 4. They differ much, because without depth structure all environmental objects move at the same velocity and, thus, lead to responses irrespective of their distance. It is obvious that the activity profile evoked by motion through the environment with its natural depth structure preserved is most similar not to the nearness map per se, but to the contrast-weighted nearness map, which is the nearness multiplied by the contrast. However, the activity profile evoked by the artificially depth-removed image sequences matches best the contrast map (Schwegmann et al., 2014). This exemplary finding is corroborated by correlation analysis based on translatory motion through several different natural environments (Figure 4).

Hence, EMD arrays do not respond best to the retinal velocity per se and, thus, to the nearness of environmental structures, but to the contrast-weighted nearness. This means that, during translational self-motion in natural environments, the arrays of EMDs represent to a large degree the nearness of high-contrast contours of objects. This conclusion holds true as long as the translational velocity varies only little and, thus, does not induce time-dependent response changes on its own (See Section Enhancing the Overall Power of Insect Brains: Reducing Computational Load by Active Vision Strategy). As mentioned above, this condition is met to a large extent for the short time of most intersaccadic intervals (Figure 3B; Schilstra and Van Hateren, 1999; van Hateren and Schilstra, 1999; Kern et al., 2012). By representing the contours of nearby objects, the distinctive feature of EMDs to jointly represent contrast and nearness information may make perfect sense from a functional point of view. Cluttered spatial sceneries are segmented in this way, without much computational expenditure, into nearby and distant objects. This finding underlines the notion that the mechanism of motion detection has been tweaked by evolution to allow the tiny brains of insects to gather behaviorally relevant information in a computationally efficient way.

However, motion measurements cannot be made instantaneously. As is reflected by the time constants that are an integral constituent of any motion detection mechanism including correlation-type EMDs, it may take some time until reliable motion information and, thus, spatial cues can be extracted from their responses. This may be a challenge as the uninterrupted translational movement phases during intersaccadic intervals are short, ranging between 30 ms up to little more than 100 ms (Figure 3B). It takes few milliseconds after a change from a saccadic rotation to an intersaccadic translational movement for the EMD response to reach a kind of steady-state level. This finding indicates that the initial part of a translational sequence cannot be used by the animal for a reliable estimation of nearness information from the EMD responses (Schwegmann et al., 2014). Even under such constraints the duration of most intersaccadic intervals appears to be long enough to allow for extracting spatial information from the optic flow patterns on the eyes.

In conclusion, during constant-velocity translatory locomotion the largest responses of the motion detection system are induced by contrast borders of nearby objects. Hence, it appears to be of functional significance that insects, such as flies and bees move essentially straight for more than 80% of their flight time and change their direction by interspersed saccadic turns of variable amplitude (Figure 2B). Since translation velocity does not change much during intersaccadic intervals, the output of the motion detection system during individual intersaccadic intervals highlights contrast borders of nearby objects. Thus, what has been conceived often to be a limitation of the insect motion detection system may turn out to be a means that allows—in combination with the active flight and gaze strategy—to parse the environment into near and far and, at the same time, enhance the representation of object borders in a computationally extremely parsimonious way. By combining contrast edge information and motion-based segmentation of the scene in a single representation, the insect vision pathway may reflect an elegant and computationally parsimonious mechanism for cue integration.

In computer vision optic flow is also used for segmentation purposes as well as for solving other spatial vision tasks, such as the recovery of the shape and relative depth of three-dimensional surface structures or the determination of the time-to-collision to an obstacle and the position of the focus of expansion to detect the heading direction (Beauchemin and Barron, 1995; Zappella et al., 2008). Since quite some time, a variety of approaches to optic flow computation has been proposed and applied to robotic applications. These algorithms are based on different assumptions on image motion and operate on different image representations, e.g., directly on the gray level values or the edges in the image sequences (Beauchemin and Barron, 1995; Fleet and Weiss, 2005). In contrast to EMDs that provide jointly information about motion and contrast edges during translatory motion, these technical optic flow approaches have in common that they attempt to estimate the optic flow field veridically, i.e., the flow vectors (up to a scaling factor) according to their velocity in the image plane. If applied to natural image sequence this, however, proofed to be possible to only some extent and erroneous velocity estimates are a common result depending on the pattern properties of the sceneries (Barron et al., 1994; McCarthy and Barnes, 2004). To what extent segmentation algorithms which compute segment borders from discontinuities in a dense field of optic flow estimates as provided by the various computer vision algorithms (Zappella et al., 2008) may be also applicable for computing segmentations based on a motion image computed by EMDs remains to be tested.

Is the environmental information provided by the insect motion detection system during the translational phases of intersaccadic intervals really used by downstream processes in the nervous system and does it eventually play a role in controlling orientation behavior? Answers to this question can only be tentative so far, although it is suggested by two lines of evidence that the EMD-based environmental information might be functionally relevant. On the one hand, detailed knowledge is available of the computational properties of one neural pathway processing the information provided by the arrays of local motion detectors. On the other hand, behavioral studies and current modeling attempts suggest that the motion-based information about the environment may well be exploited for solving behavioral tasks such as collision avoidance and landmark navigation. Both aspects will be dealt with briefly in the following.

The nearness of objects is reflected in the optic flow generated on the eyes during translational self-motion as is characteristic of the intersaccades of insect flight. In many behavioral contexts nearby objects are particularly relevant. Examples are obstacles that need to be evaded, landing sites, or landmarks that indicate the location of an inconspicuous goal. The main assumption of this review is that the behaviorally highly relevant spatial information can be gained without sophisticated computational mechanisms from the optic flow generated as a consequence of translational locomotion through the environment.

However, movement detectors as are widespread in biological systems and can be modeled by correlation-type EMDs do not represent veridically the velocity vectors of the optic flow, but rather also reflect textural information of the environment. This distinguishing feature has often been regarded as nothing but a nuisance of a simple motion detection mechanism. This opinion has been challenged recently by analyzing motion detectors with image flow as generated during translational movements through a wide range of cluttered natural environments. On this basis, the texture information has been suggested to be potentially of functional significance, because it basically reflects the contours of nearby objects. Contrast borders are thought for long to be the main carrier of functionally relevant information about objects in artificial and natural sceneries. This is evidenced by the well-established finding that contrast borders are enhanced by early visual processing in biological visual systems including that of primates (e.g., Marr, 1982; van Hateren and Ruderman, 1998; Simoncelli and Olshausen, 2001; Seriès et al., 2004; Girshick et al., 2011; Berens et al., 2012). One major function of this type of peripheral information processing is thought to be the enhancement of contrast borders at the expense of the overall brightness of the image, but also redundancy reduction in images. Independent of the particular conceptual framework, enhancing contrast borders is seen as advantageous with regard to representing visual environments.

The main conclusion of this paper is that the motion vision system of insects combines both nearness and contour information and preferentially represents contrast borders of nearby environmental structures and/or objects during translatory self-motion. It makes just use of the fact that in normal behavioral situations all this information is only required when an animal is moving. Then the motion vision system segregates, in a computationally parsimonious way, the environment into behaviorally relevant nearby objects and—at least in many behavioral contexts—less relevant distant structures. This characteristic matches—as we think—one major task of the motion detection system, to provide behaviorally relevant behavioral information about the environment, rather than only to extract the velocity of self-motion or the velocity of moving objects. Based on this conclusion, motion detection should not be conceptualized exclusively in the context of velocity representation, which is certainly important in many contexts, but also in the context of gathering behaviorally relevant information about the environment.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.