The input-output transformation of photoreceptors was elaborated incrementally by the following steps: A static saturation-like nonlinear transformation was modeled as a basic photoreceptor model (Figure 2, PRbasic), according to (1)PR=II+I0 to capture the steady-state response of photoreceptors (Naka and Rushton, 1966; Lipetz, 1971; Shoemaker et al., 2005; Schwegmann et al., 2014a). In Equation (1), PR represents the photoreceptor response, I the input light intensity and I₀ a constant determining the light intensity corresponding to half-maximum response. We introduced a model component as in (2)PR=PRlp1(I)PRlp2(I)+Ik in the first elaboration step (Figure 2, PRelab1) to describe the adaptive temporal response profile of photoreceptors and the adaptive shift of the intensity-response function. The input intensities are in one signal branch, low-pass filtered with a small time constant (PRlp1 (I)), leading to a signal that follows the time course of intensity fluctuations and in a parallel signal branch, low-pass filtered with a large time constant (PRlp2 (I)), leading to a signal reflecting the changes of the overall light level on a slower time scale. The latter signal, after adding a constant I_k, is used to modify the sensitivity of the photoreceptor (Figure 2, PRelab1, color-coded in blue). In the second elaboration step (Figure 2, PRelab2), the time constant of the fast temporal low-pass filter (Figure 2, PRelab2, color-coded in red, τ_PRlpa) adapts to the current light level (I_level) by increasing its cut-off frequency with increasing light level, according to a saturation-like nonlinearity defined by (3)  τPRlpa=1/2(τmax-τmin)(1-tanh(xτ))+τmin      xτ=κτlog10(Ilevel)-μτ     In this equation, τ_max and τ_min are the upper and lower boundary of τ_PRlpa, and μ_τ and κ_τ are the turning point in logarithmic scale and the slope of the saturation-like nonlinear dependency, respectively. Elaboration step 3 (Figure 2, PRelab3) took into account that cells cannot respond immediately to a stimulus with a large response amplitude and, thus, replaced the adaptive first-order low-pass filter (PRlpa) of PRelab2 by a second-order one.

The output of the final adaptive photoreceptor model (Figure 2, PRelab3) was fed into the LMC layer of the visual motion pathway model, which was developed incrementally in a similar way to the photoreceptor models. The basic LMC model (Figure 2, LMCbasic) was realized by a temporal band-pass filter (James, 1992; Lindemann, 2005; Shoemaker et al., 2005; Schwegmann et al., 2014a). In this way, the signal component reflecting the mean brightness level is eliminated, as is characteristic of LMC responses under light-adapted conditions. Elaboration step 1 (Figure 2, LMCelab1) takes into account that LMCs perform like temporal band-pass filters only under light-adapted conditions, but like low-pass filters under dark-adapted conditions (Juusola, 1995). To account for this feature, the photoreceptor response is split into two branches without overall gain change, i.e., the sum of the weights in the two branches is one. These two branches are unfiltered and high-pass filtered, respectively, to account for the low-pass and band-pass filtered component of LMC responses. The weight of the unfiltered branch (w₁, Figure 2, LMCelab1, color-coded in yellow) increases adaptively with decreasing photoreceptor response input. The dependency of w₁ on the photoreceptor response is described in (4)w1=1/2(w1max-w1min)(1-tanh(xw1))+w1min    xw1=κw1log10(PR)-μw1     in which w_1max and w_1min are the upper and lower boundaries of w₁, and μ_w1 and κ_w1 are constants describing the turning point in logarithmic scale and the slope of the saturation-like nonlinear dependency, respectively. Elaboration step 2 (Figure 2, LMCelab2) mimics the adaptive modulation of the contrast gain of LMC responses (Juusola, 1995); therefore, a weighting factor (w₂, Figure 2, LMCelab2, color-coded in green) was introduced which increases adaptively with decreasing light level. Similar to w₁, the dependency of w₂ on the light level (I_level) is described by (5)w2=1/2(w2max-w2min)(1-tanh(xw2))+w2min    xw2=κw2log10(Ilevel)-μw2     in which w_2max and w_2min are the upper and lower boundaries of w₂, and μ_w2 and κ_w2 are constants describing the turning point in logarithmic scale and the slope of the saturation-like nonlinear dependency, respectively.

Facing a vast dynamic range of light intensities, visual systems have to adjust their operating range according to current light conditions. The current light level is obtained within a certain retinal area and on a certain time scale. Since, to the best of our knowledge, no experimental data on the retinal range of brightness adaptation are available, the development and validation of the adaptive peripheral models of photoreceptors and LMCs had to be based on cell responses to point stimuli under a set of adapted conditions (Laughlin and Hardie, 1978; Juusola, 1995) (see below for explanation of the stimuli). Therefore, we did not make an a priori assumption regarding the retinal area and time scale of adaptation, but varied both adaptive parameters systematically to assess their potential functional role.

Specifically, the current light level (I_level) in Equations (3) and (5), that is used to adjust the time constant of the low-pass filter of photoreceptors (Figure 2, color-coded orange) and the weight of contrast gain of LMCs (Figure 2, color-coded green) are defined as follows. We varied the retinal area of adaptation by determining the overall light level (I_level) within a two-dimensional (2D) Gaussian window (1 × 1 pixel² to 71 × 71 pixel², with each pixel covering 1.25° of the visual field corresponding to the angular distance between adjacent ommatidia). The time scale of adaptation was varied by defining the current light level as average light intensity within a certain time period (0–500 ms) before the respective signal is employed for tuning the adaptive parameters. However, since we found that the variation of the retinal area and time scale of these two adaptive processes does not affect the performance in spatial vision much (data not shown), we set the adaptation to be local and instantaneous in the further analyses, i.e., I_level = I, for simplification.

With respect to the adaptive shift of the sigmoidal input response characteristic of the photoreceptors (Figure 2, PRelab1), the output of the lower signal branch (Figure 2, PRelab1, color-coded blue) reflects the current overall light level, which shifts the input-response function of the photoreceptor via a divisive nonlinear operation. The time course of this adaptive process is constrained by physiological data, whereas the retinal range is not. We, therefore, varied the retinal area of adaptation by changing the half-width of the 2D Gaussian filtering applied to the intensity input to this branch, as described above (lower branch of photoreceptor input in Figure 2, PRelab1sp). Since varying the retinal area of brightness adaptation of photoreceptors reveals an advantage of local adaptation (1 × 1 pixel²) for spatial vision (Figure 11), local adaptation is, therefore, used as a default setting.

The output of these model variants of the peripheral visual system is fed into the next processing stage consisting of correlation-type EMDs (Reichardt et al., 1961; Borst and Egelhaaf, 1989, 1993; Egelhaaf and Borst, 1993; Borst, 2000; Clifford and Ibbotson, 2002) (Figure 2, EMD). Each EMD is composed of two mirror-symmetrical subunits, the half-detectors. In each half-detector, the delayed signal originating from one LMC unit is multiplied by the undelayed signal originating in the neighboring LMC unit. This is carried out in parallel at each location in the visual field for horizontal and for vertical neighbors. Subtraction of the signals of corresponding mirror-symmetric half-detectors leads to the EMD responses to vertical (EMD_v) and horizontal motion (EMD_h), respectively. The sign of the movement detector output indicates the direction of the motion detected. The motion energy is computed by taking the norm of the motion vector given by the combination of the responses of a pair of the EMD_h and the EMD_v at a given location (x,y) of the visual field (Schwegmann et al., 2014a): (6)EMD=EMDh2+EMDv2

In order to obtain a photoreceptor model, we attributed specific experimentally determined response properties of photoreceptors to several computational processing units and integrated these incrementally into the version of the photoreceptor model that qualitatively captures most response features of photoreceptors (Figure 3) (See Figure 2, Appendix A, and Section 2 for model description and parameter setting; and Appendix B for description and parameter setting of point stimuli and response analysis). A saturation-like nonlinearity was introduced as the most basic photoreceptor model (Figure 3, PRbasic). The responses of PRbasic (Figure 3R1) to intensity steps (Figure 3S1) reflect the static nonlinear transformation of light intensity into steady-state responses, as is characteristic of photoreceptors. The time-dependent decay of the photoreceptor response to intensity steps (Figure 3R2) was modeled by low-pass filtering the two parallel signal branches subserving each sampling point in space with different time constants before the signal filtered with the smaller time constant is divided by the signal with the larger one (Equation 2, Figure 3, PRelab1). Apart from accounting for the decay in the response amplitude, this model version already accounts for the adaptive shift of the input-response function with increasing light intensity (Figures 3R3,S3). This feature is striking, since PRelab1 does not contain any genuine adaptive elements. Thus, this photoreceptor model can operate under a wide range of light conditions without being saturated. In the second elaboration step (Figure 3, PRelab2), a genuine adaptive element was included by modeling the corner frequency of the low-pass filter to shift to lower frequencies with a decreasing light level (Figures 3S4,R4). In the final elaboration (Figure 3, PRelab3), the time course of the model response was further adjusted by replacing the first-order low-pass filter by a second-order one, because the photoreceptor responses to impulse stimuli do not reach their maximum instantaneously, as is predicted when using a first-order low-pass filter (Figures 3S5,R5).

The responses of the photoreceptor model PRelab3 to various types of stimuli are compared in Figure 4 to electrophysiological data on fly photoreceptors published previously (Laughlin and Hardie, 1978; Juusola, 1995). (1) The step response of both model and biological photoreceptors show fast response decay within approximately 50 ms before reaching their steady state (Figures 4R1,E1). (2) The shift in the stimulus-response characteristic toward larger brightness values with increasing overall brightness (Figure 4R2) reproduces qualitatively that of the corresponding electrophysiological data (Figure 4E2). (3) The shift of the cutoff frequency of the frequency dependence of the contrast gain of the model responses to random brightness fluctuations (Figure 4R3) is shifted to lower values with decreasing overall brightness, similar to that of fly photoreceptors (Figure 4E3). This is accomplished by decreasing the time constant of the adaptive low-pass filter with increasing light level from 2.3 ms for the light-adapted condition to 9 ms for the dark-adapted condition. (4) The mean model and electrophysiological responses to random brightness fluctuations (Figure 4S3) increase both with increasing mean brightness until they reach a saturation level (Figures 4R4,E4). (5) The peak of the impulse responses of both the model and the photoreceptor is increasingly delayed and reduced in amplitude with decreasing overall brightness (Figures 4R5,E5). (6) The model and photoreceptor responses to contrast steps are asymmetric with respect to stimulus polarity: In particular, the responses to brightness increases are more transient and reach a smaller steady-state response amplitude than the responses to brightness decreases (Figures 4R6,E6). However, the amplitudes of the off-responses of the model are larger than those of the fly photoreceptors. Furthermore, the peak responses and time-to-peak were calculated for the long (Figure 4S6) and short contrast steps (same as Figure 4S6, but with 2 ms step stimuli). (7) The slightly smaller peak model response for a positive long contrast step than for a negative long contrast step is reflected in a similar asymmetry in photoreceptor responses, although it is less prominent (Figures 4R7,E7, □). (8) The time-to-peak of both model and photoreceptor responses to contrast steps of either polarity is rather independent of the contrast value of short stimuli (○), whereas it increases with the brightness level for long stimuli (□) with increasing negative contrast (Figures 4R8,E8).

The incrementally developed versions of LMC models and their main response features are illustrated in Figure 5 (See Figure 2, Appendix A, and Section 2 for model description and parameter setting; and Appendix B for description and parameter setting of point stimuli and response analysis). As a basic model, a first-order temporal band-pass filter was introduced (Figure 5, LMCbasic) to mimic the most distinguishing feature of LMCs under light-adapted conditions, i.e., the elimination of the average brightness level from the incoming photoreceptor signals. The contrast gain of LMCbasic responses to pseudo-random brightness fluctuations (Figures 5S1,R1) depends on frequency in a way similar to a band-pass. Since biological LMCs tend to perform like a low-pass filter representing the average brightness at very low light levels, this feature was considered in the first elaboration of the LMC model (Figure 5, LMCelab1) by summing up the weighted high-pass filtered and unfiltered output signals of the photoreceptor model. The sum of weights of the two signals is kept constant (i.e., one), with the weight of the unfiltered signal increasing with decreasing brightness. The frequency dependence of the contrast gain of this model version, as determined by random brightness fluctuations, changes smoothly from a characteristic more similar to that of a low-pass to a characteristic similar to that of a band-pass when the system shifts from a dark-adapted to a light-adapted state (Figure 5R2). A weighting factor in LMCelab2 (Figure 5, LMCelab2), which increases with decreasing brightness, is multiplied by the output signal of LMCelab1 to take into account the fact that the contrast gain of biological LMCs stays within a rather narrow range, even for a wide range of light levels, with only a slightly higher gain under medium brightness levels (Figure 5R3).

The responses of LMCelab2 are compared with the corresponding LMC responses, (Juusola, 1995). (1) The band-pass characteristic of both model and LMCs changes to a low-pass-like characteristic with decreasing overall light level; in parallel, the corner frequencies shift to smaller values (Figures 6R1,E1). (2) The mean response amplitude of model and LMCs to random brightness fluctuations is largest for intermediate overall brightness levels, while they are relatively small under dark or bright conditions (note that LMC responses to brightness increases are negative) (Figures 6R2,E2). Apart from the responses which were used for model development, LMCelab2 also reproduces, without any further adjustment, other qualitative features observed in the electrophysiological data. (3) The peak of impulse responses of both the model and LMCs is increasingly reduced and delayed, and the overshoot of the impulse responses is also increasingly reduced with decreasing light level (Figures 6R3,E3). (4, 5) Both LMCelab2 and LMCs perform like a low-pass filter under dark-adapted conditions (Figures 6R4,E4) and like a band-pass filter under light-adapted conditions (Figures 6R5,E5). However, the model reveals stronger off-responses to negative contrasts. (6, 7) Again, the peak responses and time-to-peak were calculated for the long (Figure 6S5) and short contrast steps (same as Figure 6S5, but with 2 ms step stimuli). The peak responses of both model and LMCs (Figures 6R6,E6) to short contrast steps (•) of either polarity increase with increasing amplitude less than those to long contrast steps (■). The time-to-peak of both the model and LMCs is rather independent of the contrast value and polarity (Figures 6R7,E7), although the model responses are several milliseconds faster than their biological counterparts. This feature may be a consequence of the latencies in the biological system not being implemented by this model.

In order to assess how signal processing in the peripheral visual system, including light adaptation, affects the representation of environmental information by the motion detection system during translational locomotion in cluttered environments, we combined models of photoreceptors and LMCs as developed above with correlation-type EMDs and stimulated this model of the visual motion pathway with image sequences that were based on translational movements of a panoramic camera system through a variety of natural environments (see Section 2). Model variant PRelab1-LMCbasic-EMD in Figure 7, for example, was stimulated with image sequences mimicking the visual input during translational flight in a forest. The panoramic visual stimuli (Figure 7A) and the response profiles of simulated 2D arrays of photoreceptors (Figure 7B), of LMCs (Figure 7C), and of motion energies provided by arrays of EMDs (Figure 7D), as taken during the middle of the translational sequence, are illustrated. How well environmental cues are represented at the level of motion detectors is further assessed by calculating pixelwise the correlation (Figure 7H) between the response profile of the EMDs (Figure 7D) and the contrast (Figure 7E), the nearness (Figure 7F) and the CwN maps, respectively (Figure 7G) (see Section 2). In accordance with Schwegmann et al. (2014a), the high correlation with CwN suggests, at least in this example, that arrays of EMDs represent predominantly contours of nearby objects.

In order to draw general conclusions valid for a wide range of natural environments and to understand the role of each peripheral processing unit, we applied the same performance assessment to 37 different natural environments and 14 different model variants (Figure 8). We applied a Kruskal–Wallis test and post hoc multiple comparison to test whether the performance of different model variants are significantly different (p < 0.05). Without any peripheral processing (Figure 8, white boxes), the EMD responses depend greatly on local contrast and are hardly correlated to the nearness or CwN. The other basic or adaptive photoreceptor model versions alone, i.e., without LMC processing, do not change this situation much (Figure 8, first four gray boxes). However, if the basic or adaptive LMC models are fed directly by the brightness signals without the preceding photoreceptor model, the contrast-dependence of the EMD responses is reduced and the correlation between EMD responses and nearness, as well as the CwN is enhanced significantly. Again, this conclusion is independent of whether a basic or an elaborated version of the LMC model is employed (Figure 8, last three gray boxes). If the LMCs are fed by any of the photoreceptor models, the contrast dependency of the EMD responses further decreases and the correlation with nearness and CwN further improves (Figure 8, black boxes). The model performance is robust against any various adaptive elaboration of the photoreceptors (Figure 8, first four black boxes). However, nearness or CwN are represented best for the combined photoreceptor and LMC model versions if the basic LMC versions rather than the adaptive ones are used (Figure 8, last two black boxes). The slight reduction in performance when introducing an adaptive elaboration of the LMC model (Figure 8, last two black boxes) has its reason in the representation of the overall brightness by LMC if it is not very bright, because part of the images may be dark in many environments. Figure 9 illustrates how the performance in nearness extraction decreases with an increasing representation of the mean brightness level (i.e., fixed weight of low-pass filtered signal component in the response) (Figure 9B).

To sum up, motion-based extraction of spatial information is robust against a variety of adaptive changes in the photoreceptor model (Figure 8, first four black boxes) and LMC model, as long as the average brightness information is largely eliminated from the EMD input as is characteristic of the light adapted insect peripheral visual system (Figure 8, last two black boxes, and Figure 9). Essentially, the elimination of the signal component reflecting the mean brightness of the scenery in the retinal input signals shifts a primarily contrast-based representation of natural environments at the output of the arrays of local movement detectors during translational ego-motion to a representation that reflects its spatial structure (Figure 8, white and gray boxes, and Figure 9). These conclusions lead to our most parsimonious model version for optic flow-based spatial vision: PRelab1-LMCbasic-EMD (See Supplementary Movie S1 for the performance comparison between pure EMD and the suggested model variant PRelab1-LMCbasic-EMD). The following analysis is, therefore, based on this model variant.

The brightness-dependent shift of the input-response function of photoreceptors (Figure 3R3) allows for the encoding of contrast under a vast range of light conditions. In order to further assess the role of this adaptive shift of the photoreceptors' input-response function for spatial vision, we rescaled the same set of naturalistic image sequences artificially to various light levels covering ten decades of intensities (see Equation 7 in Section 2) and compared the performance of non-adaptive PRbasic-LMCbasic-EMD with that of the adaptive PRelab1-LMCbasic-EMD model variants (Figure 10). Since photoreceptors have a limited membrane potential range to encode brightness signals and their responses are affected by noise, their capacity to resolve brightness changes is limited. We mimicked the consequences of this limited resolution of photoreceptors in a very crude way by discretizing the photoreceptor output into 12 bit (2¹²) levels between the maximal and the minimal photoreceptor responses. Simulation results reveal that the EMD response profile of the non-adaptive model variant (Figure 10A), which is most sensitive at light levels of 10⁷ (i.e., I₀ = 10⁷), represents the nearby contours of objects sufficiently well only in this brightness range 10⁶−10⁸, but largely fails under higher or lower light levels (Figure 10B). By contrast, the adaptive model (Figure 10C) represents the contours of nearby objects robustly over the entire range of light conditions examined (Figure 10D). It should be noted that, although the performance of the model version PRelab1 (see Equation 2 and Figure 2) tested appears to be adaptive, it contains no adaptive parameters nor any feedback of the current response level or average input intensity. Rather, its excellent performance in representing spatial information over a wide range of brightness conditions is based exclusively on dividing the signals of two parallel pathways reflecting light intensity on a fast and a slow time scale, respectively (See Supplementary Movie S2 for the performance of model variant PRelab1-LMCbasic-EMD under light conditions vary by 8 decades). Since we could show that the adaptive model variant allows for robust spatial vision under a vast range of light conditions, even with restricted coding resolution, the further simulations of this study are done, for the sake of simplicity, without the limitation in coding resolution.

No experimental data are available concerning the retinal area that determines the adaptive state of photoreceptors. Since we did not want to make a priori assumptions about this important aspect of brightness adaptation, we varied this area in our model simulations systematically to understand its role from the perspective of spatial vision. As the signal branch filtered by PRlpf2 (Figure 11A, color-coded blue) is responsible for reflecting the current light condition in this model variant, we varied the retinal area of the input to this branch by 2D Gaussian filtering of the retinal intensity profile (from 1 × 1 pixel² to 71 × 71 pixel², with each pixel corresponding to 1.25° of visual field; Figure 11B; See section 2.1.3). With an increasing retinal area of adaptation, the performance of arrays of motion detectors to represent nearby contours decreases slightly (Figure 11B). In a further set of simulations, we rescaled the input intensities, according to Equation (7), to either five, three or one decades, i.e., the intensity values were rescaled to 10–10⁶ (Figure 11C, black), 10²–10⁵ (Figure 11C, gray), or 10³–10⁴ (Figure 11C, white), respectively. With the increasing range of input intensity (white to black), the performance decreased strongly with an increasing retinal area of adaptation. These results suggest local adaption of photoreceptors to be advantageous. The wider the intensity range within a given scenery, the more relevant it is for photoreceptors to be locally adaptive in order to maintain the performance of EMDs in spatial vision (See Supplementary Movie S3 for the performance comparison between local adaptation and global adaptation under a wide intensity range).

The speed of fast brightness adaptation is not challenged by the day-night cycle, but rather by the dynamics of locomotion. Taking the flight dynamics of blowflies or bees as an example, fast brightness adaptation should allow the visual motion vision pathway to extract spatial information within tens of milliseconds. This fast time scale is demanded given the characteristic flight and gaze strategies of many insects, which is characterized by up to ten brief and rapid saccadic turns per second interspersed by intersaccadic intervals of mainly translational motion, which may be as short as only few tens of milliseconds (Schilstra and Hateren, 1999; Boeddeker et al., 2015). Since the retinal brightness during flight in natural environments may vary tremendously between consecutive intersaccadic intervals as a consequence of saccadic turns, brightness adaptation should take place during saccadic turns and only during the very first phase of intersaccadic intervals to allow the motion vision pathway to extract spatial information from intersaccadic optic flow.

We changed the overall brightness of the scenery during the translational trajectories to assess the adaption speed and, thus, the time after a rapid brightness change required by the motion vision system to recover its ability to provide spatial information. The light intensities during the first 300 ms of the trajectory is rescaled to the range of 10²–10⁴, and the remaining approximately 700 ms is rescaled to the range of 10⁴–10⁶. The rescaled image sequence was fed into the PRelab1-LMCbasic-EMD model, and the restoration of spatial information at the arrays of motion detectors over time after the abrupt brightness change is illustrated (Figure 12). With the default parameter setting (i.e., the parameters determined according to the physiological data in Juusola, 1995 and used so far in our analysis, Figures 12A–D), the motion detectors are almost saturated by the input signal for roughly 100 ms after the brightness change (Figure 12B). The representation of the spatial information is largely recovered after 200 ms (Figure 12C), and totally recovered about 400 ms after the brightness shift (Figures 12A,D). This speed of adaptation is much slower than that required by the flight dynamics of a fly, i.e., the duration of translational intersaccadic flight (see above). Note, however, that the saccade frequencies were measured under spatially constrained conditions and under light conditions without systematic intensity changes between saccades (Schilstra and Hateren, 1999; Boeddeker et al., 2015). Thus, intersaccadic intervals in free flight in spatially less constrained natural environments might be longer allowing for more time to adapt spatial vision to abrupt brightness changes. Moreover, even the brightest light levels used in the electrophysiological experiments (approximately 400 cd/m², Juusola, 1995) that were employed to tune our model parameters were much darker than sunlit natural environments. Therefore, the response transients and the speed of adaption under bright outdoor conditions could be faster than those obtained under laboratory conditions. Faster brightness adaptation can also be achieved in our model by adjusting τ_PRlp1 and τ_PRlp2 of PRelab1 (Figures 12E–H). With this parameter setting, the spatial information is restored to a large extent within 50 ms after an abrupt brightness change (Figure 12G) and is already fully restored within 150 ms (Figure 12H). This time scale of adaptation matches the requirements of insect flight dynamics. The restoration of spatial vision after an abrupt decrease in brightness works in a similar way, but on a somewhat slower time scale due to the asymmetries in the PR and LMC model responses to light decrements and increments, respectively (See Supplementary Movie S4 for the restoration of spatial vision after an abrupt change in brightness with default and fast-adaptive parameter settings).