The APIS conditioning chamber is an aversive conditioning device already established in olfactory conditioning (Kirkerud et al., 2013; Schott et al., 2015; Wehmann et al., 2015). The conditioning chamber consisted of acrylic glass in which a bee can move freely on ceiling and floor (Figure 1A). The bee's movement along the chamber was tracked by 26 infrared LED sensors and opposing phototransistors (16 Hz sampling rate; QEE113 & QSE113/114, Fairchild semiconductor). Interior surfaces were covered by an electrifiable metallic grid and three different colored LED light arrays arranged between the slits of the grid [HCL-1908ABD-4, “super blue“, λ^B = 465 nm, Δλ = 45 nm, Luminous Intensity (LI) = 105 mcd; HCL-1903AGC-4, “super pure green,” λ^G = 525 nm, Δλ = 45 nm, LI = 310 mcd; HCL-1904HY-4, “super yellow,” λ^Y = 590 nm, Δλ = 25 nm, LI = 330 mcd; Huey Jann Electronics]. The wavelengths of the LEDs were within the spectral sensitivity range of honeybees activating different types and amounts of photoreceptors (Figures 1B,C). The different devices in the chamber (metallic grid, LED sensors, etc.) were connected over a microcontroller SMT board (Savvy128, chip45) and an I²C bus controlled 16-channel LED driver (powered by 12 V/1A; TLC59116, Texas Instruments) to a computer via USB. Bees could be conditioned in an automated and interactive manner: The metallic grid and the LEDs were controlled via a custom-written computer software which enabled automatic stimulation by uploading a protocol script. The protocol scripts used in this study utilized interactive feedback to execute commands based on the bee's current position. The bee's movement was logged together with the executed commands and used for offline analysis.

Suction trough the center and the ends of the chamber via a vacuum pump was set to 900 mbar to avoid accumulation of alarm pheromones without compromising the bee's movement. No light was presented in the chamber between stimulation trials: Ambient light was shielded by a custom-fit card-board chassis covering the chambers. Chambers were washed frequently first with hot water and then with 70% ethanol. In total eight identical chambers were used for all experiments. In all experiments two different wavelengths were paired by co-activating two optically separate light fields on each half of the chamber (Figure 1A). Previous work on olfactory learning and pilot experiments with lights revealing the speed and reaction/decision times of the bees laid the basis of the stimuli dynamics. The stimulus duration was 14 s and the inter-stimulus intervals was set to 44 s (from light onset to light onset). In each of the specific protocols we presented λ⁺ to the chamber side the bee was in (bee side). In the other half we presented λ⁻ or no light (darkness). The onset side of the light depended on the bee's position in the chamber for each trial, thus, place learning could be ruled out. In reinforced paradigms, electric shock pulses of 10 V with 100 ms duration at 4 Hz were initiated 3 s after light onset on the λ⁺ side. Shock lasted 11 s (total: 44 pulses) and shared the offset with the light stimulus. We identified exhausted bees by means of their walking speed: bees with an average speed of lower than 2 cm/s for the test trials were discarded from the analysis (Figure S1). All learning experiments consisted of a differential aversive conditioning protocol. The training phase had nine identical trials with one reinforced and one safe chamber side (λ⁺ with shocks and λ⁻ without, respectively) if not indicated differently. This was always followed by an immediate test phase without shocks to assess short-term memory.

We conditioned a total of 405 bees in six different protocols as described above (example protocol illustrated in Figure 2A). Approximately half of the bees were trained in a reinforced paradigm (n = 201), whereas the other half was tested for light preferences only (n = 204). A total of 30 bees (7.5%) was excluded following the exclusion criterion described above. The protocols correspond to the six possible light combinations of the 3 LEDs (each serving as λ⁺ or λ⁻) at maximal light intensity. The test phase consisted of four unreinforced trials in the corresponding color combination.

We conditioned a total of 279 bees, of which 34 (12%) were excluded following the exclusion criterion described above. Of the total, 159 bees followed the reinforced paradigm and 120 the unreinforced. Conditioning was conducted as in experiment I, with the blue and the green light (λ^B and λ^G) resulting in two protocols where each light served as reinforced or safe, respectively. To investigate what bees learned about the light stimuli the test phase consisted of six pseudorandomized trials with two trials testing the conditioned lights against an untrained third light (λ^Y), one trial with the λ⁻ (safe light) on the bee side and λ⁺ (shocked light) on the opposite side and 3 trials testing each of the three lights (λ^B, λ^G, and λ^Y) presented on the bee side against darkness on the opposite side (Protocol illustrated in Figure 2B). Hence, the test phase addressed the aversion of the reinforced light, attraction of the safe light (test against λ^Y or darkness), passive avoidance of the reinforced light (presentation of λ⁺ on opposite side) and whether learned avoidance could overcome innate phototaxis or whether bees generalized between lights.

A total of 103 bees was tested, of which 5 (4.8%) were excluded. To determine whether bees were biased by the light intensities of the three different LEDs an extensive preference test was performed by varying the intensity of one LED type paired with another LED kept at constant 100% intensity as reference. All possible combinations were tested resulting in six different protocols. The intensity of the variable test light either decreased in the first and increased in the second series (0–100% by 10% increments) or the other way around. Additionally, each stimulus was presented twice, with the reference light on the bee side, and then on the opposite side, in a reciprocal fashion.

We conditioned a total of 99 bees in four protocols consisting of the two color pairs λ^Bλ^G and λ^Yλ^G. Relative light intensity levels without initial preference bias obtained from experiment III were used. The protocols were similar to the ones described in experiment I, with the exception that instead of an unreinforced group a single unreinforced test was included just prior to the conditioning to assess the bee's innate light preference. A total of 4 bees (4%) was excluded following the exclusion criterion described above.

We analyzed the APIS-logdata with a custom-written script in R (R-Core-Team, 2012). To capture the different aspects of the bees' behavioral response and evaluate their performances we used three variables: Preference Index (PI), Shocks received, and Speed.

PI was an assessment of the relative time spent on the safe side during the full response of each trial. It was calculated by subtracting time spent on the λ⁻ side (t_λ−) with time spent on opposing λ⁺ side (t_λ+) and dividing on total trial time: PI = (t_λ−–t_λ+)/(t_λ− + t_λ+), thus ranging from −1 to 1, similar to Performance Index from e.g., (Brembs and Wiener, 2006). Subsequently, the PI data were fitted with a linear mixed model (lme function in R nlme package, Pinheiro et al., 2012) with trial, light configuration, and paradigm (Reinforced or Unreinforced) as fixed effects, and bee as random effect to account for repeated measurements.

Number of shocks received served as a direct indicator of learning performance for the training phase. Since unreinforced bees in the learning experiments did not receive any actual shocks we estimated the number of fictive shocks that they would receive based on their movement during the 3–14 s (relative to trial onset) if the shocks had been available. In doing this, we could quantify how well-reinforced bees learned to avoid shocks relative to behavior elicited by the light stimuli alone. Number of received shocks in reinforced and unreinforced (actual or fictive, respectively) was fitted in a generalized linear mixed model with a Poisson error distribution where trial, light configuration and paradigm served as fixed effects (predictor variables), and bee identity as random effect (function lmer in the R lme4 package, Bates et al., 2013).

Bees react to electric shocks of 1–12 V by increasing their speed (Figure S2). To assess whether they also increased their speed to light stimuli predicting shocks, and to look separately at the immediate part of the response to the lights without the influence of the shocks, we calculated the absolute speed (distance covered in cm per second) for both the 3 s period just previous (pre) and posterior (post) to light-onset in each trial. The pre-speed was necessary to evaluate whether eventual changes in the post-speed were due to a response to the light itself, or merely a hangover effect from the shocks of the previous trial. Consequently, the speed would constitute the reflex-dominated part of the response. To evaluate whether this reflex was strengthened by the conditioning, we calculated the difference in speed from the first training trial to the first test trial (henceforth referred to as ΔSpeed). One-sample t-tests were used to evaluate whether ΔSpeed (in reinforced or unreinforced) was different from 0. Additionally, we fitted post-speed data from all trials in a linear mixed model to assess the effect over trials and between reinforced and unreinforced paradigm. Finally, we evaluated differences in speed for the pre- and post-periods over trials for each protocol. Goodness of fit for residuals of each of the fitted outcome variables was assessed by qq-plots to justify for model selection.

With the first experiment, we investigated whether walking bees could learn to differentiate between two spatially separated but simultaneously presented light fields signaling either danger or safety. All six combinations of the three lights (λ^B, λ^G, and λ^Y) were tested on different groups of bees. An unreinforced group (without shocks) ran in parallel for each light-pair combination.

We found that reinforced bees reached higher overall preferences to the safe light during the training phase compared to unreinforced bees (PI_reinf = 0.16 ± 0.02 vs. PI_unreinf. = −0.01 ± 0.02, mean ± SE over training trials) [protocol BG: t₍₆₆₎ = 3.93, p < 0.001] (Figure 4A). Consequently, the reinforced bees reduced the number of shocks they received throughout the training phase by ~49% from 18.7 ± 1.7 to 9.5 ± 1.6, while unreinforced bees would have received a constant and higher amount of shocks over trials had they been activated (fictive shocks from 21 ± 1.9 to 18 ± 2.1; actual vs. fictive shocks in BG: z = −4.42, p < 0.001; Figure 4C). Learning occurred rapidly: reinforced bees displayed efficient avoidance of the shock-paired light within three trials. After nine training trials, a robust short-term memory had formed, reflected in a continued elevated preference to the safe light (PI_reinf. = 0.24 ± 0.05) compared to in unreinforced bees [PI_unreinf. = 0.03 ± 0.05; protocol BG: t₍₆₆₎ = 3.42, p = 0.001] which was retrievable for four consecutive test trials without apparent extinction [t₍₂₀₃₎ = −1.95, p = 0.052].

We trained bees in all six light combinations of the three LED types at their maximum intensity values (see Figures S3–S8 for example traces, and Figure S9 for summary over training and test period). In addition to in the BG protocol, reinforced bees showed an increased preference to the safer light compared to unreinforced bees in the test phase of BY [PI_reinf. = 0.16 ± 0.05 vs. PI_unreinf. = −0.15 ± 0.05; t₍₅₃₎ = 3.61, p < 0.001], YB [PI_reinf. = 0.07 ± 0.05 vs. PI_unreinf. = −0.13 ± 0.04; t₍₆₆₎ = 3.99, p < 0.001], and the YG protocol [PI_reinf. = 0.25 ± 0.05 vs. PI_unreinf. = 0.07 ± 0.05; t₍₆₂₎ = 3.52, p < 0.001] (Figure 4B). Strikingly, a preference change was not evident when λ^G served as the shock-paired light: protocols GB [PI_reinf. = −0.20 ± 0.05 vs. PI_unreinf. = −0.29 ± 0.05; t₍₅₈₎ = 1.026, p = 0.31] and GY [PI_reinf. = −0.25 ± 0.05 vs. PI_unreinf. = −0.25 ± 0.05; t₍₅₈₎ = −1.044, p = 0.30]. PI was higher than zero for test trials in reinforced bees in protocols BG (t = 5.43, p < 0.001), BY (t = 3.05, p = 0.0026), and YG (t = 5.23, p < 0.001), similar to zero for YB (t = 1.60, p = 0.11), and lower than zero for protocols GB (t = −3.92, p < 0.001) and GY (t = −5.01, p < 0.001). This indicated that bees chose the safe side in a non-random way when blue and yellow, but not green light served as λ⁺.

We found that the reinforced bees reacted by increasing their speed over training trials to the light onset alone (averaged over first 3 s following light onset) by ~37% from 3.8 ± 0.3 to 5.2 ± 0.2 cm/s in bees trained in the BG protocol, while the corresponding unreinforced bees kept a near constant speed (from 4.5 ± 0.2 to 4.1 ± 0.2 cm/s; Figure 4D). Indeed, speed increase in training trials was larger for reinforced than unreinforced bees in all six protocols [BG: t₍₆₆₎ = 3.49, p < 0.001; GB: t₍₅₈₎ = 2.98, p = 0.0043; BY: t₍₅₃₎ = 2.28, p = 0.027; YB: t₍₆₆₎ = 4.11, p < 0.001; YG: t₍₆₂₎ = 3.66, p < 0.001; GY: t₍₅₈₎ = 3.54, p < 0.001], suggesting that bees modified their behavior even when λ^G served as the shock-paired light. The speed of the reinforced bees also increased over training trials in the dark pre periods (3 s previous to light onset), indicating a general activity increase due to the presence of shocks, but this increase was weaker compared to the post period for all protocols save the GB and GY (Figure S10). For the unreinforced bees, no difference in speed following light onset was apparent (Figure S11). When considering the change in speed from the first training trial to the first test trial (ΔSpeed), reinforced bees in all six protocols displayed a positive speed change of 31.1 ± 4.5% on average in the post light onset period, while in the pre light onset period the speed change was negligible (5.1 ± 3% on average; Figure 4E). For the unreinforced bees, positive ΔSpeed was not observed in any of the protocols. Taken together, these results indicated that the bees anticipated the shocks upon light presentation.

When evaluated over all trials, unreinforced bees showed a slight but significant preference to λ^G over both λ^B (PI = 0.012 ± 0.006) and λ^Y (PI = 0.014 ± 0.005), when λ^G was presented on the bee side [GB: t₍₃₇₂₎ = −2.01, p = 0.045; GY: t₍₃₆₀₎ = −2.76, p = 0.0061]. This raised the question of whether the observed difficulties in learning to avoid λ^G for the reinforced bees was due to the stronger phototactic drive toward this light. We addressed this issue in our fourth and last experiment.

In summary, our results from the first experiment revealed that bees trained in an aversive paradigm change their responses to light stimuli in an adaptive way to avoid electric shocks. Since the λ⁺ and λ⁻ were always presented together, we could not resolve to which extent bees learned to avoid the former, or approach the latter. We also could not tell whether bees would still display the preference shifts when the conditioned lights are presented with darkness on the opposite side of the chamber, thereby opposing phototaxis. These issues were addressed in our second experiment.

In the second experiment, we focused on the BG and GB protocols from the previous experiment, and tested the trained lights λ^B and λ^G against darkness and against the untrained light λ^Y to investigate the extent of which the bees learned to avoid the shock-paired light or approach the safe light. As in the first experiment a reinforced and an unreinforced group of bees were trained to either light configuration.

We found that similar to the first experiment, the reinforced bees of the BG protocol learned to avoid the shocked λ^B within few trials and shifted their preference to the safe λ^G to a stronger extent than unreinforced bees [PI_reinf. = 0.41 ± 0.02 vs. PI_unreinf. = 0.03 ± 0.02; t₍₁₂₁₎ = 8.05, p < 0.001] (Figure 5A; see example trace in Figure S12). Consequently, they received fewer shocks over trials (12.9 ± 1.2) compared to fictive shocks in the unreinforced bees (21.5 ± 1.6; z = −19.56, p < 0.001; Figure 5B). In the test phase, the reinforced bees preferred the safe λ^G [PI_reinf. = −0.61 ± 0.05; GB tested against zero: t₍₃₂₀₎ = −10.68, p < 0.001] and even darkness over the previously shocked λ^B [PI_reinf. = 0.30 ± 0.06; B0: t₍₃₂₀₎ = 5.18, p < 0.001], and they did so to a greater extent than the unreinforced bees [PI_unreinf. = −0.18 ± 0.07 and PI_unreinf. = −0.45 ± 0.05, for the GB and B0 trial, respectively; unpaired t-tests, trial GB: t₍₁₀₅₎ = −5.13, p < 0.001; trial B0: t₍₁₁₈₎ = 10.2, p < 0.001] (Figure 5A). Reinforced bees also preferred the untrained light λ^Y over λ^B [PI_reinf. = 0.41 ± 0.06; tested against zero: t₍₃₂₀₎ = 7.15, p < 0.001] to a greater extent than did the corresponding unreinforced bees [PI_unreinf. = −0.10 ± 0.06; reinf. vs. unreinf: t₍₁₂₀₎ = 5.96, p < 0.001] (data not shown). This clearly indicated that the λ^B acquired a strong negative valence that even overruled phototaxis. However, both the reinforced and the unreinforced bees preferred the λ^G over the untrained λ^Y without any apparent difference [PI_reinf. = 0.09 ± 0.07 vs. PI_unreinf. = 0.16 ± 0.06; trial YG: t₍₁₂₁₎ = −0.77, p = 0.44] (data not shown). Thus, the data indicated that the bees generalized λ^Y, probably due to its proximity in perceptual space to λ^G, and that it could not be considered as a perceptually neutral stimulus. For this reason, test trials including the untrained λ^Y were omitted from the plot, and we could not conclude that the safe λ^G light had acquired an increased positive valence in the BG protocol.

Reinforced bees reacted with a ~30% higher speed than unreinforced in the 3 s period following light onset throughout the training phase (Speed_reinf. = 4.7 ± 0.2 vs. Speed_unreinf. = 3.6 ± 0.2) and also during the test trials (except for test trial GB), suggesting that they anticipated the shocks whenever surrounded by λ^B, while reducing their speed when surrounded by the safe λ^G [t₍₁₂₁₎ = 6.39, p < 0.001] (Figure 5C). Reinforced bees clearly accelerated upon light onset, which resulted in a significantly higher post onset speed than pre onset speed after the initial training trials [Speed_pre = 4.3 ± 0.2 vs. Speed_post = 4.7 ± 0.2; t_{(1, 103)} = −5.18, p < 0.001] (Figure S13A). The unreinforced bees on the other hand, reduced their speed upon light onset throughout the training trials from 4.1 ± 0.2 to 3.6 ± 0.2 cm/s (averaged over trials) [t₍₉₈₄₎ = 5.89, p < 0.001] (Figure S13B). This strongly indicated that the bees were able to predict shocks, after only a few conditioning trials.

As for the GB protocol, reinforced bees had an initial preference to the shock-paired λ^G (first trial PI = −0.12 ± 0.07) which slowly changed toward the safe λ^B side over trials (last trial PI = 0.18 ± 0.05). However, the preference shift was weaker than for bees trained with the opposite light configuration [t₍₁₂₃₎ = −3.71, p < 0.001], and PI considered over all training trials of the reinforced bees was only marginally positive [PI = 0.057 ± 0.025; t₍₁₂₀₎ = 2.24, p = 0.027] despite being higher than the PI of the unreinforced bees [PI_unreinf. = −0.11 ± 0.02; t₍₁₂₀₎ = 4.67, p < 0.001] (Figure 5D).

The bees trained in the GB protocol were significantly worse at avoiding shocks than the bees trained in the reciprocal protocol (z = 9.83, p < 0.001). Whereas, bees trained in the reciprocal BG protocol on average reduced the number of received shocks by 52.5% (from 20 to 9.5), GB bees reached a reduction of only 20.4% (from 23 to 18.3) from the first to the last training trial (Figure 5E). Indeed, the bees only reduced shocks slightly more than accounted for by innate preference and/or prior experience, which was reflected by the small but significant difference between average number of shocks in reinforced vs. number of fictive shocks in unreinforced bees (20.7 ± 1.4 vs. 24.5 ± 1.4, respectively; z = −4.54, p < 0.001; Figure 5E).

In the test phase, reinforced bees did not show any indication of having learned to avoid λ^G, and clearly spent more time with this light which had just previously been shock-paired instead of crossing over to the dark side as was the case with the opposite protocol [trial G0, PI_reinf. = −0.53 ± 0.06; tested against zero: t₍₂₉₅₎ = −8.59, p < 0.001]. They did so to the same extent as unreinforced bees in the corresponding test trial [PI_unreinf. = −0.54 ± 0.06; reinf. vs. unreinf. trial G0: t₍₁₁₇₎ = 0.18, p = 0.86] (Figure 5D). When λ^G was tested against the untrained λ^Y, reinforced bees achieved a negative PI of −0.46 ± 0.05 [trial GY tested against zero: t₍₂₉₅₎ = −7.39, p < 0.001] (data not shown). In fact, they spent more time with the λ^G than the corresponding unreinforced bees did [PI_unreinf. = −0.25 ± 0.07; trial GY: df = 116, t = −2.33, p = 0.021]. This confirmed our findings from the previous experiment, that λ^G would not acquire a negative valence under these conditions. On the other hand, reinforced bees did change their attitude toward the safe λ^B, and showed a preference to this light over the shock-paired λ^G [PI_reinf. = −0.29 ± 0.06; BG tested against zero: t₍₂₉₅₎ = −4.77, p < 0.001], the untrained λ^Y [PI_reinf. = 0.32 ± 0.06; YB: t₍₂₉₅₎ = 5.23, p < 0.001], and over darkness [PI_reinf. = −0.60 ± 0.05; B0: t₍₂₉₅₎ = −9.71, p < 0.001]. These preferences toward the safe λ^B were stronger for the reinforced than the unreinforced bees [BG: PI_unreinf. = 0.01 ± 0.05; YB: PI_unreinf. = 0.09 ± 0.06; B0: PI_unreinf. = −0.34 ± 0.06; PI_reinf. vs. PI_unreinf. trial BG: t₍₁₂₀₎ = −3.73, p < 0.001; YB: t₍₁₁₉₎ = 2.64, p = 0.009; B0: t₍₁₁₉₎ = −3.35, p = 0.0011].

Reinforced bees trained with the GB protocol also increased their speed over training trials compared to the unreinforced bees by ~24% [Speed_reinf. = 4.2 ± 0.2 vs. Speed_unreinf. = 3.4 ± 0.17; t₍₁₂₀₎ = 4.29, p < 0.001], but during the test phase the only clear effect of the reinforcement was an increase in speed to the λ^G vs. darkness (G0) trial [t₍₁₁₉₎ = 2.75, p = 0.007] (Figure 5F). Similar to in the opposite color configuration, the reinforced bees increased their speed at light onset in anticipation of shocks after a couple of trials [Speed_pre = 3.8 ± 0.2 vs. Speed_post = 4.2 ± 0.2; t_{(1, 018)} = −4.07, p < 0.001] (Figure S13C), while the unreinforced bees slowed down at light onset [Speed_pre = 4.0 ± 0.2 vs. Speed_post = 3.4 ± 0.17; t_{(1, 052)} = 8.17, p < 0.001] (Figure S13D). Taken together, the results confirmed that bees conditioned in the GB protocol predict the shocks during exposure to λ^G without showing an adaptive avoidance behavior lasting through the whole trial as they do when λ^B is predicting danger, and suggests that light of this wavelength have a strong innate connotation of safety to foragers.

The results from our second experiment revealed that the valence of either light stimulus (signaling danger or safety) can be modified in the aversive learning paradigm, and that this depends on the light configuration used. Furthermore, learned avoidance can overcome natural phototaxis. Phototaxis was an integral part of our conditioning experiments, and our results from the unreinforced bees in the two first experiments indicated that the bees had slight differences in relative preferences toward the three LED types at their maximum intensities. The role that light intensity played on preference for the different lights was addressed in our third experiment.

In the third experiment, we investigated the bees' preferences for colored light-pairs over varying levels of relative intensities. The goal was to determine if the preferences were intensity-dependent, and whether shifted preferences could be neutralized by adjusting the relative intensities.

Different combinations of a reference light of constant intensity and a test light of variable intensity were tested in absence of reinforcement (example movement trace in Figure S14). Since adaptation in the photoreceptors could potentially be influenced differently to increasing and decreasing intensity increments of lights of different wavelengths, half of the bees received the increasing series first, and the other half the decreasing series first. Each stimulus was presented twice, first with the reference light presented on the bee side, and then on the opposite side, in a reciprocal way. This terminated eventual biases in preference as a result of the bees starting positions at trial onset.

We found that the bees displayed strong phototaxis for all three LEDs at 100% intensities vs. 0% (darkness) with no differences in PI between the colored lights [PI_λB = −0.49 ± 0.08, PI_λG = −0.46 ± 0.09, PI_λY = −0.52 ± 0.08; Anova: F₍₂₎ = 0.03, p = 0.97] (Figure 6). Relation between PI and relative intensities were fitted with a Michaelis-Menten function for each stimulus pair [y = a + bx/(c+x), where y is the PI, x the light intensity in %, and a,b,c are fitted constants], and levels of zero-preference were estimated by solving x for y = 0. With λ^B as the reference-light (at 100% intensity), bees preferred λ^G for intensities >62%, while λ^B was preferred over all tested intensity values of λ^Y. When λ^G served as reference, it was preferred over all intensity levels for both λ^B and λ^Y. With λ^Y as the reference, λ^B reached zero-preference at 84% and plateaued, while λ^G was preferred for intensities >46%. Altogether, these results indicate that λ^G was the preferred light over both λ^B and λ^Y, but that this preference could be titered by adjusting intensity levels. The increasing or decreasing series of intensities for the test-light did not result in significant preference differences when evaluated over the full intensity range [t-test, BG: t₍₃₇₂₎ = 0.34, p = 0.73; GB: t₍₄₃₈₎ = −1.0, p = 0.32; BY: t₍₄₁₆₎ = −1.85, p = 0.065; YB: t₍₄₁₆₎ = −0.49, p = 0.63; YG: t₍₄₁₆₎ = −0.34, p = 0.73; GY: t₍₄₁₆₎ = −0.28, p = 0.78]. The slight discrepancies between preferences observed in this experiment vs. the ones observed in the unreinforced bees of the first two experiments were expected, due to the test light being presented on the opposite side of the bee at onset in half of the trials.

The results from our third experiment revealed that the bees' preferences to lights of different wavelengths depend on intensity, and that they can be neutralized by adjusting the intensities of the individual lights. This experiment provided us with the intensity values needed to explore learning absent of biases in phototaxis for the different colored lights.

In the fourth experiment, we conditioned bees with intensity adjusted light pairs of equal relative preferences obtained from the previous experiment. The four color-configurations of the first experiment that included the λ^G (BG, GB, YG, and GY) were used, but instead of unreinforced groups, a single preference test was introduced prior to training (see Figures S15–S18 for example traces). The λ^B or λ^Y lights at 100% intensity was paired with either 62% or 46% of λ^G to ensure that there was no initial preference bias for any of the lights (values acquired from experiment III).

Bees conditioned with either λ^B or λ^Y as shock-paired lights and λ^G as safe light successfully changed their preference toward the latter [BG: PI = 0.18 ± 0.03, t₍₁₉₉₎ = 2.78, p = 0.006; YG: PI = −0.15 ± 0.02, t₍₁₉₉₎ = 4.12, p < 0.001] (Figures 7A,G), and consequently reduced number of received shocks over trials from 22.1 ± 2.7 to 11.6 ± 2.2 and from 22.4 ± 2.0 to 13.0 ± 1.7 shocks for the BG and YG protocol, respectively (BG: z = −7.45, p < 0.001; YG: z = −8.62, p < 0.001; Figures 7B,K). This preference was still apparent in the test phase, demonstrating robust short-term memory [tested against zero, BG: PI = 0.15 ± 0.05, t₍₉₁₎ = 2.97, p = 0.0038; YG: PI = 0.13 ± 0.04, t₍₉₁₎ = 2.59, p = 0.011]. On the contrary, bees trained in the reciprocal light configurations with λ^G as the shock-paired light did not learn to avoid it, and PI was either flat or even decreasing over training trials [GB: PI = −0.03 ± 0.03, t₍₁₈₃₎ = −1.76, p = 0.08; GY: PI = −0.08 ± 0.02, t₍₁₇₅₎ = −2.04, p = 0.0439] (Figures 7D,J), and they received an increasing amount of shocks over trials (GB: z = 3.36, p < 0.001; GY: z = 5.49, p < 0.001; Figures 7E,K). In the test phase, these bees either showed no preference to either light or preference to the previously shock-paired light for the GB and GY protocols, respectively [GB tested against zero: PI = −0.02 ± 0.04, t₍₉₁₎ = −0.33, p = 0.739; GY: PI = −0.15 ± 0.04, t₍₉₁₎ = −2.75, p = 0.0073]. Bees in all protocols showed anticipative behavior by a ~46–67% speed increase to the light onset alone during the course of training [RM-Anova: F₍₇₅₆₎ = 58.8, p < 0.001]. No difference between protocols was found, which justified pooling of the speed data [RM-Anova: F_{(3, 91)} = 1.2, p = 0.3] (Figures 7C,F,I,L). Bees did not increase speed over training trials in the period previous to light onset [RM-Anova: F_{(7, 56)} = 0.16, p = 0.68], and again no difference between protocols was evident [RM-Anova: F_{(3, 91)} = 0.98, p = 0.4]. The outcome was congruent with that of the previous experiments where relative light intensities differed. Hence, we excluded that relative intensity differences affected the learning, and also that bees failed to learn to avoid λ^G due to a strong phototactic drive, but rather due to innate or acquired positive meaning of light of this wavelength.

In short, our results from the fourth experiment revealed that bees learn the lights of equal relative preference similarly to when the relative intensities differ. Thus, in the aversive paradigm relative intensity played only a minor role compared to the wavelength of the lights, suggesting that chromaticity is the most salient feature for the bee to utilize.

We found that avoidance learning was asymmetric for lights of three different wavelengths, even though these wavelengths are known to activate overlapping subsets of photoreceptors. By using these wavelengths, we could confirm that bees trained in an avoidance paradigm could differentiate between colors that are clearly separated in the perceptual space, λ^B (465 nm, human blue) and λ^Y (590 nm, human yellow) or λ^G (525 nm, human green), as well as two colors that are considerably closer, λ^G and λ^Y. Essentially, when either λ^B or λ^Y predicted shocks, bees learned to effectively avoid these lights within few trials. However, when λ^G was shock-paired, avoidance responses remained comparatively unchanged (Figures 4, 5). Even after removing the initial bias in phototaxis by adjusting light intensities, our results revealed that bees still had severe difficulties in learning to avoid λ^G (Figure 7). This indicates that bees have an innate representation of safety for green, which is hard to overcome by aversive operant conditioning.

To the best of our knowledge, difficulties in associating green light (or objects) with danger has not been previously shown in bees. Variation in learning rates of different colors has been described in appetitive conditioning of free flying bees, but all tested colors (within the visual range) were eventually learned, and mostly within a handful of training trials (Menzel, 1967). Africanized honeybees were recently shown to display a similar asymmetric color dependent learning in an appetitive PER assay (Jernigan et al., 2014). The bees could readily learn that blue and green light predicted a reward, but failed to do so for violet light. The authors speculated in whether light within the UV range needs to be polarized for the bees to learn it. Indeed, another study demonstrated that bees were able to learn in a discriminatory task when polarized UV light, but not polarized blue or green light was rewarded (Sakura et al., 2012). Under these circumstances, the resistance of green light to acquire a negative valence would be interesting to retest with directionality added to the light. Like us, bees can see more shades of green than of any other color (Giurfa et al., 1996), and it is conceivable that it could be an evolutionary benefit to keep the most abundant color in nature from easily acquiring negative valence. Alternatively, green might be irrelevant for bees as a target color, given that plant leaves are generally the background to flowers, and flowers have evolved colors that are not green. Nature presents colors in various patterns and combinations. Thus, it would be interesting to test bees with combinations of colors to study further asymmetries in aversive learning. For instance, it is conceivable that the occurrence of green together with light of another wavelength could acquire a negative valence. Ultimately, it remains to be elucidated whether this phenomenon is a result of acquired information about green through extensive foraging or of inherent properties of the network responsible for shock evaluation and color processing.

In the second experiment, we disentangled the aversive and the appetitive memory constituents by presenting the danger (λ⁺) and safe-predicting light (λ⁻) against darkness and an untrained light (λ^Y) in the test phase. We found that honey bees can either form a negative association to the shock-paired or a positive one to the safe light, but not both at once for the light combinations we tested (Figure 5). The increased preference for the safe-predicting light (λ⁻) that we observed argues for honeybees being capable of relief learning, similar to what has been demonstrated in mammals conditioned in avoidance paradigms with discriminatory stimuli (references within Mackintosh, 1974, chapter 6 & 10). In most groups of three conditioned stimuli there will be two that are closer in perceptual space—in our experiments, these are green and yellow light. Since bees learned to differentiate between green and yellow when the latter was shock-paired in experiment I and IV (Figures 4B,G), yellow light was selected as untrained test light with blue and green as either λ⁺ or λ⁻ in experiment II. Our results indicate that bees generalized the green and yellow light. Thus, we could not assume that yellow served as perceptually neutral in this setting. In order to draw conclusions about what was learned about the blue and green lights without the interference of generalization, we opted to emphasize on the test of trained lights against darkness instead. Mackintosh discusses that mammals show learning toward both the S+ and S– in most cases, but that what is learned about S+ and S– varies a great deal depending on the experimental procedures. Having both acquisition processes running in parallel would be more demanding for the nervous system, and thus the most important feature (dependent on context and state of the animal) could be prioritized. In our case, it is conceivable that we only observed that bees learned either λ⁺ or λ⁻ strictly due to their difficulties in assigning valence to the green light. Alternatively, generalization might be stronger for the safe light compared to the dangerous one, causing green and yellow to be treated more similarly than blue and yellow due to the shorter distance in color space. A third conceivable possibility is that the phototactic preference to the green light over yellow (with both lights at max intensity) which was evident in the unreinforced bees, over-shadowed an eventual learned attraction to the safe green light. However, the two latter points would not explain why we did not observe an increased preference to green (when serving as λ⁻) against darkness in reinforced bees compared to unreinforced bees. To pinpoint whether a bee can learn about both the shock-paired and safe light at once, the green light would need to be replaced with a third light (in addition to blue and yellow) which also can acquire a valence. Light in the UV range is an obvious candidate, and should be tested in future experiments.

Learning the valence of the light predicting danger is arguably of higher importance to the animal than learning about the safe light, but acquiring a compound memory for both the danger and the safety signal would be the optimal. Behavioral responses shaped by relief learning appear to be of weaker strength (Tanimoto et al., 2004; Gerber et al., 2014; Vogt et al., 2015), which corresponds well with our own findings. However, it is important to note that our paradigm differed from the previous studies on relief-learning in Drosophila, most notably in that it contained an operant element (shocks could be avoided). Classical and operant processes involve partly separate cellular and molecular pathways (Lorenzetti et al., 2006; Ostlund and Balleine, 2007; Brembs and Plendl, 2008). Thus, it is possible that the experienced relief differs as a consequence of this.

Due to its implications in a range of afflictions like addiction, phobia, post-traumatic stress, compulsive disorders and anxiety, interest in the reward-punishment brain circuitry engaged during avoidance conditioning and relief learning is increasing (Krypotos et al., 2015). Since fundamental aspects of valence attribution to conditioned stimuli due to stimulus timing is shared across phyla (Gerber et al., 2014), underlying cellular mechanisms are likely to be related.

Recently, honeybees were shown to exhibit learned helplessness following inescapable and unavoidable punishment (Dinges et al., 2017), a renown and well-studied phenomenon in higher animals considered to be an underlying cause of depression. Interestingly, bees did not show the typical reduction in activity, rather they remained highly active but failed to restrict their movements to a safe location. This behavior corresponded with what we observed for bees trained with green light as the shock-paired stimulus in our avoidance paradigm. However, since the bees in our paradigm still were in control of the shocks, we cannot ascertain whether the exhibited behavior of the bees from the two studies was due to the same underlying mechanisms.

The relative simplicity and accessibility of the honeybee nervous system combined with its sophisticated behavior and cognitive capabilities makes it an ideal model organism, and the paradigm we present here could readily be combined with pharmacological manipulations to further the understanding of the underlying mechanisms of important phenomena like relief learning and learned helplessness.

Since the LEDs we used had different brightness levels at max intensities (λ^B = 105 mcd, λ^G = 310 mcd, λ^Y = 330 mcd), and the honeybee photoreceptor types are differently sensitive to these wavelengths, we expected that the bees would show different levels of attraction toward the three colors when presented in pairs. Phototaxis in bees is considered to be independent of chromatic properties of light (Labhart, 1974; Menzel and Greggers, 1985), particularly for wavelengths that activate the M- and L-receptors (medium and long wavelength-activated, respectively). Assuming that the M- and L-receptor types contribute similarly to phototaxis, we would expect our λ^G LED to be the most attractive one given its high absolute intensity (310 mcd) and that it activates L-type strongly (relative activation ≈ 0.8, Figure 1C) and M-type weakly (0.05). While the λ^Y despite having a high absolute intensity as well (330 mcd) only activates the L-type moderately (0.3), and the λ^B despite activating all three receptor types (S-type = 0.05, M-type = 0.7, and L-type = 0.3 relative sensitivity, respectively) had a much weaker absolute intensity level (105 mcd). This corresponds well with our results from the unreinforced light preference experiments, where we observed a preference to the λ^G over the other two lights (Figure 6). Interestingly, we found that for the λ^Bλ^Y light pair, the λ^B preference was comparably stronger when serving as the constant intensity reference light, than when serving as the intensity variable test light. This indicates that the visual system in honeybees is less sensitive to intensity changes in light of long wavelengths (yellow). The fact that the preference bias could be eliminated by reducing the intensity of λ^G argued for the chromaticity being of little importance for bees in a phototaxis test. Nonetheless, it is important to point out the distinction between color-ignorance and color-blindness. Clearly, the bees are able to see the difference between lights based on chromaticity, but without the right incentives in the right context, emphasis is shifted toward relative intensities instead.

In the two first learning experiments, we used the three LED types at their maximum values. This meant that bees could potentially take advantage of the difference in perceived intensity in addition to the difference in wavelengths to discriminate between the lights. However, our results from these experiments were congruent with results of the fourth experiment in which light intensities were balanced. This implies that preference bias due to intensity had negligible effect on the learning performance, and confirmed that the bees mainly learned the chromaticity of the lights. Other color conditioning experiments with either free flying or harnessed bees where colored lights of equal or different brightness were used, have indicated that bees learn the chromatic properties of the lights before the intensity differences when the visual angle and the chromatic distances are large (Giurfa et al., 1997; Mota et al., 2011b). By increasing the relative intensity differences, while reducing the color distance between the light stimuli, one could assess the point where the two light characteristics are of equal informational value to the bees during aversive conditioning in APIS.

Behavioral responses from the period between light onset and shock onset is of particular interest, since it is dissociated from the reinforcement and contains the part of the response where the initial action selection takes place. Restriction in temporal resolution prohibits assessment of short periods in conventional operant learning assays, but due to the high sampling rate of the IR tracking sensors in APIS, we could isolate and evaluate the responses from this period. By comparing the speed from the 3 s period just previous to the light onset, with the speed from the period 3 s just after the light onset (but before shock onset), we found that bees in the reinforced paradigm increased their speed upon light onset (Figure 4E, Figures S10, S13). This speed increase in the post onset period took place after a couple of training trials, and since it was larger than the speed increase in the corresponding dark pre-period, it indicated that the bees could predict the upcoming shock period. Interestingly, we observed this predictive behavior even in the protocols in which green light was reinforced, and avoidance behavior considered for the full trial period was absent. This indicates that the initial response is most likely to the light onset itself, and appears to be independent on chromaticity.

Prediction of the upcoming shock events has also been revealed in the immediate response of walking bees conditioned with odors in APIS (Kirkerud et al., 2013). There we quantified the directional speed (velocity) in the period between odor and shock onset and revealed that bees increased velocity away from the CS+ but not the CS-. Unlike odors that are injected from the distal ends into the chamber, colored light fields do not provide adequate directional information to the bees. Thus, the most advantageous strategy for the bee upon light onset, is to increase the speed in the current movement direction until arriving at the safe half of the chamber. Since the starting position and movement direction of the bee at light onset was random, any variable that evaluates the immediate response of the bee and depends on the direction or starting position will have a high degree of variance. Therefore, we found non-directional speed to be the most suitable variable to capture US-independent anticipative behavior in our visual learning paradigm.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.