The virtual environment (VE) was an advanced version of the VE described in deCamp (2013). The treadmill consisted of a Styrofoam sphere (10 cm diameter) placed in a half-spherical plastic cup with several symmetrically located holes through which a laminar airflow passed and let the sphere float on air.

The projector (Epson EMP-TW 700, Suwa, Japan, digital scanning frequency: pixel clock: 13.5–81 MHz, horizontal sweep: 15–60 kHz, vertical sweep: 50–85 Hz) was positioned above a Faraday cage and illuminated the inner surface of a cone-shaped screen (height 60 cm, bottom diameter 7 cm, top diameter 75 cm) via a large surface mirror and a Perspex window (Figure 2). The inner surface of the cone consisted of white paper. The shape of the patterns projected onto this screen were adjusted so that they appeared undistorted to the bee. During an experiment, the Faraday cage was closed. A web camera (c920, Logitech, Morges Gesellschaft, Switzerland) positioned above it imaged the head of the animal via a 500 mirror objective allowing observation of the animal during the experiment. The light from the projector, which fell directly on the upper view of the animal, was blocked by a screen.

The virtual environment and the recognition of the bee's movement was under the control of the custom program BeeWorld. It was implemented in Java by using OpenGL-Bindings for Java (LWJGL). Two optical high precision computer mice (Imperator, Razer Europe GmbH, Hamburg, Germany; G500, Logitech Europe S.A.) detected the movement of the sphere, initiated by the walking bee. The mice were accurately positioned under 90° at the equator of the Styrofoam sphere and precisely aligned to the optimal distance with x/y micro drives. The readings of the optical mice were precisely calibrated by rotating the sphere around the vertical axes. Thus, it was possible to convert ticks produced by the mice to forward movement, horizontal movement and rotatory movement of the bee in inch. Rotatory movement was obtained by calculating the mean of the movement along the x-axes of both mice. This redundancy is making the signal more reliable. The translational component of the movement results from the differences between the ticks of the y-axes of both mice. Both y-signals were multiplied with their position vectors and summed up. The resulting vector gives the translational components of the movement (Figure 2C):

To obtain the new position of the bee after a move, the translational vector was added in the direction the bee was virtually orientated:

T is the translation vector. O is the orientation.

The data from the mice was read at a frequency of 500 Hz by the computer. The bee was able to control the virtual scenery by rotatory and translatory movements of the sphere. Multiple scenarios were designed and stored as xml files. These files were loaded by the custom program BeeWorld. They contained the positions, widths, and colors (RGB) of a variable number of vertically oriented stripes or other structures. In order to improve the feedback to the bee about the rotatory components, every scenario had a checkerboard pattern projected onto the ground immediately in front of the bee, and gray stripes of different height at the background simulating a far-distant skyline. The rotation and translation of the checkerboard pattern was coupled one-to-one to the intended movement of the animal and the rotation of the skyline was set to a lower angular velocity simulating further distance by motion parallax. The angular velocities of these patterns as well as that of all objects in between could be separately adjusted.

The field of view in OpenGL was limited to 179°, the scenarios projected onto the screen, however, simulated a 360° view. To solve this, four 90° views were combined and transformed to fit the cone-shaped screen. In addition to the 360° view, the checkerboard pattern was projected in the middle of the screen. Movement and rotation speed of our 360° skyline and the checkerboard ground pattern was set relative to the rotation speed of the stripes in the scenarios. The depth components between virtual objects seemingly in the background and virtual objects closer to the animal were simulated by occlusion, size of the objects, and motion parallax that changed with movement of the animal.

Data from walking traces were synchronized with the data from spike recordings as collected with an analog/digital converter (micro3, CED, Cambridge Electronic Design, Cambridge, UK, 20 KHz sampling frequency per channel). A photodiode directed at the projector detected a short light signal under the control of the BeeWorld program and fed it into the ADC input of the analog/digital converter for synchronization. The scenario used in all experiments here consisted of one blue and one yellow vertically oriented stripe with an angular width of 30°, the horizontal checkerboard pattern immediately in front of the bee and the gray skyline pattern in the background. The angular rotation of the checkerboard pattern was equal to the angular movement of the sphere simulating a respective movement of the floor directly below and in front of the bee. The angular rotation of the stripe pattern was set to 75% of the checkerboard pattern, and a skyline pattern projected onto the screen together with the stripe pattern moved with 50% of the checkerboard pattern. Thus, these three patterns simulated different distances to the honeybee.

Free-flying bees were trained in a T-maze (70 cm long until T-junction, about 5 cm in width, 4 cm high with a 55 cm long head side, see Figure 3) with one color (either blue or yellow) rewarded with 30% sucrose and another color punished with a 0.5 M potassium chloride solution. To attract foragers, a feeding station offering 1–10% sucrose solution was placed near the experimental set-up. During the first foraging flights, animals were actively induced into the entrance of the T-maze with a help of a sucrose-containing syringe. Individual foragers were marked, and all T-maze approaches were noted. Only one bee was tested at a time. Bees could fly into the T-maze but had to walk until a point of decision and decide for one side. In the beginning, sucrose droplets in the entrance of the maze showed the direction to the point of decision. The experimental set-up consisted of a plastic T-maze covered with UV-transparent Plexiglas to ensure daylight conditions within the maze. Little doors inside the maze ensured that the bee had to walk in one direction after the point of decision and could not turn back in order to perceive the colors again. The animals were trained over 25 trials.

Afterwards, they flew back to their hives or—in case of a wrong decision—could enter again. To avoid side preferences, we switched the sides of reward and punishment. Subsequently the animals were transferred to the virtual environment and tested in a scenario similar to the T-maze situation without reward.

Worker honeybees (Apis mellifera carnica) were caught at the hive entrance during summertime. In winter, sugar water foraging animals flying in an indoor flight room were collected at the feeding sites. The bees were immobilized by cooling, mounted in Plexiglas tubes, and kept in a high humidity chamber. During the night, bees were held on Styrofoam spheres and were able to move freely on the spheres. All animals were fed to saturation after capture and on each subsequent day at 4 p.m. with 16 μl of 30% sucrose solution.

During the night, bees were on Styrofoam spheres held in an apparatus to measure walking activity that only allowed the animal to walk forward and backward. The animals were able to move on the spheres held by balances, which kept the animal on the spherical treadmill with its own weight. This balance allowed the animal to also change the distance to the surface of the treadmill during walking. Walking activity was detected by means of walking distance. The walking distance was measured with light barriers that assessed the light shining through a wheel connected to the moving sphere with differently colored (black, gray, or transparent) subdivision. The percentage of measured light gave information about the turning direction and speed when measured over time. A connected Arduino microprocessor sent counted subdivision crosses per time to a computer that analyzed the data. Animals that performed well were later transferred to the VE.

To study conditioning in the VE we allowed the animal to walk freely through the VE. Two different training paradigms were used in the VE. Either a color was rewarded (color-only) or a color was rewarded in combination with an odor (color-odor).

The animal was rewarded with 30% sucrose solution in the acquisition phase in both paradigms after arriving at a previously determined color. The colored stripe had to have a particular size and location in front of the animal to be rewarded (rewarding site). When the animal walked toward one stripe, the stripe turned in front of it depending on the rotatory movement of the spherical treadmill. When it kept walking toward the stripe (translatory movement of the sphere), it got bigger, simulating the object getting closer, and when it reached it, it covered 180° of the screen. Arriving at one color (10 cm walking distance) was counted as a decision. A custom-built rewarding device automatically turned toward the animal. This device consisted of a metal arm with an angle bracket at the end, holding a droplet of sucrose solution. Via Spike2 a digital command was sent to an analog-digital converter that sent a five-volt signal to a rotating motor (XFLY 400, Motraxx, Burgthann, Germany) that turned the arm toward the animal, so that it could reach the droplet. The honeybee could lick the sucrose solution for 10 s, either in combination with an odor or without. Afterwards the arm turned back automatically. Only in the color-odor paradigm, was the odor presented during the rewarded color for 4 s followed by a sucrose reward for 10 s with an overlap of 1 s. The odor was switched on when the animal reached the rewarding site. Spike2 controlled the odor stimulation, marked the time and sent a digital command, which triggered a relay board that in turn controlled the valves for odor stimulation. The odor pulse lasted for 4 s. In both designs, the color was switched off simultaneously with the ending of the reward. The reward lasted for 10 s. When the animal walked toward the unrewarded color (C–) nothing happened, and it could continue walking in the VE until it reached the rewarded color (C+). If the animals stopped walking for more than 20 min during the experiment, we switched to a different acquisition trial presenting the CS+ color, the odor and the US. All animals received at least eight acquisition trials including one to eight trials actively walking. The inter-trial interval was 10 min. For every animal, at least five trials were successful rewarded trials. Thus, we used the minimum of five trials for comparison between animals.

In a pre-test before the acquisition started, the naïve responses toward the stimuli were tested three times. Five stimuli were presented independently (see also Figure 5) in a pseudo-random fashion with an inter-trial interval of 1 min independent of the animal's movement.

The following terminology was used for the various training and test conditions:

During training, the odor was only present when the animal approached the rewarded color. The odor was paired with the sucrose reward for 1 s.

In a memory test on the day after the acquisition (post-test), all color stimuli (CC, C+, and C–), the reward alone (US) and the odor alone (O) were presented to the honeybee again. This remained the only test situation in the experiment.

The behavioral response evaluated as a measure of learning was the approach to the rewarded color or color/odor combination. The time until the animalreached the rewarded color was the measure of the learning effect (Figure 6A). As not all animals received five operant acquisition trials, we excluded one animal from the analysis.

Brains were fixed with 4% paraformaldehyde (PFA, Roth, Karlsruhe, Germany) or a mixture of 1.3% PFA and 0.7% glutaraldehyde (GA, Sigma-Aldrich, Munich, Germany) for at least 4 h. Subsequently, brains were washed three times in phosphate buffered saline [PBS; NaCl (37 mM), KCl (2.7 mM), Na₂HPO₄ (8 mM), KH₂PO₄(1.4 mM), pH 7.2] for 10 min each, dehydrated in an ascending alcohol series (50, 70, 90, 99, and 100% each 10 min), cleared in methylsalicylate (Roth, Karlsruhe, Germany) for 10 min, and mounted on a special object slide (a metal plate of 0.5 mm thickness with a central hole and cover slips on both sides) in methylsalicylate.

Confocal image stacks of the whole brains were acquired using a confocal laser scanning microscope (Leica TCS SP2, Wetzlar, Germany) using a 40 × 0.4 IMM lens objective or a 20 × 0.5 water lens objective. Per stack, around 400 sections were scanned with a resolution of 1024 × 1024 voxels each, and with a voxel size of 0.61 × 0.61 × 1.3 μm or 0.73 × 0.73 × 1.1 μm. Stained neurons were scanned at 633 nm. Linear intensity compensation was used to adjust differences in brightness depending on scanning-depth.

For statistical analysis of multiple comparison, a Wilcoxon signed-rank test was performed (Statistica version 8.0, StatSoft, Inc., Tulsa, OK, USA). For normally distributed data, a two-way ANOVA was used to analyze the interaction between two variables (Statistica version 8.0, StatSoft, Inc., Tulsa, OK, USA) with an additional Fisher LSD post-hoc test. For comparison of nominal paired data, a McNemar test was performed in (R Core Team, 2017) as well as a binomial test. A two one-sided t-test (TOST) for equivalence was performed in (R Core Team, 2017) using the equivalence package for a paired sample with epsilon for small sample size = difference in standard deviations. In all tests, differences were considered significant if p ≤ 0.05.

The terms used to describe structural components of the honeybee brain have been adapted according to the nomenclature system of the Insect Brain Name Working Group (Ito et al., 2014).

The virtual environment (VE) simulated a simplified 3D world for a honeybee walking stationary at the center of an air-supported spherical treadmill (Figure 2A). An LCD projector projected the visual patterns from above via a mirror into the inner surface of a cone. The distortion of the patterns caused by the projection on the inner surface of the cone were compensated for by the design of projected patterns. The depth components were simulated by occlusion, size, and motion parallax. The rotatory and translatory movements of the treadmill were translated into respective changes of the visual patterns. Different scenarios were written in a custom program implemented in Java by using OpenGL-Bindings for Java (LWJGL). We used a simple set of stimuli consisting of two colored stripes (blue and yellow) appearing in front of a structured gray skyline in the background, which moved with different parallaxes mimicking far-distant objects. To simulate the movement with the bee's actual walking speed, the animal walked on a checkerboard pattern projected directly onto the sphere in front of it, thus providing a direct visual-motor feedback by the ground over which the animals walked. Two optical mice recorded the movement of the sphere. To adapt to the walking situation, the experimental animals were held on similar spherical treadmills during the night. Animals on the spherical treadmills showed circadian activity (Figure 2B). All data reported here came from experiments at times of the day at which free-flying honeybees usually performed foraging behavior and accordingly the highest activity on the treadmill.

To analyze whether animals perceived the visual stimuli in the VE and recognized them as meaningful stimuli, we performed a transfer experiment. Free-flying bees were trained to enter a T-maze (Figure 3A) in which they ran toward a T-junction with two differently colored legs. Approaching one color was rewarded with sucrose and approaching the other color was punished with potassium chloride solution. After leaving the T-maze, they flew back to their hive or -in case of a wrong decision- could enter again. To avoid side preferences, we switched the sides of reward and no-reward in a pseudo-random fashion. Subsequently the animals were transferred to the VE and tested in a scenario similar to that in the T-maze (Video S1). Here, no reward was presented. The acquisition curve of correct choices in the T-maze shows that the honeybees learned to make the correct turn toward the rewarded color (C+) and discriminated it from the unrewarded color (C–, Figure 3B, from trial 5 onwards: binomial test p < 0.01). After the transfer into the VE, the same bees were faced with a similar situation as in the T-maze (Figure 3A). They could choose between the two colors that they had discriminated in the T-maze by walking toward one of them. At the starting point of the test two colored stripes were presented, similar to the point of decision in the T-maze experiments. Here and in the following experiments this stimulus situation is called color choice (CC). When the animal walked toward one stripe, the stripe moved toward it by increasing its visual angle in front of it simulating the object to get closer. When the animal reached it, the color covered 180° of the screen. Arriving at one color (10 cm walking distance) was counted as a decision. Afterwards the scenario was set back to CC and the animal could choose again (Figure 3C). To avoid side preferences, the sides of the colors were changed every 30 s. Each animal (N = 6) was tested for 5 min. In the first trial, all bees chose the previous C+ over the C– (Figure 3B, binomial test p = 0.015). Significantly more decisions were made for C+ in all decisions (Figure 3D, N = 6 animals, n = 150 choices, binomial test p = 0.0001). A comparison of the percentage of choices demonstrates that there was no side preferences for left or right in the VE without colored stripes present [Figure 3E, N = 6 animals, n = 114 choices, two one-sided t-test (TOST), p = 0.03, TOST test analyzes data for equivalence (Walker and Nowacki, 2011)]. Naïve bees introduced into the VE for the first time showed no difference in choosing between blue and yellow (Figure 3F, N = 12 animals, n = 12 choices, TOST test, p = 0.018).

The search for neural correlates of learning in a VE requires the combination of stable recording from central neurons and free movement of an animal. We performed 24-h lasting extracellular recordings from the area where A3 mushroom body extrinsic neurons bifurcate (Figure 4). The recording site could be well selected visually by placing the recording electrodes at the lateral rim of the vertical lobe (red arrow in Figure 4D). After the experiment, the recording site was verified by dye marking via the tip of the recording electrode (Figures 4A–C).

To study operant learning in the VE we allowed the animal to walk freely through the VE. Two different training paradigms were used in the VE: Either only a color was rewarded (color-only) or the rewarded color was paired with an odor (color-odor, Figure 5A). Each bee was trained only in one paradigm. At the beginning of each acquisition trial, two colored stripes were presented and the bee walked toward one of the two colors such that it reached a particular size and location in front of the animal. This size and location equaled 10 cm walking distance from the starting point toward the center of the stripe thus covering 180° of the screen (rewarding site). After arriving at a previously determined color (blue or yellow, randomly selected by the experimenter), the animal was rewarded with 30% sucrose solution (rewarded color, C+). When this situation was reached a rewarding device turned toward the animal, and the animal could lick the sucrose solution for 10 s. Sides were switched to distinguish between side and color effects on unit activity.

Single unit activity was analyzed (for sorting procedure see Figure 1) in peri-stimulus time histograms with a bin size of 50 ms in all trials (Figure 5B). Activity exceeding the mean plus three standard deviations (from activity before stimulus onset) threshold (m+3^*SD) or falling below m−3^*SD threshold was noted as an excitatory or inhibitory response to the stimulus, respectively. The response strength in Hz to the stimulus was calculated with the sum of the unit's activity divided by the duration of the particular stimulus in seconds.

We performed pre-tests to get the neural responses of the respective naïve animal toward the stimuli that were tested in a pseudo-random fashion before operant training started: (1) both a blue and a yellow stripe (CC) for 14 s. In this moment, the bee could walk toward the C+ or C– (called CC for color choice), (2) C+ (later the to be rewarded color) alone for 14 s, (3) C– (later the not to be rewarded color) alone for 14 s, (4) US three times for 10 s each. (5) In the color-odor paradigm, we additionally tested the odor three times in the pre-test for 4 s.

In a memory test on the day after acquisition (post-test), all color stimuli (CC, C+, and C–), the US, and the odor were presented again, separately. This remained the only extinction tests in the experiment.

Comparing all bees, there were only two points in time that were fixed: The time the color was switched on and the time the reward was delivered. Therefore, we analyzed the neural activity when C+, C–, or CC was switched on, during the time the animal had arrived at C+ shortly before the reward (US) was switched on, and during the US. We compared the activity of the single units to the stimuli prior to acquisition (pre-test), during acquisition (acquisition) and in a memory test after acquisition (post-test) without a reward. No significant inhibitory responses were determined, most likely due to the rather low spike activity and the high criterion for the significant test (mean minus three standard deviation). Only animals that received all pre-tests, acquisition trials and post-test with a stable extracellular recording over two consecutive days were analyzed (N = 29 Units).

During the five acquisition trials of the color-only training, the mean of the time until the animals reached the reward decreased although not significantly [Figure 6A, N = 3; rmANOVA, N = 3, F_{(4, 8)} = 0.61, p = 0.66]. Walking traces from acquisition trials with corresponding unit activity showed changes in neural activity (Figure 6B). During random walk without any colored stripes in the VE, the single unit activity varied over the course of the walk and showed no indication of a place preference (Figure 6C).

After analyzing the changes in the absolute number of units responding to the stimuli, we next analyzed the changes in spike firing rates (SFR) to the respective stimuli. Even though the absolute number of units responding might not be different, this analysis helps us detect an increase or a decrease in firing rate to a stimulus. We compared the conditions during the pre-tests with those during acquisition, and during the pre-tests with those during the post-test for all tested stimuli. We calculated the ΔSFR using the changes of spike rates during stimulation as compared to the spontaneous spike rate 20 s before, including only those data in which the changes exceeded the 3^*SD threshold in the PSTHs. The spike activity was normalized by taking the ratio between stimuli response in pre-test and training or pre-training and test (see Experimental Procedures). Thus, ΔSFR defines the change in spike firing rate from e.g., pre-test to test toward a stimulus divided by total response. A positive ΔSFR indicates an increase in firing rate, a negative ΔSFR a decrease, and ΔSFR of 0 indicates no change (Hussaini and Menzel, 2013). ΔSFRs of all units were analyzed together applying the Wilcoxon signed-rank test against a hypothetical change of zero. Again, we shall deal with the color-only paradigm first and then with the color-odor paradigm.

Firing rate decreased significantly for US during color-only training (Figure 9A, training as compared to pre-test, Wilcoxon signed-rank test, N = 12, US: p = 0.04) but increased during the test (Figure 9A, Wilcoxon signed-rank test, N = 12, p < 0.0001). Similarly firing rate increased significantly to both the rewarded and unrewarded color (Figure 9B, Wilcoxon signed-rank test, N = 12, C–: p = 0.02, C+; p = 0.04). A significant increase was found for the rewarded odor (Wilcoxon signed-rank test, N = 17, O+: p < 0.00046) during color-odor training but not for the rewarded color (C+). The post-tests revealed a significant increase in firing rate for the non-rewarded color C– (Wilcoxon signed-rank test, N = 17, C–: p = 0.016). Again, a very different pattern of learning-related plasticity was found between the two paradigms, color-only and color-odor. These results indicate changes in spike firing to the stimuli in the range of short-term memory (training) but also changes in neuronal activity that persist until the next day (post-test).

We found learning-related spike rate changes for several stimuli, but not for the CC situation. Animals saw both colors appearing at the same time and walked toward one or the other color during the CC scenario in order to get a reward (compare Figure 2A). It seems surprising that no neural response changes were observed during the CC condition. To detect fast spike rate changes that are not detected by our global analyses during stimulation, we compared the time courses of spiking by inter-spike interval (ISI) analysis during pre-test, acquisition, and post-test, and focused on the time window directly after stimulus onset (600 ms). This method detects subtle excitatory or inhibitory rate changes that extend over several 100 ms during stimulation (Strube-Bloss et al., 2011). Additionally, we analyzed the ISI that occurred during acquisition by comparing median ISI during onset of C+ to median ISI during US onset. The medians of the pre-test- and the post-test- ISI was extracted and statistically analyzed for every unit using a Wilcoxon signed-rank test (see also Figure 1 for median ISI). We subtracted the median ISI in the post-test from the median ISI in the pre-test for every unit separately for depicting the changes.

No significant differences in the changes of ISI medians were found for any tested conditions for animals trained in the color-only paradigm (Figure 10A, N = 12, Wilcoxon signed-rank test, p > 0.05).

However, significant differences appeared for the CC situation in animals trained in the color-odor paradigm (Figure 10B right side, Wilcoxon signed-rank test, N = 17, p = 0.046). The changes in ISI medians indicate that learning lead to higher spike frequencies in the CC test conditions immediately after stimulus onset. Furthermore, the ISI medians became significantly shorter at color onset as compared to sugar onset during acquisition (Figure 10B left side, Wilcoxon signed-rank test, p = 0.005). The ISI medians did not change significantly from pre-test to post-test for the C+ and C–, the odor and the US (data not shown, Wilcoxon signed-rank test p > 0.05). As a control we analyzed the ISI over the whole time of the experiment. ISI did also not change significantly when we compared the ISI of the time before the pre-test without stimulation with the ISI of the time before the post-test without stimulation (Wilcoxon signed-rank test p > 0.05).

These results show that even though no changes for the CC were found when comparing the spike rate changes over the whole stimulus (Figure 9), fast changes indeed occurred within 600 ms after stimulus onset in one of our paradigms, the color-odor training but not the color-only training marking the operant conditions of this experimental conditions.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.