Understanding neural networks requires the characterization of how neurons transfer synaptic inputs to a sequence of outgoing action potentials. A basic feature of neuronal interaction is the emission of action potentials, each causing a small change of the membrane potential in the receiving cell. The integrate-and-fire neuron model has a long history in neuroscience (Lapicque, 1907; Gerstein and Mandelbrot, 1964; Stein, 1965) reviewed in Brunel and van Rossum (2007). Despite its simplicity the model successfully reproduces prominent features of neuronal dynamics, for example it qualitatively captures the relation between injected current and firing rate (Rauch et al., 2003; La Camera et al., 2004). Equipped with an additional adaptation mechanism, the model also reproduces with high accuracy the sequence of action potentials of a neuron subject to current injection (Jovilet et al., 2008).

The transmission properties of neuron models not only depend on the signal to be transmitted, but also on the activity of the remaining input channels of the cell. The remaining inputs can be regarded as noise with respect to the signal under consideration. If this background noise level is low, a population of neurons faithfully transmits an injected current stimulus up to high frequencies (Knight, 1972; Gerstner, 2000). Independent of the origin of the noise and the associated noise model, the framework of spike response models (Gerstner, 2000) shows that for increasing noise level the transmission becomes progressively more smooth. Often the impact of the signal on the neural dynamics can be assumed to be small compared to the entire synaptic barrage so that a linear approximation of the response is justified. Linear responses to arbitrary transient signals can be constructed as superpositions of responses to sinusoidal variations of the considered signal variable, such as the afferent firing rate. Since a neuron receives many synaptic afferents each having only a small impact, a common approach is to replace the total synaptic input by a Gaussian white noise in the so called diffusion approximation: only the mean μ and the variance σ² of the total input are kept (Siegert, 1951; Johannesma, 1968; Ricciardi and Sacerdote, 1979; Lánský, 1984; Risken, 1996). In this approximation, the afferent spike trains to a neuron translate into – possibly time dependent – parameters μ(t) and σ(t) of the corresponding Gaussian white noise. Mean and variance can vary independently for example by different co-modulations of the rate of excitatory and inhibitory afferents. For the leaky integrate-and-fire neuron a modulation of the mean was found to cause a low-pass response, whereas modulations of the variance are transmitted instantaneously (Lindner and Schimansky-Geier, 2001) up to linear order. Experiments confirmed these transfer properties for Gaussian white noise currents injected into cells in vivo (Silberberg et al., 2004).

Realistic synaptic currents are extended in time, causing low-pass filtering and the suppression of fast fluctuations in the total input current. Given such filtered background noise, modulations of the mean are transmitted up to arbitrarily high frequencies (Brunel et al., 2001; Fourcaud and Brunel, 2002), equivalent to an immediate response. For more realistic onset dynamics of the action potential the neuron model becomes a low-pass (Fourcaud-Trocmé et al., 2003). However, the cutoff frequency increases with the speed of voltage change at action potential onset (Naundorf et al., 2005) and can reach very high frequencies.

The literature cited above rests on the assumption, that the sum of the synaptic inputs can be approximated as Gaussian white noise. However, synaptic inputs arrive as small impulses. Therefore, the work reviewed here (Helias et al., 2010a) reconsiders the integrate-and-fire model neuron, the simplest neuron model which allows us to study the impact of finite and identical synaptic impulses on the stationary and transient neural properties. The analysis of such neural dynamics is complicated, because it comprises two qualitatively different processes. On the one hand, the membrane voltage continuously changes in time in a deterministic way due to ionic currents through the membrane. On the other hand, the incoming synaptic impulses typically arrive at discrete, unpredictable time points and cause rather sudden changes of the voltage. Very recently, for the case of finite synaptic amplitudes drawn randomly from an exponential distribution at each incoming impulse, an exactly solvable framework was presented (Richardson and Swarbrick, 2010). The firing rate, the membrane potential distribution and the response properties are in qualitative agreement with the results for identical synaptic amplitudes, focused on in the present review.

In Section 2 we illustrate that the frequently invoked diffusion approximation causes artifacts and we show how to remedy them by the recently developed theory (Helias et al., 2010a). As a result, in Section 3 we show how neurons can employ the pulse-like synaptic interaction in order to foster fast signal transfer and to perform non-linear operations on short transient signals. In the presence of many synaptic afferents, a single synaptic impulse is intuitively lost in the total synaptic barrage. In Section 4 we show, however, why the reverse is true: in the presence of a certain number of synaptic inputs a neuron operates optimally. The issue of artifacts by shortcomings of neural simulation tools and how they can theoretically be addressed (Helias et al., 2010) is the topic of Section 5. Finally, in Section 6 we summarize our results, put them into the context of the existing literature and provide an outlook.

In this review we illustrate recent conceptual advances in the theory of neuronal dynamics using a model reduced to the fundamental mechanisms. The model is not an abstract equation but has a concrete analogy. The shishi odoshi, as it is depicted in Figure 1A is frequently found in Japanese gardens and used to chase away deers (or birds) and please humans. A bamboo tube, open at one end, accumulates water. Once the tube is filled, the shishi odoshi tilts and the water drains (Figure 1C). Returning into the initial position the shishi odoshi starts over again. The basic operation of a neuron is very similar: Each incoming excitatory synaptic impulse from other neurons causes the membrane potential to depolarize by a small amount, like the shishi odoshi in the rain (Figure 1B). Once a threshold value V_θ is reached, the neuron fires an action potential, the voltage returns to the reset potential V_r, and the process begins from the start. This analogy has probably been made several times before; just recently we found the work by Misonou (2010). Technically, the equation describing both the shishi odoshi and the above reduction of neuronal dynamics is called the perfect integrator. Despite its simplicity, the dynamics still captures some properties that remained unexplained by contemporary theory. For illustration, we decided to resort to the analogy of the shishi odoshi, because it allows an intuitive understanding of the concepts underlying the mathematical formulation (Helias et al., 2010a; Richardson and Swarbrick, 2010).

Consider a shishi odoshi in persistent, heavy rain (Figure 1B). If we watched the level of water V(t) in the tube, we would see a typical time course as shown in Figure 1D. The same holds for a neuron receiving a bombardment of excitatory synaptic impulses. The membrane voltage V(t) increases until it reaches the threshold value V_θ and the neuron emits an action potential (indicated by the markings at the top of Figure 1D). After such an event, both in case of the shishi odoshi and the neuron, V is reset to V_r. On the left side of the figure the probability of the perfect integrator to assume a particular value of V is shown. Since V travels through the range V_r to V_θ with constant speed on average, the probability density is the same for all values in the range. The number of raindrops falling into the tube within a time interval varies from interval to interval, so a frequently made approximation of the total input is a constant current with some fluctuations, as opposed to many individual inputs. Formally, this replacement is achieved by considering the effect of each input to vanish as the rate of arrival goes to infinity (see Lánský, 1984, for a rigorous derivation). The stationary properties of a neuron in this so called diffusion approximation or Fokker–Planck theory (Risken, 1996; Ricciardi et al., 1999) are known since long (Siegert, 1951; Johannesma, 1968; Ricciardi and Sacerdote, 1979; Risken, 1996). For the special case of individual excitatory synaptic impulses the steady state probability density can still be obtained analytically (Sirovich et al., 2000; Sirovich, 2003). If the amplitude of excitatory and inhibitory synaptic impulses are drawn randomly from an exponential distribution, the stationary firing rate is known analytically (Jacobsen and Jensen, 2007). For this case, a recently presented analytical framework also allows to calculate the membrane potential distribution and the response properties up to linear order (Richardson and Swarbrick, 2010).

Before we delve into the theory let us pause for a moment and consider what happens in the vicinity of the threshold in the diffusion approximation in contrast to the situation in the model. In this region, shaded in Figure 2A, random fluctuations of the input cause a threshold crossing the more likely the closer V gets to V_θ. At V_θ, the system immediately crosses the threshold, or fluctuates to a slightly smaller value, since it is impossible to have no fluctuation at all. Hence in the diffusion approximation, the probability p(V) has to drop to zero at V_θ (Brunel et al., 2001) as shown in Figure 2A. However, this is not what happens in the real system where the input is composed of many small input events, drops of water or excitatory synaptic impulses respectively. Here inputs are not received continuously, but rather at specific points in time. Upon reception of such an input event V(t) jumps to a new, slightly larger value. These little jumps happen often, and drive V(t) up close to the threshold. Contrary to the diffusion case, the probability p(V) does not have to approach 0 close to threshold. Instead V(t) jumps over the threshold when the next input event is received, so for all voltages below threshold the picture is just the same. Thus in the actual system, p(V) is flat all over the range from V_r to V_θ, in particular also in the shaded area in Figure 2B. This fact has been known for deterministic input since the work of Knight (1972) and here we extended the argumentation to the arrival of input pulses at random points in time. So apparently the description of the system in the diffusion approximation leads to discrepancies in the probability distribution of V close to the threshold V_θ. Formally, the value p(V_θ) at threshold is called the boundary condition.

As indicated above, the preceding argumentation is based on simplifications in order to explain the concepts. First of all, real neurons do not stay depolarized, but any deviation of the membrane potential from its resting value will decay in the absence of further input because of leak currents through the membrane. In the analogy of the shishi odoshi, such a leak corresponds to a small hole at the lower end of the bamboo tube. Furthermore here we considered excitatory input alone. In case of neurons in the brain, there are both excitatory and inhibitory synaptic inputs from other cells. The original publications deal with the leaky integrate-and-fire model (Helias et al., 2010, 2010a), including leak and inhibition in order to derive a general theory. Qualitatively, however, the above arguments apply just in the same manner. The correction provided by the theory becomes most important for the case of strong mean drive, as in the illustrative example presented here Figure 1. For excitation and inhibition of comparable strength (balanced regime), the deviation of the density from the diffusion approximation is typically smaller.

Depending on what is considered the “code” by which neurons communicate with each other, different aspects of the transformation of the synaptic input signal to the outgoing sequence of action potentials are of interest. Assuming action potentials to appear stochastically, their activity is well described by a time varying firing rate. In this view neurons communicate with a rate code and their transfer properties are determined by the stationary firing rate ν₀. This rate obviously depends on the size of the postsynaptic potentials w elicited by an incoming synaptic impulse and by the frequency λ of their arrival. For the perfect integrator, the firing rate is ν₀ = λw/(V_θ − V_r), because (V_θ − V_r)/w impulses are needed to bring the membrane voltage from reset to threshold. In this particular case the diffusion limit yields exactly the same firing rate. For more realistic neuron models, however, the firing rate differs from the diffusion approximation. In Helias et al. (2010a) we derive an analytical approximation correcting for these deviations for small impulses w, which is simpler than the iterative analytical solution (Cope and Tuckwell, 1979). For exponentially distributed synaptic amplitudes an analytically exact result for the firing rate became available recently (Richardson and Swarbrick, 2010).

Many studies investigating recurrent networks rest on the assumption that a population of neurons can be described by a time varying firing rate and that the barrage of synaptic impulses is well approximated in the diffusion limit, or equivalently, as an effective Gaussian white noise. The time varying firing rate then translates into continuously changing mean and variance of the corresponding Gaussian noise current (Brunel and Hakim, 1999; Lansky and Sacerdote, 2001; Mattia and Del Guidice, 2002, 2004). For small modulations of the population rate the modulation of the neural firing rate can be obtained in linear approximation (Brunel and Hakim, 1999; Brunel et al., 2001; Lindner and Schimansky-Geier, 2001). Non-linear neuron models require numerical solutions (Omurtag et al., 2000; De Kamps, 2003; Richardson, 2007, 2008) and neuron models with several dynamical variables, like synaptic currents, can be treated in the framework of the refractory density approximation (Chizhov and Graham, 2008). The approximation of neural transfer properties by linear response in the diffusion approximation has been proven to successfully capture the global features of recurrent network dynamics (Brunel and Hakim, 1999; Brunel, 2000; Mattia and Del Guidice, 2002, 2004) as well as qualitative properties of the transmission of correlated activity by pairs of neurons (De la Rocha et al., 2007; Shea-Brown et al., 2008). In this section we illustrate which features of neural transfer are missed by neglecting that neurons communicate by synaptic impulses. To this end, we focus on the firing rate of a neuron triggered on the arrival of an incoming synaptic impulse of amplitude s, as illustrated in Figure 1E, assuming the other afferents are active irregularly.

Although the differences in the membrane potential distribution for Gaussian noise input and for finite synaptic jumps are most pronounced within a narrow range of voltages below threshold (Figures 2A,B, shaded area), these differences noticeably affect the transmission of transient signals by a population of neurons. The reason is, that the width of the range is comparable to the amplitude w of a postsynaptic potential of the background activity. A single synaptic impulse of amplitude s can elicit an action potential in the cell, if the membrane potential is closer to threshold than s. This means that the fraction P_inst. of cells that fire a spike in direct response to this impulse is equal to the shaded area in Figure 2C. The shape of the density close to threshold (Figures 2A,B) determines the size of this area. Since for Gaussian white noise background input the density goes to zero at threshold, the response probability to lowest order grows quadratically for small synaptic amplitudes s, as shown in Figure 2D. Synaptic background input of finite impulses causes a finite, positive value of the density at threshold. So there are more neurons ready to fire and the response probability is larger and grows linearly in the amplitude s. Similar arguments have been invoked to explain the fast component in the presence of filtered Gaussian noise (Brunel et al., 2001) and in the spike response model at low noise (Gerstner, 2000).

In the example of the perfect integrator, the response to an incoming synaptic impulse is instantaneous in time. The fraction P_inst. of neurons in the population fires a spike in direct succession to the received input impulse. Also the diffusion approximation yields an instantaneous response for an excitatory impulse, because the event causes an impulse-like co-modulation of the mean and of the variance. The response to modulations of the variance is instantaneous in neurons (Silberberg et al., 2004) and in the integrate-and-fire model (Lindner and Schimansky-Geier, 2001), giving rise to an impulse in the firing rate response.

In the perfect integrator the firing rate immediately returns to its baseline value after the instantaneous response (Figure 3C). The reason is that the probability density remains effectively unchanged by the extra excitatory input: all neurons that where shifted across threshold are reinserted at the lower end of the density as illustrated in Figure 3A. For an inhibitory incoming input, however, the density is shifted away from threshold toward negative voltages. Consequently, after the impulse has been received, the firing rate of the population is zero, since there are no neurons with a voltage close enough to threshold, as shown in Figure 3B. Only gradually, as the neurons receive excitatory background spikes, will the gap in the density below threshold be reoccupied so the population reapproaches its equilibrium firing rate, as shown in Figure 3D. Hence, the responses to excitatory and inhibitory synaptic impulses are asymmetric, as recently observed in somatosensory cortex in vivo (London et al., 2010). In particular, the inhibitory response does not exhibit a fast component. This is in contrast to the prediction of the diffusion approximation, because here an inhibitory impulse causes the same impulse-like perturbation of the variance as an excitatory impulse, so the same fast response is expected in both cases. For exponentially distributed synaptic background pulses, this asymmetry persists. Richardson and Swarbrick (2010) showed that sinusoidal modulations of the rate of excitatory incoming impulses are transmitted in the limit of arbitrary high frequencies, while the modulation of inhibition is suppressed in this limit. The reason is the same as in the case of impulses illustrated in Figure 2C: excitatory synaptic impulses cause jumps over the threshold, whereas inhibitory impulses do not. The diffusion approximation would yield a symmetric result for excitation and inhibition. In motor neurons, Fetz and Gustafsson (1983) found that the integral of the firing rate response follows the time course of the EPSP on its rising flank, but not when the voltage decays. For the integrate-and-fire model neuron, where the EPSPs jump abruptly, this observation directly translates to the impulse-like response present for EPSPs but not for IPSPs. In more realistic models which include a rise time of the EPSP, this immediate response is dispersed over the rising flank of the voltage deflection. Earlier studies pointed out that this sharpening of the response is due to voltage trajectories crossing the threshold on the upstroke of the EPSP and correspondingly found a dependence on the rate of change of the voltage (Herrmann and Gerstner, 2001; Chizhov and Graham, 2008; Goedeke and Diesmann, 2008). Spike response models fitted to simulations of integrate-and-fire models provide a quantitative numerical solution for the non-linear asymmetric response in this case (Herrmann and Gerstner, 2001).

A similar observation is made in the leaky integrate-and-fire model (Helias et al., 2010a). In this case, however, the response to an excitatory impulse not only has an instantaneous component, but the firing rate is also slightly elevated thereafter over a time-span of milliseconds. The reason is the non-uniform probability density between reset and threshold. The shift due to the extra impulse deflects the population from its stationary state and it only gradually relaxes back.

A neuron in the cortex receives synaptic afferents from numerous presynaptic sources. The activity of these inputs often shows only small correlations (Ecker et al., 2010; Hertz, 2010; Renart et al., 2010). Intuitively one would suspect that the presence of the other synaptic inputs acts like noise (indicated as σ in Figure 4A), so that the effect of the single synaptic pulse at t₀ becomes the more negligible, the stronger this noise is. One can therefore ask the question, whether the sequence of action potentials generated by the neuron still contains information about the spike timing of a particular incoming synaptic impulse (London et al., 2010). For some non-linear systems, like neurons, it is known that the transfer of a signal from input to output becomes optimal at a certain level of background noise. The effect is called stochastic resonance (reviewed in McDonnell and Abbott, 2009). Experimentally it has been found in mechanoreceptors of crayfish (Douglass et al., 1993), in the cercal sensory system of crickets (Levin and Miller, 1996), and in human muscle spindles (Cordo et al., 1996). Thresholded Gaussian processes are simple theoretical models exhibiting this phenomenon (Boven and Aertsen, 1990). For the leaky integrate-and-fire neuron in linear approximation stochastic resonance has been reported for sinusoidal periodic input currents (Lindner and Schimansky-Geier, 2001). Also for non-periodic signals which are slow compared to the dynamics of the neuron, an adiabatic approximation reveals stochastic resonance (Collins et al., 1996). Neither approximation, however, is valid for a synaptic impulse, which is a fast non-periodic signal. In (Helias et al., 2010, 2010a) we demonstrate that the non-linear fast response of the leaky integrate-and-fire model exhibits pronounced stochastic resonance.

The intuitive explanation of stochastic resonance in neurons rests on the observation that a single synaptic impulse cannot depolarize the neuron sufficiently to cause an action potential on its own, as a single raindrop will not tilt an empty shishi odoshi. In the presence of many synaptic afferents, or when the shishi odoshi is in torrential rain, there is at each point in time a certain probability that the system is already close to its threshold so that a single additional impulse suffices to ultimately cause an action potential, a single additional raindrop turns the bamboo tube. This explains why the response increases with increasing background noise. In order to understand why the response decreases again when there is too much noise, we need to specify the model a bit more carefully. We consider an integrator as before, but with balanced excitatory and inhibitory synaptic inputs, so that the mean voltage does not change over time. For simplicity, in the following we assume that the mean value corresponds to the reset potential V_r. In the perfect integrator the presence of inhibition causes the problem that the membrane potential might become arbitrarily negative. In neurons this is prevented by the leak current, driving the voltage back to the mean and by the finite value of the inhibitory reversal potential. For the case of V above the mean V_r, we follow Fusi and Mattia (1999), Mattia and Del Guidice (2002) and introduce a constant negative current, so that in the absence of any synaptic input (noise σ = 0), the membrane potential rests at V_r. In their model, arbitrary negative voltages cannot occur because the membrane potential is constrained to V > V_reversal, mimicking the inhibitory reversal potential. In contrast, we imagine the mean potential between reversal and threshold and therefore introduce a current that is constant and positive, if the membrane voltage V is below its mean level V_r. In our operating range we expect the model of Fusi and Mattia (1999) to exhibit qualitatively the same results. Gradually increasing the noise, the probability of finding the neuron at voltages larger than V_r increases; the membrane potential distribution becomes wider, as shown in Figure 4B (blue curve). The density right below threshold grows as well, as indicated in Figure 4C. Here a single additional impulse of size w is sufficient to cause a threshold crossing. For even higher noise the density near threshold ultimately decreases again. The threshold prevents the density to spread beyond, so for larger σ the density is effectively pushed toward negative voltages, as shown in Figure 4B (red curve). Consequently the probability density below threshold assumes a maximum at an optimal noise level σ_opt (Figure 4C) which directly translates into the maximum of the instantaneous response (Figure 4D).

The classical notion of stochastic resonance considers the signal-to-noise ratio in the output to exhibit a maximum at a certain level of noise in the input. In the system at hand, the absolute value of the instantaneous response probability shows a maximum. In this sense, we use the term stochastic resonance in its broader meaning, as recently suggested (McDonnell and Abbott, 2009). The leaky integrate-and-fire neuron exhibits a mechanism in complete analogy to the simple example presented here (Helias et al., 2010, 2010a).

In simulations of recurrent neuronal networks, time is customarily discretized into an equidistant grid (for example Gerstein and Mandelbrot, 1964; MacGregor and Lewis, 1977), see Brette et al. (2007) for a recent review. The spacing of the grid is called the computation step size h. This approach enables the simulation of large neuronal networks on laptop computers and high-performance clusters in an efficient manner. In traditional implementations, the evolution of the membrane potential progresses in steps of h and the times of action potentials are constrained to the time grid. This, however, may cause artificial synchronization in neural networks (Hansel et al., 1998), which can be avoided by resetting the membrane potential at a continuous time threshold crossing found by interpolating between two points on the grid (Hansel et al., 1998; Shelley and Tao, 2001). The accuracy required in a simulation depends on the scientific question. In simulations in discrete time, the error of spike timing drops only linearly with the simulation step size (Morrison et al., 2007). Therefore recently developed algorithms combine the representation of spikes in continuous time with the efficient progression of time in discrete steps (Morrison et al., 2007; Hanuschkin et al., 2010; See Figure 5B). Nevertheless, many simulations still rely on time discretization and if only a low accuracy in spike timing is required grid constrained implementations commonly have shorter run times. This is particularly relevant for simulations involving synaptic plasticity which typically run over long periods of biological time. It is, however, important to distinguish between the accuracy of the mathematical model of nature and the accuracy of the solver used to integrate the equations. Otherwise the researcher cannot decide whether the observed discrepancies with nature are just due to a technical problem or whether the model fails to represent a relevant aspect.

In Helias et al. (2010) we observed that time discretization changes the shape of the membrane potential density. In particular the probability density near threshold increases with coarser time discretization. We have seen above that the shape of the membrane potential distribution determines the response properties of the model. Let us therefore assess the artifacts caused by the discretization of time. Here we characterize the artifacts using the model with constant restoring force introduced in Section 4. A treatment of the full leaky integrate-and-fire model is given in Helias et al. (2010). Figure 5A illustrates the time evolution of the membrane potential density in one cycle of the simulation. The time evolution in neural simulators is typically divided into three different phases (Morrison and Diesmann, 2008): (1) Starting at t, the deterministic neural dynamics evolves. As in the example of the last section, this evolution is a drift of the membrane potential away from threshold, so that the density contracts and a gap emerges between the highest observable voltage (red) and the threshold V_θ. (2) At the end of the time step at t + h the incoming synaptic events are taken into account. Each such event causes a jump of the voltage, indicated by the green arrows in the diagram. This way the synaptic input again spreads the density and the voltage might exceed the threshold, indicated by the narrow stripe of density above threshold at t + h. (3) Due to the firing threshold of the neuron, these supra-threshold states are immediately reset to the reset potential. As a result, the density observed at the end of the third step has a finite value at threshold which is elevated compared to the model in continuous time because all spikes arriving in the interval h are projected to grid position t + h. For the leaky integrate-and-fire neuron, discretization of time slightly lowers the stationary firing rate, while the transient response properties experience only minor changes (Helias et al., 2010). The division of the time evolution operator into three subsequent steps allows us to extend the Fokker–Planck theory to processes evolving in discrete time steps. Next to this approximation, Helias et al. (2010) presents a numerical scheme to obtain the stationary properties of such processes with arbitrary precision.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.