Cortical circuits are modified by experience. It is widely thought that such modifications enhance the representations of behaviorally important sensory stimuli, such as natural scenes for the visual cortex, or speech for the auditory cortex. These modifications can lead to the development or refinement of receptive fields in cortical neurons through synaptic plasticity. Across many species, the amount of cortical plasticity has been shown to depend on age. In juvenile mice, for example, tuning curves can shift from responding maximally to a preferred stimuli to responding maximally to a training stimulus (Dorrn et al., 2010). This is not thought to require neuromodulation and, interestingly, is associated with a state of unbalanced excitation and inhibition. Conversely, experimental data from adults suggest that sensory stimulation is not sufficient to induce a change in receptive fields. Experiments in adult mice suggest that activation of neuromodulatory systems is also necessary for this change (Bear and Singer, 1985; Bakin and Weinberger, 1996; Kilgard and Merzenich, 1998; Shulz et al., 2000; Gu, 2002; Ma and Suga, 2005; Froemke et al., 2007; Drever, 2011; Chun et al., 2013; Martins and Froemke, 2015). In primary auditory cortex, Froemke et al. (2007) observed that repeated exposure to a training sound frequency is not sufficient to evoke experience-dependent plasticity in adult rats. However, the tuning curve shifts to the training frequency if the stimulus is paired with cholinergic stimulation (from Nucleus Basalis). Interestingly, the stability of receptive fields is thought to be associated with the balance between excitation and inhibition, which is observed across many regions in the adult brain (Destexhe and Sejnowski, 2003; Shu et al., 2003; Wehr and Zador, 2003; Haider et al., 2006; Froemke et al., 2007; Okun and Lampl, 2008; Froemke et al., 2013; Graupner and Reyes, 2013; Xue et al., 2014). Experiments indicate that some neuromodulators act to disrupt this balance (Froemke et al., 2007, 2013; Letzkus et al., 2011), enabling cortical plasticity. These neuromodulatory systems may be responsible for communicating the behavioral context of sensory stimuli to other brain regions (Shulz et al., 2000; Gu, 2002).

Neuromodulators are observed to induce different effects in neural circuits. One main neuromodulatory effect is to gate plasticity by modifying the spike-timing-dependent plasticity (STDP) learning window (Bissière et al., 2003; Couey et al., 2007; Seol et al., 2007; Caporale and Dan, 2008; Lin et al., 2008; Pawlak and Kerr, 2008; Shen et al., 2008; Zhang et al., 2009; Pawlak et al., 2010). For example, in lateral amygdala, activation of D2 dopamine receptors was shown to be necessary to induce long-term potentiation (LTP) (Bissière et al., 2003). While in dorsal striatum, dopamine signaling via D1/D5 receptors is required for both long-term potentiation and long-term depression (LTD) (Pawlak and Kerr, 2008). In prefrontal cortex layer 5 pyramidal neurons, nicotine was shown to be able to reverse LTP into LTD (Couey et al., 2007). In visual cortex, the combined action of acetylcholine and noradrenaline is necessary for standard STDP, whereas the action of noradrenaline alone was shown to reverse LTD into LTP, and acetylcholine alone allows only LTD (Seol et al., 2007). Therefore, depending on the brain region and, possibly, the stimulation protocol, neuromodulatory signaling can completely reshape STDP learning windows. Additionally, dopamine has been shown to be important for reinforcement learning (Schultz, 2002). Unlike previous work on dopamine and reinforcement learning, we focus this perspective article on other less explored neuromodulatory effects in an unsupervised learning scheme.

Neuromodulators have also been shown to upregulate neuronal activity. For example, cholinergic stimulation is known to lower feedforward inhibition (Woody and Gruen, 1987; Metherate et al., 1992; Xiang, 1998; Froemke et al., 2007, 2013). Similarly, noradrenaline is known to trigger a disinhibitory effect (Kuo and Trussell, 2011). It has been shown that stimulation of Locus Coeruleus, the main source of noradrenaline, reduces tonic inhibition in auditory cortex (Martins and Froemke, 2015). These disinhibitory mechanisms are thought to be essential for adult cortical plasticity (Hensch, 2005; Letzkus et al., 2011; Kuhlman et al., 2013). Indeed, a computational study by Clopath et al. (2016) demonstrated that a disinhibitory gate is required for adult cortical plasticity. These findings indicate that a disinhibited system promotes learning, consistent with recent experimental work (Letzkus et al., 2011; Kuhlman et al., 2013).

In this perspective article, we review the possible effects of neuromodulators by using a simple computational model of a plastic feedforward network. First, we hypothesize four different learning windows that would result from the action of different neuromodulators. Then we show the consequences of these rules on receptive field plasticity. We verify that an antisymmetric STDP rule allows for receptive field development, whereas a rule with more potentiation allows for a greater modification of sensory representation. Finally, we compare the effect of upregulating the learning rate to the effect of upregulating activity and show that they are not necessarily equivalent. In the simple model, upregulating activity can lead to either a sharpening or a broadening of receptive field tuning. Upregulating the learning rate, however, only amplifies the existing structure.

To illustrate the effect of neuromodulation in cortical plasticity, we use four possible STDP learning rules for excitatory synapses (Figure 1A). Each one of the STDP rules can be thought of as the action of a specific neuromodulatory state. The first rule is the standard antisymmetric STDP rule, in which a presynaptic spike preceding a postsynaptic action potential leads to potentiation of synaptic connections, whereas the reverse leads to depression. We refer to this rule as the Depression-Potentiation (DP) rule (Figure 1A, blue curve). Although widely observed in neocortical neurons of juvenile animals (Markram et al., 1997; Feldman, 2000; Sjöström et al., 2001; Nevian and Sakmann, 2006), the DP rule seems to be neuromodulator dependent in adults. This rule has been observed in visual cortex when both noradrenaline and acetylcholine are present (Seol et al., 2007), whereas in dorsal striatum it can be observed under activation of D1/D5 (dopamine-specific) receptors (Pawlak and Kerr, 2008). The second rule is a symmetrical STDP rule, in which all pairs of pre- and postsynaptic spikes lead to potentiation, regardless of their order. As such, we refer to this as the Potentiation-Potentiation (PP) rule (Figure 1A, red curve). This rule can be observed in adult visual cortex under the activation of β-adrenergic (noradrenaline-specific) receptors (Seol et al., 2007), and in hippocampal neurons under the effect of dopamine (Zhang et al., 2009). For the third rule, only presynaptic spikes followed by postsynaptic action potentials elicit synaptic weight changes, leading to potentiation. Thus, we refer to this as the Unchanged-Potentiation (UP) rule (Figure 1A, green curve). This rule can be associated with dopaminergic action via D2 receptors in Lateral amygdala (Bissière et al., 2003). Lastly, the fourth rule states that synaptic weights are weakened every time a postsynaptic spike precedes presynaptic action potentials, and is unchanged otherwise and hence we refer to this as the Depression-Unchanged (DU) rule (Figure 1A, pink curve). This rule has been reported in prefrontal cortex under nicotinergic modulation (Couey et al., 2007).

We simulate a feedforward network, in which a set of presynaptic input neurons project onto one postsynaptic neuron (Figure 1B). Input neurons fire with a time-varying firing rate with Poisson statistics. Neighboring input neurons have correlated activity, as for example neurons with similar frequency/orientation (sensory feature) preference in auditory/visual cortex (for more details, see Section 4 and Supplementary Figure 1). Using this network, we illustrate the receptive field formation and adaptation under different learning rules, which are shaped by different neuromodulatory states.

In this perspective article, we used four different learning rules, each associated with one or more neuromodulatory states, to illustrate how neuromodulation can affect receptive field plasticity. In order to explore the effects of different neuromodulatory states, we implemented these four learning rules in a feedforward network. As expected (Song et al., 2000), we observed that receptive field development was only possible for one of these rules. This learning rule (DP) can be associated with the combined action of noradrenaline and acetylcholine in visual cortex, or the action of dopamine via D1/D5 receptors in dorsal striatum. It suggests that these neuromodulators can be important to the development of receptive fields in these brain regions, under the assumption that STDP is the dominant player in cortical plasticity. In our analysis, we also asked what would be the best rule to change a receptive field once it is formed. To this end, we combined each learning rule with trained input. This simulates the association of a neuromodulatory state paired with a stimulus (e.g., a tone frequency, or an oriented bar). We observed that the rule with the largest amount of potentiation leads to faster receptive field plasticity. This rule can be associated with the action of noradrenaline on visual cortex, for example. This provides a mechanistic understanding of why noradrenaline can be important to receptive field plasticity in cortical areas, as seen in Martins and Froemke (Martins and Froemke, 2015).

Finally, we asked whether the modulation of presynaptic activity is equivalent to the modulation of plasticity. Our analysis suggests that the upregulation of learning rates can lead to faster learning. However, for low presynaptic activity, strong synaptic weights learn relatively faster than weak weights, regardless of the learning rate amplitude. Since strong weights become stronger, we see a sharpening of receptive field tuning. Upregulating activity, on the other hand, can lead to a scenario in which weak weights learn relatively faster than strong weights. This indicates that modulation of presynaptic activity could lead to a weakening of receptive fields by broadening their tuning.

In this perspective article, we illustrate some of the potential effects that neuromodulators can have in cortical plasticity. By assuming simple neuromodulator-meditated modifications of learning rules, we see interesting differences in the outcomes of receptive field development and adaptation. In all our simulations, we have only used pair-based spike-timing dependent plasticity rules (Gerstner et al., 1993, 1996; Abbott and Blum, 1996; Bi and Poo, 1998; Zhang et al., 1998; Roberts, 1999; Abbott and Nelson, 2000; Mehta et al., 2000; Song et al., 2000; Sjöström et al., 2001; Caporale and Dan, 2008). This allowed us to explore a wide range of different possibilities within a frequently explored and well described framework. Some of the behaviors shown in this perspective article can also be explored analytically, using established techniques of plasticity in feedforward network (Kempter et al., 1999; Song et al., 2000; van Rossum et al., 2000; Kempter et al., 2001; Rubin et al., 2001; Câteau and Fukai, 2003; Izhikevich and Desai, 2003; Zhu et al., 2006; Burkitt et al., 2007; Gilson et al., 2009, 2010; Gjorgjieva et al., 2011; Ocker et al., 2015; Babadi and Abbott, 2016).

Experimental exploration of the effects of neuromodulation in cortical plasticity is a rapidly growing topic of interest. However, the precise effects of neuromodulators in neuronal networks remain unclear, and further experimental data is required. More biologically detailed rules can be explored as descriptions of the effect of neuromodulation on cortical plasticity come to light. Previous voltage-dependent or calcium-dependent models of synaptic plasticity may be modified in this regard (Senn et al., 2001; Shouval et al., 2002; Pfister and Gerstner, 2006; Clopath et al., 2010; Graupner and Brunel, 2012). The effects of neuromodulators under these complex learning rules could be even vaster than for those studied in this perspective article. Moreover, we limited our perspective to a qualitative analysis. With more experimental data and more detailed models, it would be possible to extend this study to a quantitative description.

Synaptic connections in young animals are thought to be highly plastic and are associated with a state of unbalanced excitation and inhibition. In this case, neuromodulation might not be needed for plasticity. However, in adults, experiments indicate that neuromodulation is necessary to open a window of plasticity. We illustrate here that this can be done either by modulating the learning rule (by gating learning) or by modulating neuronal activity (e.g., by disinhibition—Clopath et al., 2016). However, these two scenarios do not yield similar plasticity outcomes in our computational model. In the future, it would be interesting to have more experimental data on how behavior - mediated by neuromodulation - affects both the learning rule and neuronal activity. These insights can then be fed back to computational models to further understand their functional implications.

See Supplementary Material. The code is posted in ModelDB.

VP and CC planned the research, VP performed the simulations and wrote the paper.

This work was supported by CAPES Foundation, process n. 99999.001758/2015-02, by EPSRC, the Google Faculty Award, the Leverhulme Trust and the Wellcome Trust.