Neural systems regulate synaptic plasticity avoiding overly strong growth or shrinkage of the connections, thereby keeping the circuit architecture operational. Accordingly, experimental studies have shown that synaptic weights increase only in direct relation to their current value, resulting in reduced growth for stronger synapses (Bi and Poo, 1998; Sjöström et al., 2001; Frömke et al., 2005). It is, however, difficult to extract unequivocal evidence about the underlying biophysical mechanisms that control weight growth.

The theoretical neurosciences have addressed this problem by exploring mechanisms for synaptic weight change that contain limiting factors to regulate growth. Most basic limiting mechanisms act subtractively or multiplicatively (Bienenstock et al., 1982; Oja, 1982; Miller and MacKay, 1994) such that individual weight growth decreases with the total synaptic weight in a network. Other mechanisms, such as additional thresholds in input or output (Sejnowski and Tesauro, 1989; Gerstner and Kistler, 2002), yield a similar weight-limiting effect. The effectiveness of such mechanisms notwithstanding, some are difficult to justify from a biophysical perspective, in particular those that require knowledge of global network status (e.g., knowledge of the “sum of all weights”) for normalization.

The discovery of spike-timing dependent plasticity (STDP; Gerstner et al., 1996; Magee and Johnston, 1997; Markram et al., 1997) enabling weight growth as well as shrinkage, has offered a potential alternative solution to the problem because STDP can stabilize synaptic weight distributions (Song et al., 2000; Van Rossum et al., 2000; Gütig et al., 2003; Clopath et al., 2010). However, stability is not always guaranteed by STDP because various types of plasticity exist across different neurons and even at the same neuron, depending on the location of the synapses (Frömke et al., 2005; Bender and Feldman, 2006; Sjöström and Häusser, 2006). Weight dependent STDP has been introduced to solve this issue (Van Rossum et al., 2000; Gütig et al., 2003).

In 1998, a series of studies initiated by Turrigiano augmented this discussion by demonstrating that network activity is homeostatically regulated (Turrigiano et al., 1998; Turrigiano and Nelson, 2004; Stellwagen and Malenka, 2006), where overly active networks will down-scale their activity and vice versa. This results from synaptic scaling suggesting that weights w are regulated by the difference between actual and target activity (≈v − v_T).

Here we suggest that synaptic scaling is combined with different types of plasticity mechanisms in the same circuit or even at the same neuron and regulates synaptic diversity across the circuit. Our study demonstrates that synapses are stabilized strictly in an input-determined way thereby capturing characteristic features of the inputs to the network. As an interesting result, we show that such systems are capable of representing a given input pattern via stably changed weights along several stages of signal propagation. This holds even in circuits containing a substantial number of random recurrent connections but no particular additional architecture.

We will first describe the mathematical framework which leads to the analytical as well as numerical observations in the Results Section, a few important equations are repeated there, too. In the second part we will present all parameters and other constraints used.

We will first present some general mathematical results concerning the stability of combined plasticity and scaling mechanisms. This is supplemented by numerical experiments showing how simple single- or multi-synapse systems behave. In the next section we will reproduce some biological findings and finally we will show some predictions that arise from using plasticity and scaling in recurrent networks.

The tendency of real neural networks to achieve firing rate homeostasis by synaptic scaling (Turrigiano et al., 1998; Abbott and Nelson, 2000; Turrigiano and Nelson, 2000, 2004; Stellwagen and Malenka, 2006) has been discussed by these authors as a potential solution to the problem of regulating synaptic weight growth. The current study has shown that plasticity mechanisms augmented by an additive scaling term will lead to synaptic stability but maintain synaptic diversity. Conventionally, plasticity rules are extended by stabilization terms (for example subtractive or multiplicative terms) to assure limited weight growth (Sejnowski and Tesauro, 1989; Miller and MacKay, 1994; Gerstner and Kistler, 2002). Different biophysical mechanisms exist, which achieve this (Desai et al., 1999). In this article we identified the broad and general stabilizing properties of synaptic scaling. Scaling can stabilize a network either independently or in conjunction with other weight regularization mechanisms.

All conventional, correlation-based plasticity rules without additional stabilization terms (e.g., Hebb, Anti-Hebb, STDP, BCM, and others) are limited to a second order weight dependence (G ∼ w²). Thus, the second order form G = aw² + bw characterizes all generic plasticity mechanisms considered in this work. Other plasticity rules have been formulated to explicitly include weight stabilization terms [for example Oja’s rule (Oja, 1982), or rules with subtractive weight normalization (Miller and MacKay, 1994)]. Those rules by construction contain higher order terms that may be stabilizing, even without synaptic scaling. At this point the literature provides a diverse and non-congruent picture about the mathematical form of such terms. In general, one finds that stable fixed points arise for all “correctly constructed” plasticity rules which contain a third order weight dependency. There, the third order needs to be introduced a compensatory, convergent (e.g., negative, divisive) term. Some more recent approaches [e.g., Clopath et al. (2010)] successfully make use of this fixed point behavior implicitly.

The situation becomes even more diverse when considering STDP. The weight stabilization properties of this mechanism have been discussed in many studies and different additional mechanisms have been suggested to improve on this intrinsic property [like weight dependent STDP (Van Rossum et al., 2000) or hard boundaries (Song et al., 2000)]. The current study does not produce any conflict with these results as scaling can co-act with any plain or extended STDP rule. Whether or not such an interaction indeed exists could be measured as suggested in Figure 3. In particular, the strong link between STDP and BCM as discussed next should be considered here. The long time-scale of scaling might indeed remove the distinction between STDP and BCM rules.

Izhikevich and Desai showed in 2003 that BCM and STDP are strongly related (Izhikevich and Desai, 2003). This can be understood if we consider the mean firing rate v of the post-synaptic neuron over a long time interval and distribute the spikes equidistantly in time. This leads to a situation where a high post-synaptic firing rate results in LTP (v > Θ) and a low rate in LTD (v < Θ). This relation holds only for nearest-neighbor STDP (spike-to-spike association), but Pfister and Gerstner (2006) showed that STDP with triplets (three spikes leading to a weight change) is under some conditions equal to BCM, too. In the context of this study one must consider that synaptic scaling is a rather slow process. Thus, immediate effects of STDP, which happen on a much shorter time scale are expected to average out. As a consequence, we expect that on the time scale of synaptic scaling an existing STDP characteristic may indeed be represented by a BCM rule.

Conventionally the BCM rule is endowed with a sliding threshold Θ (Bienenstock et al., 1982). By this the rule is stabilized. We have omitted this aspect and investigate the constant-threshold BCM rule instead. The reason for this is that such a sliding Θ corresponds to a shift of the LTP and/or LTD windows along the ΔT axis in STDP. For wSTDP [weight-dependent STDP, Van Rossum et al. (2000)] the problem is comparable to the sliding threshold scheme. Here the LTD part is weight dependent, resulting in stronger depression for larger synapses. Therefore, again the LTP and LTD windows are changed over time. There is however no clear-cut experimental evidence existing for temporal shifts of the STDP window over long time scales. Thus, the constant-threshold BCM rule may well represent the most generic STDP-compatible rule when considering scaling mechanisms.

Our study indicates that synaptic scaling with sufficiently strong, convex non-linear weight dependence yields stabilization in the presence of all forms of conventional plasticity. This offers a unifying alternative to the currently existing manifold of theories of how synaptic plasticity may be stabilized in neural systems.

Earlier studies have suspected a general dependence of scaling on the current synaptic weight w itself (Abbott and Nelson, 2000; Turrigiano and Nelson, 2004). Our analysis suggests that synaptic scaling actually needs to be weight-dependent and must increase with weight to ensure stability and diversity, a prediction that may be tested experimentally. For plain Hebbian plasticity, positive synapses globally stabilize if scaling is linearly weight dependent (Figure 1B). Generic stabilization also for other plasticity mechanisms is observed for non-linear convex scaling (Figures 1C,D). Little is known to what degree Hebbian mechanisms would also apply to inhibitory synapses (Kullmann and Lamsa, 2007). The analysis also shows that the BCM rule together with synaptic scaling excludes inhibitory synapses as they tend to converge to zero. Thus, stabilization of Hebbian plasticity by a linear synaptic scaling mechanism may indeed represent the most prevalent constellation in real networks. The dependency on w makes scaling a synapse-dependent mechanism as suggested earlier (Rabinowitch and Segev, 2006, 2008). Future experimental studies could address this question by measuring the exponent in H ∼ wⁿ. Currently there are several mechanisms discussed how synaptic scaling might be biophysically realized in neural systems, most of which favor changes in AMPA receptor accumulation at the synaptic site (Turrigiano and Nelson, 1998, 2004). It remains, thus, an interesting question whether and how such an accumulation process would depend on the actual status of the synaptic weight.

Another experimentally measurable parameter is the target firing rate v_T. As mentioned in the “Materials and Methods Section,” the mean firing rate of a neuron will normally not be equal to v_T as synaptic plasticity shifts the synaptic weights and, therefore, the activity to higher values. Thus, only by switching off all synaptic plasticity without influencing the biological mechanisms of synaptic scaling would lead to activities equal to the target firing rate of the synaptic scaling term.

For small groups of neurons (e.g., two or three cells), embedded in the large cortical network, some local connectivity patterns (called “synaptic motifs”) are over-represented as compared to their expected occurrence frequency (Sporns and Kötter, 2004; Bullmore and Sporns, 2009). Our results suggest that plasticity combined with synaptic scaling may naturally stabilize motifs. For instance, in Figure 4 we show that the simplest, bi-directional motif is stabilized depending on the input activity that arrives at the synapses. We conjecture that the synaptic scaling may stabilize more complex motifs, too.

Synaptic patterns (e.g., motifs) are, thus, related to activity patterns (Timme, 2007). The representation of activity patterns and their storage by learning mechanisms is a long-standing problem in neural network theory. In early studies storage has been robustly achieved for static patterns in time-discrete systems [e.g., Hopfield network, Hopfield (1982)]. Such approaches for static pattern representation are still dominating because so far it has been exceedingly difficult to achieve stable representations of spatial-temporal patterns in time-continuous systems. This is problematic, because the dynamic behavioral stability of animals but also of artificial agents (Steingrube et al., 2010) fundamentally relies on the stability of activation patterns, which need to be acquired and stabilized by learning. With spike-timing dependent plasticity it is possible to learn and stabilize weights (Song et al., 2000; Van Rossum et al., 2000). As a consequence STDP has been discussed as a potential mechanism to store input patterns in a network in a stable way thereby creating cell assemblies (Izhikevich, 2006). These assemblies would then activate as soon as the respective input is present (Hebb, 1949; Harris, 2005; Izhikevich, 2006). While promising, only a few studies have actually achieved pattern representation in such STDP-regulated networks. These studies, however, make rather strong, sometimes biophysically questionable assumptions about network topology (Bienenstock, 1995; Hertz and Prügel-Bennett, 1996; Diesmann et al., 1999; Jun and Jin, 2007), and plasticity mechanisms (Sougné, 2001; Matsumoto and Okada, 2002). The analysis here suggests that this is due to intrinsically instable fixed points in systems that change their synaptic weights with STDP without additional mechanisms [see also Kunkel et al. (2010)].

The current study shows that such a storage process is easily achieved by combining plasticity and scaling (Liu and Buonomano, 2009; Rossi Pool and Mato, 2010; Savin et al., 2010). Here, we have specifically addressed the hard problem of a randomly connected network, where recurrent connections in conjunction with synaptic dynamics are often assumed to imply instability (Rossi Pool and Mato, 2010). Contrary to this expectation, we found that a representation trace of the input emerges along several connection stages due to synaptic scaling (Figure 6). This phenomenon is highly robust against variations of the only three existing parameters (μ, γ, and v_T) suggesting that plasticity combined with scaling may solve the problem of pattern representation in dynamic networks.

In the numerical experiments shown, network size and connectivity has been kept small to limit input dispersion and reduce simulation times. Note that already with 1% (Holmgren et al., 2003) to 10% connectivity, similar to that suggested for cortical networks, input dispersion is bounded such that in a micro-circuit of about 10⁶ neurons, a single input may distribute across the entire network in only 2 stages. This estimate clearly shows that the formation of long (multi-stage) input traces by plasticity requires additional, biophysically justifiable developmental constraints on the network topology.

Experimental and theoretical studies have indeed shown that real networks follow certain spatial and topological constraints, which influence the formation of activity patterns (Eguiluz et al., 2005; Bonifazi et al., 2009; Bullmore and Sporns, 2009). It has been discussed (Bullmore and Sporns, 2009) that more complex, non-random topological features can be achieved in a generic way by structural plasticity (Poirazi and Mel, 2001; Chklovskii et al., 2004; Holtmaat and Svoboda, 2009), by which axons and dendrites are anatomically and, thus, functionally rearranged, sometimes over very long time scales.

As shown above, combining plasticity and scaling mechanisms allows generating and storing certain basic assembly structures and creating simple synaptic motifs. Structural plasticity might provide the additionally required mechanisms for obtaining topologically correct, more complex networks that exhibit functionally relevant activity patterns across many more network stages (Tetzlaff et al., 2010). Thus, the dynamic interaction between plasticity, synaptic scaling, and structural plasticity will be a highly relevant topic for future investigations. This study indicates that the stable formation of diverse synaptic patterns may well be possible in this case. This can lead to systems in which synapses and behavioral patterns are stabilized at the same time.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.