<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Comput. Neurosci.</journal-id>
<journal-title>Frontiers in Computational Neuroscience</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Comput. Neurosci.</abbrev-journal-title>
<issn pub-type="epub">1662-5188</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fncom.2013.00131</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Neuroscience</subject>
<subj-group>
<subject>Original Research Article</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>A unified view on weakly correlated recurrent networks</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes">
<name><surname>Grytskyy</surname> <given-names>Dmytro</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="author-notes" rid="fn001"><sup>&#x0002A;</sup></xref>
</contrib>
<contrib contrib-type="author">
<name><surname>Tetzlaff</surname> <given-names>Tom</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
</contrib>
<contrib contrib-type="author">
<name><surname>Diesmann</surname> <given-names>Markus</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
</contrib>
<contrib contrib-type="author">
<name><surname>Helias</surname> <given-names>Moritz</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6), J&#x000FC;lich Research Centre and JARA</institution> <country>J&#x000FC;lich, Germany</country></aff>
<aff id="aff2"><sup>2</sup><institution>Medical Faculty, RWTH Aachen University</institution> <country>Germany</country></aff>
<author-notes>
<fn fn-type="edited-by"><p>Edited by: Ruben Moreno-Bote, Foundation Sant Joan de Deu, Spain</p></fn>
<fn fn-type="edited-by"><p>Reviewed by: Brent Doiron, University of Pittsburgh, USA; Shigeru Shinomoto, Kyoto University, Japan</p></fn>
<fn fn-type="corresp" id="fn001"><p>&#x0002A;Correspondence: Dmytro Grytskyy, J&#x000FC;lich Research Centre and JARA, Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6), Building 15.22, 52425 J&#x000FC;lich, Germany e-mail: <email>d.grytskyy&#x00040;fz-juelich.de</email></p></fn>
<fn fn-type="other" id="fn002"><p>This article was submitted to the journal Frontiers in Computational Neuroscience.</p></fn>
</author-notes>
<pub-date pub-type="epub">
<day>18</day>
<month>10</month>
<year>2013</year>
</pub-date>
<pub-date pub-type="collection">
<year>2013</year>
</pub-date>
<volume>7</volume>
<elocation-id>131</elocation-id>
<history>
<date date-type="received">
<day>22</day>
<month>04</month>
<year>2013</year>
</date>
<date date-type="accepted">
<day>10</day>
<month>09</month>
<year>2013</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x000A9; 2013 Grytskyy, Tetzlaff, Diesmann and Helias.</copyright-statement>
<copyright-year>2013</copyright-year>
<license license-type="open-access" xlink:href="http://creativecommons.org/licenses/by/3.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p>
</license>
</permissions>
<abstract><p>The diversity of neuron models used in contemporary theoretical neuroscience to investigate specific properties of covariances in the spiking activity raises the question how these models relate to each other. In particular it is hard to distinguish between generic properties of covariances and peculiarities due to the abstracted model. Here we present a unified view on pairwise covariances in recurrent networks in the irregular regime. We consider the binary neuron model, the leaky integrate-and-fire (LIF) model, and the Hawkes process. We show that linear approximation maps each of these models to either of two classes of linear rate models (LRM), including the Ornstein&#x02013;Uhlenbeck process (OUP) as a special case. The distinction between both classes is the location of additive noise in the rate dynamics, which is located on the output side for spiking models and on the input side for the binary model. Both classes allow closed form solutions for the covariance. For output noise it separates into an echo term and a term due to correlated input. The unified framework enables us to transfer results between models. For example, we generalize the binary model and the Hawkes process to the situation with synaptic conduction delays and simplify derivations for established results. Our approach is applicable to general network structures and suitable for the calculation of population averages. The derived averages are exact for fixed out-degree network architectures and approximate for fixed in-degree. We demonstrate how taking into account fluctuations in the linearization procedure increases the accuracy of the effective theory and we explain the class dependent differences between covariances in the time and the frequency domain. Finally we show that the oscillatory instability emerging in networks of LIF models with delayed inhibitory feedback is a model-invariant feature: the same structure of poles in the complex frequency plane determines the population power spectra.</p></abstract>
<kwd-group>
<kwd>correlations</kwd>
<kwd>linear response</kwd>
<kwd>Hawkes process</kwd>
<kwd>leaky integrate-and-fire model</kwd>
<kwd>binary neuron</kwd>
<kwd>linear rate model</kwd>
<kwd>Ornstein&#x02013;Uhlenbeck process</kwd>
</kwd-group>
<counts>
<fig-count count="27"/>
<table-count count="2"/>
<equation-count count="90"/>
<ref-count count="60"/>
<page-count count="19"/>
<word-count count="14749"/>
</counts>
</article-meta>
</front>
<body>
<sec sec-type="introduction" id="s1">
<title>1. Introduction</title>
<p>The meaning of correlated neural activity for the processing and representation of information in cortical networks is still not understood, but evidence for a pivotal role of correlations increases (recently reviewed in Cohen and Kohn, <xref ref-type="bibr" rid="B13">2011</xref>). Different studies have shown that correlations can either decrease (Zohary et al., <xref ref-type="bibr" rid="B60">1994</xref>) or increase (Sompolinsky et al., <xref ref-type="bibr" rid="B51">2001</xref>) the signal to noise ratio of population signals, depending on the readout mechanism. The architecture of cortical networks is dominated by convergent and divergent connections among the neurons (Braitenberg and Sch&#x000FC;z, <xref ref-type="bibr" rid="B5">1991</xref>) causing correlated neuronal activity by common input from shared afferent neurons in addition to direct connections between pairs of neurons and common external signals. It has been shown that correlated activity can faithfully propagate through convergent-divergent feed forward structures, such as synfire chains (Abeles, <xref ref-type="bibr" rid="B1">1991</xref>; Diesmann et al., <xref ref-type="bibr" rid="B15">1999</xref>), a potential mechanism to convey signals in the brain. Correlated firing was also proposed as a key to the solution of the binding problem (von der Malsburg, <xref ref-type="bibr" rid="B58">1981</xref>; Bienenstock, <xref ref-type="bibr" rid="B4">1995</xref>; Singer, <xref ref-type="bibr" rid="B50">1999</xref>), an idea that has been discussed controversially (Shadlen and Movshon, <xref ref-type="bibr" rid="B49">1999</xref>). Independent of a direct functional role of correlations in cortical processing, the covariance function between the spiking activity of a pair of neurons contains the information about time intervals between spikes. Changes of synaptic coupling, mediated by spike-timing dependent synaptic plasticity (STDP, Markram et al., <xref ref-type="bibr" rid="B33">1997</xref>; Bi and Poo, <xref ref-type="bibr" rid="B3">1999</xref>), are hence sensitive to correlations. Understanding covariances in spiking networks is thus a prerequisite to investigate the evolution of synapses in plastic networks (Burkitt et al., <xref ref-type="bibr" rid="B11">2007</xref>; Gilson et al., <xref ref-type="bibr" rid="B21">2009</xref>, <xref ref-type="bibr" rid="B22">2010</xref>).</p>
<p>On the other side, there is ubiquitous experimental evidence of correlated spike events in biological neural networks, going back to early reports on multi-unit recordings in cat auditory cortex (Perkel et al., <xref ref-type="bibr" rid="B38">1967</xref>; Gerstein and Perkel, <xref ref-type="bibr" rid="B19">1969</xref>), the observation of closely time-locked spikes appearing at behaviorally relevant points in time (Kilavik et al., <xref ref-type="bibr" rid="B31">2009</xref>; Ito et al., <xref ref-type="bibr" rid="B29">2011</xref>) and collective oscillations in cortex [recently reviewed in Buzs&#x000E1;ki and Wang (<xref ref-type="bibr" rid="B12">2012</xref>)].</p>
<p>The existing theories explaining correlated activity use a multitude of different neuron models. Hawkes (<xref ref-type="bibr" rid="B25">1971</xref>) developed the theory of covariances for linear spiking Poisson neurons (Hawkes processes). Ginzburg and Sompolinsky (<xref ref-type="bibr" rid="B23">1994</xref>) presented the approach of linearization to treat fluctuations around the point of stationary activity and to obtain the covariances for networks of non-linear binary neurons. The formal concept of linearization allowed Brunel and Hakim (<xref ref-type="bibr" rid="B8">1999</xref>) and Brunel (<xref ref-type="bibr" rid="B7">2000</xref>) to explain fast collective gamma oscillations in networks of spiking leaky integrate-and-fire (LIF) neurons. Correlations in feed-forward networks of LIF models are studied in Moreno-Bote and Parga (<xref ref-type="bibr" rid="B34">2006</xref>), exact analytical solutions for such network architectures are given in Rosenbaum and Josic (<xref ref-type="bibr" rid="B46">2011</xref>) for the case of stochastic random walk models, and threshold crossing neuron models are considered in Tchumatchenko et al. (<xref ref-type="bibr" rid="B52">2010</xref>) and Burak et al. (<xref ref-type="bibr" rid="B10">2009</xref>). Covariances in structured networks are investigated for Hawkes processes (Pernice et al., <xref ref-type="bibr" rid="B39">2011</xref>), and in linear approximation for LIF (Pernice et al., <xref ref-type="bibr" rid="B40">2012</xref>) and exponential integrate-and-fire neurons (Trousdale et al., <xref ref-type="bibr" rid="B54">2012</xref>). The latter three works employ an expansion of the propagator (time evolution operator) in terms of the order of interaction. Finally Buice et al. (<xref ref-type="bibr" rid="B9">2009</xref>) investigate higher order cumulants of the joint activity in networks of binary model neurons.</p>
<p>Analytical insight into a neuroscientific phenomenon based on correlated neuronal activity often requires a careful choice of the neuron model to arrive at a solvable problem. Hence a diversity of models has been proposed and is in use. This raises the question which features of covariances are generic properties of recurrent networks and which are specific to a certain model. Only if this question can be answered one can be sure that a particular result is not an artifact of oversimplified neuronal dynamics. Currently it is unclear how different neuron models relate to each other and whether and how results obtained with one model carry over to another. In this work we present a unified theoretical view on pairwise correlations in recurrent networks in the asynchronous and collective-oscillatory regime, approximating the response of different models to linear order. The joint treatment allows us to answer the question of genericness and moreover naturally leads to a classification of the considered models into only two categories, as illustrated in Figure <xref ref-type="fig" rid="F1">1</xref>. The classification in addition enables us to extend existing theoretical results to biologically relevant parameters, such as synaptic delays and the presence of inhibition, and to derive explicit expressions for the time-dependent covariance functions, in quantitative agreement with direct simulations, which can serve as a starting point for further work.</p>
<fig id="F1" position="float">
<label>Figure 1</label>
<caption><p><bold>Mapping different descriptions of neuronal dynamics to linear rate models (LRM)</bold>. The arrows indicate analytical methods which enable a mapping from the original spiking (LIF model, Hawkes model) or binary neuron dynamics to the analytically more tractable linear rate models. Depending on the original dynamics (spiking or binary) the resulting LRM contains an additive noise component <italic>x</italic> either on the output side (left) or on the input side (right).</p></caption>
<graphic xlink:href="fncom-07-00131-g0001.tif"/>
</fig>
<p>The remainder of this article is organized as follows. In the first part of our results in &#x0201C;Covariance structure of noisy rate models&#x0201D; we investigate the activity and the structure of covariance functions for two versions of linear rate models (LRM); one with input the other with output noise. If the activity relaxes exponentially after application of a short perturbation, both models coincide with the OUP. We mainly consider the latter case, although most results hold for arbitrary kernel functions. We extend the analytical solutions for the covariances in networks of OUP (Risken, <xref ref-type="bibr" rid="B45">1996</xref>) to the neuroscientifically important case of synaptic conduction delays. Solutions are derived first for general forms of connectivity in &#x0201C;Solution of the convolution equation with input noise&#x0201D; for input noise and in &#x0201C;Solution of convolution equation with output noise&#x0201D; for output noise. After analyzing the spectral properties of the dynamics in the frequency domain in &#x0201C;Spectrum of the dynamics,&#x0201D; identifying poles of the propagators and their relation to collective oscillations in neuronal networks, we show in &#x0201C;Population-averaged covariances&#x0201D;&#x00027; how to obtain pairwise averaged covariances in homogeneous Erd&#x000F6;s-R&#x000E9;nyi random networks. We explain in detail the use of the residue theorem to perform the Fourier back-transformation of covariance functions to the time domain in &#x0201C;Fourier back transformation&#x0201D; for general connectivity and in &#x0201C;Explicit expression for the population averaged cross covariance in the time domain&#x0201D; for averaged covariance functions in random networks, which allows us to obtain explicit results and to discuss class dependent features of covariance functions.</p>
<p>In the second part of our results in &#x0201C;Binary neurons,&#x0201D; &#x0201C;Hawkes processes,&#x0201D; and &#x0201C;Leaky integrate-and-fire neurons&#x0201D; we consider the mapping of different neuronal dynamics on either of the two flavors of the linear rate models discussed in the first part. The mapping procedure is qualitatively the same for all dynamics as illustrated in Figure <xref ref-type="fig" rid="F1">1</xref>: Starting from the dynamic equations of the respective model, we first determine the working point described in terms of the mean activity in the network. For unstructured homogeneous random networks this amounts to a mean-field description in terms of the population averaged activity (i.e., firing rate in spiking models). In the next step, a linearization of the dynamical equations is performed around this working point. We explain how fluctuations can be considered in the linearization procedure to improve its accuracy and we show how the effective linear dynamics maps to the LRM. We illustrate the results throughout by a quantitative comparison of the analytical results to direct numerical simulations of the original non-linear dynamics. The appendices &#x0201C;Implementation of noisy rate models,&#x0201D; &#x0201C;Implementation of binary neurons in a spiking simulator code,&#x0201D; and &#x0201C;Implementation of Hawkes neurons in a spiking simulator code.&#x0201D; describe the model implementations and are modules of our long-term collaborative project to provide the technology for neural systems simulations (Gewaltig and Diesmann, <xref ref-type="bibr" rid="B20">2007</xref>).</p>
</sec>
<sec>
<title>2. Covariance structure of noisy rate models</title>
<sec>
<title>2.1. Definition of models</title>
<p>Let us consider a network of linear model neurons, each characterized by a continuous fluctuating rate <italic>r</italic> and connections from neuron <italic>j</italic> to neuron <italic>i</italic> given by the element <italic>w</italic><sub><italic>ij</italic></sub> of the connectivity matrix <bold>w</bold>. We assume that the response of neuron <italic>i</italic> to input can be described by a linear kernel <italic>h</italic> so that the activity in the network fulfills
<disp-formula id="E1"><label>(1)</label><mml:math id="M1"><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mi>h</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x025E6;</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02217;</mml:mo><mml:mo stretchy='false'>[</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x025E6;</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>b</mml:mi><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x025E6;</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>]</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
where <italic>f</italic>(&#x025E6; &#x02212; <italic>d</italic>) denotes the function <italic>f</italic> shifted by the delay <italic>d</italic>, <bold>x</bold> is an uncorrelated noise with
<disp-formula id="E2"><label>(2)</label><mml:math id="M2"><mml:mrow><mml:mo>&#x02329;</mml:mo><mml:msub><mml:mi>x</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:mo>=</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mtext>&#x02003;&#x02003;</mml:mtext><mml:mo>&#x02329;</mml:mo><mml:msub><mml:mi>x</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msub><mml:mi>x</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:mo>=</mml:mo><mml:msub><mml:mi>&#x003B4;</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mi>&#x003B4;</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>s</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mtext>&#x02009;</mml:mtext><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
e.g., a Gaussian white noise and (<italic>f</italic> &#x0002A; <italic>g</italic>)(<italic>t</italic>) &#x0003D; &#x0222B;<sup><italic>t</italic></sup><sub>&#x02212; &#x0221E;</sub><italic>f</italic>(<italic>t</italic> &#x02212; <italic>t</italic>&#x02032;) <italic>g</italic>(<italic>t</italic>&#x02032;)<italic>dt</italic>&#x02032; is the convolution. With the particular choice <bold><italic>b</italic></bold> &#x0003D; <bold>w</bold>&#x003B4;(&#x025E6; &#x02212; <italic>d</italic>)&#x0002A; we obtain
<disp-formula id="E3"><label>(3)</label><mml:math id="M3"><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mo stretchy='false'>[</mml:mo><mml:mi>h</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x025E6;</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02217;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x025E6;</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x025E6;</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>]</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>We call the dynamics (3) the linear noisy rate model (LRM) with noise applied to output, as the sum <italic>r</italic> &#x0002B; <italic>x</italic> appears on the right hand side. Alternatively, choosing <bold><italic>b</italic></bold> &#x0003D; <bold>1</bold> we define the model with input noise as
<disp-formula id="E4"><label>(4)</label><mml:math id="M4"><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mi>h</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x025E6;</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02217;</mml:mo><mml:mo stretchy='false'>[</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x025E6;</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x025E6;</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>]</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>Hence, Equations (3) and (4) are special cases of (1). In the following we consider the particular case of an exponential kernel
<disp-formula id="E5"><label>(5)</label><mml:math id="M5"><mml:mrow><mml:mi>h</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>&#x003C4;</mml:mi></mml:mfrac><mml:mi>&#x003B8;</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>s</mml:mi><mml:mo>/</mml:mo><mml:mi>&#x003C4;</mml:mi></mml:mrow></mml:msup><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
where &#x003B8; denotes the Heaviside function, &#x003B8;(<italic>t</italic>) &#x0003D; 1 for <italic>t</italic> &#x0003E; 0, 0 else. Applying to (1) the operator <inline-formula><mml:math id="M6"><mml:mrow><mml:mi>O</mml:mi><mml:mo>=</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mfrac><mml:mi>d</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mi>s</mml:mi></mml:mrow></mml:mfrac><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:math></inline-formula> which has <italic>h</italic> as a Green&#x00027;s function (i.e., <italic>Oh</italic> &#x0003D; &#x003B4;) we get
<disp-formula id="E6"><label>(6)</label><mml:math id="M7"><mml:mrow><mml:mi>&#x003C4;</mml:mi><mml:mfrac><mml:mi>d</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>b</mml:mi><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
which is the equation describing a set of delay coupled Ornstein-Uhlenbeck-processes (OUP) with input or output noise for <bold><italic>b</italic></bold> &#x0003D; <bold>1</bold> or <bold><italic>b</italic></bold> &#x0003D; <bold>w</bold>&#x003B4;(&#x025E6; &#x02212; <italic>d</italic>)&#x0002A;, respectively. We use this representation in &#x0201C;Binary neurons&#x0201D; to show the correspondence to networks of binary neurons.</p>
</sec>
<sec>
<title>2.2. Solution of the convolution equation with input noise</title>
<p>The solution for the system with input noise obtained from the definition (4) after Fourier transformation is
<disp-formula id="E7"><label>(7)</label><mml:math id="M8"><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>R</mml:mi></mml:mstyle><mml:mo>=</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>R</mml:mi></mml:mstyle><mml:mo>+</mml:mo><mml:mi>H</mml:mi><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>X</mml:mi></mml:mstyle><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
where the delay is consumed by the kernel function <inline-formula><mml:math id="M9"><mml:mrow><mml:msub><mml:mi>h</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>&#x003C4;</mml:mi></mml:mfrac><mml:mi>&#x003B8;</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>s</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>s</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>/</mml:mo><mml:mi>&#x003C4;</mml:mi></mml:mrow></mml:msup></mml:mrow></mml:math></inline-formula>. We use capital letters throughout the text to denote objects in the Fourier domain and lower case letters for objects in the time domain. Solved for <bold>R</bold> &#x0003D; (1 &#x02212; <italic>H</italic><sub><italic>d</italic></sub><bold>w</bold>)<sup>&#x02212;1</sup><italic>H</italic><bold>X</bold> the covariance function of <bold>r</bold> in the Fourier domain is found with the Wiener&#x02013;Khinchin theorem (Gardiner, <xref ref-type="bibr" rid="B18">2004</xref>) as &#x02329;<bold>R</bold>(&#x003C9;)<bold>R</bold><sup><italic>T</italic></sup>(&#x02212;&#x003C9;)&#x0232A;, also called the cross spectrum
<disp-formula id="E8"><label>(8)</label><mml:math id="M10"><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>C</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mo>&#x02329;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>R</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>R</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mi>H</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02329;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>X</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>X</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mi>H</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo>&#x02212;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:math></disp-formula>
where we introduced the matrix <bold>D</bold> &#x0003D; &#x02329;<bold>X</bold>(&#x003C9;)<bold>X</bold><sup><italic>T</italic></sup>(&#x02212;&#x003C9;)&#x0232A;. From the second to the third line we used the fact that the non-delayed kernels <italic>H</italic>(&#x003C9;) can be replaced by delayed kernels <italic>H</italic><sub><italic>d</italic></sub>(&#x003C9;) and that the corresponding phase factors <italic>e</italic><sup><italic>i</italic>&#x003C9;<italic>d</italic></sup> and <italic>e</italic><sup>&#x02212; <italic>i</italic>&#x003C9;<italic>d</italic></sup> cancel each other. If <bold>x</bold> is a vector of pairwise uncorrelated noise, <bold>D</bold> is a diagonal matrix and needs to be chosen accordingly in order for the cross spectrum (8) to coincide (neglecting non-linear effects) with the cross spectrum of a network of binary neurons, as described in &#x0201C;Equivalence of binary neurons and Ornstein&#x02013;Uhlenbeck processes&#x0201D;.</p>
</sec>
<sec>
<title>2.3. Solution of convolution equation with output noise</title>
<p>For the system with output noise we consider the quantity <italic>y</italic><sub><italic>i</italic></sub> &#x0003D; <italic>r</italic><sub><italic>i</italic></sub> &#x0002B; <italic>x</italic><sub><italic>i</italic></sub> as the dynamic variable representing the activity of neuron <italic>i</italic> and aim to determine pairwise correlations. It is easy to get from (3) after Fourier transformation
<disp-formula id="E9"><label>(9)</label><mml:math id="M11"><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>R</mml:mi></mml:mstyle><mml:mo>=</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>R</mml:mi></mml:mstyle><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>X</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
which can be solved for <bold>R</bold> &#x0003D; (1 &#x02212; <italic>H</italic><sub><italic>d</italic></sub><bold>w</bold>)<sup>&#x02212;1</sup><italic>H</italic><sub><italic>d</italic></sub><bold>w</bold><bold>X</bold> in order to determine the Fourier transform of <bold>Y</bold> as
<disp-formula id="E10"><label>(10)</label><mml:math id="M12"><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>Y</mml:mi></mml:mstyle><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>R</mml:mi></mml:mstyle><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>X</mml:mi></mml:mstyle><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>X</mml:mi></mml:mstyle><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>The cross spectrum hence follows as
<disp-formula id="E11"><label>(11)</label><mml:math id="M13"><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>C</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mo>&#x02329;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>Y</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>Y</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo>&#x02329;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>X</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>X</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:math></disp-formula>
with <bold>D</bold> &#x0003D; &#x02329;<bold>X</bold>(&#x003C9;)<bold>X</bold><sup><italic>T</italic></sup>(&#x02212;&#x003C9;)&#x0232A;. <bold>D</bold> is a diagonal matrix with the <italic>i</italic>-th diagonal entry &#x003C1;<sup>2</sup><sub><italic>i</italic></sub>. For the correspondence to spiking models <bold>D</bold> must be chosen appropriately, as discussed in &#x0201C;Hawkes processes&#x0201D; and &#x0201C;Leaky integrate-and-fire neurons&#x0201D; for Hawkes processes and LIF neurons, respectively.</p>
</sec>
<sec>
<title>2.4. Spectrum of the dynamics</title>
<p>For both linear rate dynamics, with output and with input noise, the cross spectrum <bold>C</bold>(&#x003C9;) has poles at certain frequencies &#x003C9; in the complex plane. These poles are defined by the zeros of det(<italic>H</italic><sub><italic>d</italic></sub>(&#x003C9;)<sup>&#x02212;1</sup> &#x02212; <bold>w</bold>) and the corresponding term with the opposite sign of &#x003C9;. The zeros of det(<italic>H</italic><sub><italic>d</italic></sub>(&#x003C9;)<sup>&#x02212; 1</sup> &#x02212; <bold>w</bold>) are solutions of the equation
<disp-formula id="E12"><mml:math id="M14"><mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo>+</mml:mo><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>d</mml:mi></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:msub><mml:mi>L</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow></mml:math></disp-formula>
where <italic>L</italic><sub><italic>j</italic></sub> is the <italic>j</italic>-th eigenvalue of <bold>w</bold>. The same set of poles arises from (1) when solving for <bold>R</bold>. For <italic>d</italic> &#x0003E; 0 and the exponential kernel (5), the poles can be expressed as
<disp-formula id="E13"><label>(12)</label><mml:math id="M15"><mml:mrow><mml:msub><mml:mi>z</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>L</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac><mml:mo>&#x02212;</mml:mo><mml:mfrac><mml:mi>i</mml:mi><mml:mi>d</mml:mi></mml:mfrac><mml:msub><mml:mi>W</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:msub><mml:mi>L</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mfrac><mml:mi>d</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mfrac><mml:mi>d</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac></mml:mrow></mml:msup></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
where <italic>W</italic><sub><italic>k</italic></sub> is the <italic>k</italic>-th of the infinitely many branches of the Lambert-W function (Corless et al., <xref ref-type="bibr" rid="B14">1996</xref>). For vanishing synaptic delay <italic>d</italic> &#x0003D; 0 there is obviously only one solution for every <italic>L</italic><sub><italic>j</italic></sub> given by <inline-formula><mml:math id="M16"><mml:mrow><mml:mi>z</mml:mi><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>i</mml:mi></mml:mrow><mml:mi>&#x003C4;</mml:mi></mml:mfrac><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>L</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:math></inline-formula>.</p>
<p>Given the same parameters <italic>d</italic>, <bold>w</bold>, &#x003C4;, the pole structures of the cross spectra of both systems (8) and (11) are identical, since the former can be obtained from the latter by multiplication with (<italic>H</italic><sub><italic>d</italic></sub>(&#x003C9;)<italic>H</italic><sub><italic>d</italic></sub>(&#x02212;&#x003C9;))<sup>&#x02212;1</sup> &#x0003D; (<italic>H</italic>(&#x003C9;)<italic>H</italic>(&#x02212;&#x003C9;))<sup>&#x02212;1</sup>, which has no poles. The only exception causing a different pole structure for the two models is the existence of an eigenvalue <italic>L</italic><sub><italic>j</italic></sub> &#x0003D; 0 of the connectivity matrix <bold>w</bold>, corresponding to a pole <inline-formula><mml:math id="M17"><mml:mrow><mml:mi>z</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac></mml:mrow></mml:math></inline-formula>. However, this pole corresponds to an exponential decay of the covariance for input noise in the time domain and hence does not contribute to oscillations. For output noise, the multiplication with the term (<italic>H</italic>(&#x003C9;)<italic>H</italic>(&#x02212;&#x003C9;))<sup>&#x02212;1</sup>, vanishing at <inline-formula><mml:math id="M18"><mml:mrow><mml:mi>&#x003C9;</mml:mi><mml:mo>=</mml:mo><mml:mfrac><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac></mml:mrow></mml:math></inline-formula>, cancels this pole in the covariance. Consequently both dynamics exhibit similar oscillations. A typical spectrum of poles for a negative eigenvalue <italic>L</italic><sub><italic>j</italic></sub> &#x0003C; 0 is shown in Figures <xref ref-type="fig" rid="F2">2B,D</xref>.</p>
<fig id="F2" position="float">
<label>Figure 2</label>
<caption><p><bold>Pole structure determines dynamics</bold>. Autocovariance of the population activity <bold>(A,C)</bold> measured in &#x003C1;<sup>2</sup>/&#x003C4; and its Fourier transform called power spectrum <bold>(B,D)</bold> of the rate models with output noise (dots) (3) and input noise (diagonal crosses) (4) for delays <italic>d</italic> &#x0003D; 3 ms <bold>(A,B)</bold>, and <italic>d</italic> &#x0003D; 1 ms <bold>(C,D)</bold>. Black symbols show averages over the excitatory population activity and gray symbols over the inhibitory activity obtained by direct simulation. Light gray curves show theoretical predictions for the spectrum (20) and the covariance (21) for output noise and the spectrum (17) and the covariance (18) for input noise. Black crosses (12) in <bold>(B,D)</bold> denote the locations of the poles of the cross spectra - with the real parts corresponding to the damping (vertical axis), and the imaginary parts to oscillation frequencies (horizontal axis). The detailed parameters for this and following figures are given in &#x0201C;Parameters of simulations&#x0201D;.</p></caption>
<graphic xlink:href="fncom-07-00131-g0002.tif"/>
</fig>
</sec>
<sec>
<title>2.5. Population-averaged covariances</title>
<p>Often it is desirable to consider not the whole covariance matrix but averages over subpopulations of pairs of neurons. For instance the average over the whole network would result in a single scalar value. Separately averaging pairs, distinguishing excitatory and inhibitory neuron populations, yields a 2 by 2 matrix of covariances. For these simpler objects closed form solutions can be obtained, which already preserve some useful information and show important features of the network. Averaged covariances are also useful for comparison with simulations and experimental results.</p>
<p>In the following we consider a recurrent random network of <italic>N</italic><sub><italic>e</italic></sub> &#x0003D; <italic>N</italic> excitatory and <italic>N</italic><sub><italic>i</italic></sub> &#x0003D; &#x003B3;<italic>N</italic> inhibitory neurons with synaptic weight <italic>w</italic> for excitatory and &#x02212;<italic>gw</italic> for inhibitory synapses. The probability <italic>p</italic> determines the existence of a connection between two randomly chosen neurons. We study the dynamics averaged over the two subpopulations by introducing the quantities <inline-formula><mml:math id="M19"><mml:mrow><mml:msub><mml:mi>r</mml:mi><mml:mi>a</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:msub><mml:mi>N</mml:mi><mml:mi>a</mml:mi></mml:msub></mml:mrow></mml:mfrac><mml:mstyle displaystyle='true'><mml:msub><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>j</mml:mi><mml:mtext>&#x0200A;</mml:mtext><mml:mo>&#x02208;</mml:mo><mml:mtext>&#x0200A;</mml:mtext><mml:mi>a</mml:mi></mml:mrow></mml:msub><mml:mrow><mml:msub><mml:mi>r</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow></mml:mstyle></mml:mrow></mml:math></inline-formula> and noise terms <inline-formula><mml:math id="M20"><mml:mrow><mml:msub><mml:mi>x</mml:mi><mml:mi>a</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:msub><mml:mi>N</mml:mi><mml:mi>a</mml:mi></mml:msub></mml:mrow></mml:mfrac><mml:mstyle displaystyle='true'><mml:msub><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>j</mml:mi><mml:mtext>&#x0200A;</mml:mtext><mml:mo>&#x02208;</mml:mo><mml:mtext>&#x0200A;</mml:mtext><mml:mi>a</mml:mi></mml:mrow></mml:msub><mml:mrow><mml:msub><mml:mi>x</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow></mml:mstyle></mml:mrow></mml:math></inline-formula> for <italic>a</italic> &#x02208; {<graphic xlink:href="fncom-07-00131-i0002.tif"/>, <graphic xlink:href="fncom-07-00131-i0003.tif"/>}; indices <graphic xlink:href="fncom-07-00131-i0003.tif"/> and <graphic xlink:href="fncom-07-00131-i0002.tif"/> stand for inhibitory and excitatory neurons and corresponding quantities. Calculating the average local input <italic>N</italic><sup>&#x02212;1</sup><sub><italic>a</italic></sub>&#x02211;<sub><italic>j</italic> &#x02208; <italic>a</italic></sub><italic>w</italic><sub><italic>jk</italic></sub><italic>r</italic><sub><italic>k</italic></sub> to a neuron of type <italic>a</italic>, we obtain
<graphic xlink:href="fncom-07-00131-i0001.tif"/>
where, from the second to the third line we used the fact that in expectation a given neuron <italic>k</italic> has <italic>pN</italic><sub><italic>a</italic></sub> targets in the population <italic>a</italic>. The reduction to the averaged system in (13) is exact if in every column <italic>k</italic> in <bold>w</bold><sub><italic>jk</italic></sub> there are exactly <italic>K</italic> non-zero elements for <italic>j</italic> &#x02208; <graphic xlink:href="fncom-07-00131-i0002.tif"/> and &#x003B3;<italic>K</italic> for <italic>j</italic> &#x02208; <graphic xlink:href="fncom-07-00131-i0003.tif"/>, which is the case for networks with fixed out-degree (number of outgoing connections of a neuron to the neurons of a particular type is kept constant), as noted earlier (Tetzlaff et al., <xref ref-type="bibr" rid="B53">2012</xref>). For fixed in-degree (number of connections to a neuron coming in from the neurons of a particular type is kept constant) the substitution of <italic>r</italic><sub><italic>j</italic> &#x02208; <italic>a</italic></sub> by <italic>r</italic><sub><italic>a</italic></sub> is an additional approximation, which could be considered as an average over possible realizations of the random connectivity. In both cases the effective population-averaged connectivity matrix <bold>M</bold> turns out to be
<disp-formula id="E14"><label>(14)</label><mml:math id="M21"><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi></mml:mstyle><mml:mo>=</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mtd></mml:mtr></mml:mtable></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mtext>&#x0200B;</mml:mtext><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
with <italic>K</italic> &#x0003D; <italic>pN</italic>. So the averaged activities fulfill the same Equations (3) and (4) with the non-averaged quantities <bold>r</bold>, <bold>x</bold>, and <bold>w</bold> replaced by their averaged counterparts <overline><bold>r</bold></overline> &#x0003D; (<italic>r</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub>, <italic>r</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub>)<sup><italic>T</italic></sup>, <overline><bold>x</bold></overline> &#x0003D; (<italic>x</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub>, <italic>x</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub>)<sup><italic>T</italic></sup>, and <bold>M</bold>. The population averaged activities <italic>r</italic><sub><italic>a</italic></sub> are directly related to the block-wise averaged covariance matrix <graphic xlink:href="fncom-07-00131-i0007.tif"/>, with <italic>c</italic><sub><italic>ab</italic></sub> &#x0003D; <italic>N</italic><sup>&#x02212;1</sup><sub><italic>a</italic></sub><italic>N</italic><sup>&#x02212;1</sup><sub><italic>b</italic></sub>&#x02211;<sub><italic>i</italic> &#x02208; <italic>a</italic></sub>&#x02211;<sub><italic>j</italic> &#x02208; <italic>b</italic></sub> <italic>c</italic><sub><italic>ij</italic></sub>. With
<disp-formula id="E15"><label>(15)</label><mml:math id="M22"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:msub><mml:mover accent='true'><mml:mi>D</mml:mi><mml:mo>&#x000AF;</mml:mo></mml:mover><mml:mrow><mml:mi>a</mml:mi><mml:mi>b</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:msubsup><mml:mi>N</mml:mi><mml:mi>a</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msubsup><mml:msubsup><mml:mi>N</mml:mi><mml:mi>b</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msubsup><mml:mrow><mml:mo>&#x02329;</mml:mo> <mml:mrow><mml:mstyle displaystyle='true'><mml:munder><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>&#x02208;</mml:mo><mml:mi>a</mml:mi></mml:mrow></mml:munder><mml:mrow><mml:msub><mml:mi>x</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:mstyle><mml:mstyle displaystyle='true'><mml:munder><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>j</mml:mi><mml:mo>&#x02208;</mml:mo><mml:mi>b</mml:mi></mml:mrow></mml:munder><mml:mrow><mml:msub><mml:mi>x</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow></mml:mstyle></mml:mrow> <mml:mo>&#x0232A;</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msubsup><mml:mi>N</mml:mi><mml:mi>a</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msubsup><mml:msubsup><mml:mi>N</mml:mi><mml:mi>b</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msubsup><mml:mstyle displaystyle='true'><mml:munder><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>&#x02208;</mml:mo><mml:mi>a</mml:mi></mml:mrow></mml:munder><mml:mrow><mml:mstyle displaystyle='true'><mml:munder><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>j</mml:mi><mml:mo>&#x02208;</mml:mo><mml:mi>b</mml:mi></mml:mrow></mml:munder><mml:mrow><mml:msub><mml:mi>D</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mstyle></mml:mrow></mml:mstyle></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msub><mml:mi>&#x003B4;</mml:mi><mml:mrow><mml:mi>a</mml:mi><mml:mi>b</mml:mi></mml:mrow></mml:msub><mml:msub><mml:mi>N</mml:mi><mml:mi>a</mml:mi></mml:msub><mml:mo>/</mml:mo><mml:msubsup><mml:mi>N</mml:mi><mml:mi>a</mml:mi><mml:mn>2</mml:mn></mml:msubsup><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>=</mml:mo><mml:msub><mml:mi>&#x003B4;</mml:mi><mml:mrow><mml:mi>a</mml:mi><mml:mi>b</mml:mi></mml:mrow></mml:msub><mml:msubsup><mml:mi>N</mml:mi><mml:mi>a</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msubsup><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
we replace <bold>D</bold> by <inline-formula><mml:math id="M23"><mml:mrow><mml:mover accent='true'><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mo stretchy='true'>&#x000AF;</mml:mo></mml:mover><mml:mo>=</mml:mo><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:msup><mml:mi>N</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mrow></mml:mtd><mml:mtd><mml:mn>0</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>0</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>N</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:math></inline-formula> and <bold>c</bold> by <overline><bold>c</bold></overline> so that the same Equations (11) and (8) and their general solutions also hold for the block-wise averaged covariance matrices.</p>
<p>The covariance matrices separately averaged over pairs of excitatory, inhibitory or mixed pairs are shown in Figure <xref ref-type="fig" rid="F2">2</xref> for both linear rate dynamics (3) and (4). (Parameters for all simulations presented in this article are collected in &#x0201C;Parameters of simulations,&#x0201D; the implementation of LRM is described in &#x0201C;Implementation of noisy rate models&#x0201D;). The poles of both models shown in Figure <xref ref-type="fig" rid="F2">2B</xref> are given by (12) and coincide with the peaks in the cross spectra (8) and (11) for output and input noise, respectively. The results of direct simulation and the theoretical prediction are shown for two different delays, with the longer delay leading to stronger oscillations.</p>
<p>Figure <xref ref-type="fig" rid="F3">3C</xref> shows the distribution of eigenvalues in the complex plane for two random connectivity matrices with different synaptic amplitudes <italic>w</italic>. The model exhibits a bifurcation, if at least one eigenvalue assumes a zero real part. For fixed out-degree the averaging procedure (13) is exact, reflected by the precise agreement of theory and simulation in Figure <xref ref-type="fig" rid="F3">3D</xref>. For fixed in-degree, the averaging procedure (13) is an approximation, which is good only for parameters far from the bifurcation. Even in this regime still small deviations of the theory from the simulation results are visible in Figure <xref ref-type="fig" rid="F3">3B</xref>. On the stable side close to a bifurcation, the appearance of long living modes causes large fluctuations. These weakly damped modes appearing in one particular realization of the connectivity matrix are not represented after the replacement of the full matrix <bold>w</bold> by the average <bold>M</bold> over matrix realizations. The eigenvalue spectrum of the connectivity matrix provides an alternative way to understand the deviations. By the averaging the set of <italic>N</italic> eigenvalues of the connectivity matrix is replaced withby the two eigenvalues of the reduced matrix <bold>M</bold>, one of which is zero due to identical rows of <bold>M</bold>. The eigenvalue spectrum of the full matrix is illustrated in Figure <xref ref-type="fig" rid="F3">3C</xref>. Even if the eigenvalue(s) <italic>L</italic><sup><bold>M</bold></sup> of <bold>M</bold> are far in the stable region (corresponding to <graphic xlink:href="fncom-07-00131-i0010.tif"/>(<italic>z</italic>(<italic>L</italic><sup><bold>M</bold></sup>)) &#x0003E; 0) some eigenvalues <italic>L</italic><sup><bold>w</bold></sup> of the full connectivity matrix in the vicinity of the bifurcation region may still have an imaginary part becoming negative and the system can feel their influence, shown in Figure <xref ref-type="fig" rid="F3">3D</xref>.</p>
<fig id="F3" position="float">
<label>Figure 3</label>
<caption><p><bold>Limits of the theory for fixed in-degree and fixed out-degree</bold>. Autocovariance <bold>(A)</bold> and covariance <bold>(B)</bold> in random networks with fixed in-degree (dots) and fixed out-degree (crosses). Simulation results for <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub>, <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub>, and <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> are shown in dark gray, black and light gray, respectively for synaptic weight <italic>w</italic> &#x0003D; 0.011 far from bifurcation. For larger synaptic weight <italic>w</italic> &#x0003D; 0.018 close to bifurcation (see text at the end of &#x0201C;Population-averaged covariances&#x0201D;), <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub> is also shown in <bold>(D)</bold> for fixed in-degree (dark gray dots) and for fixed out-degree (black dots). Corresponding theoretical predictions for the autocovariance (34) <bold>(A)</bold> and the covariance (18) <bold>(B,D)</bold> are plotted as light gray curves throughout. The set of eigenvalues is shown as black dots in panel <bold>(C)</bold> for the smaller weight. The gray circle denotes the spectral radius <inline-formula><mml:math id="M24"><mml:mrow><mml:mi>w</mml:mi><mml:msqrt><mml:mrow><mml:mi>N</mml:mi><mml:mi>p</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo>+</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:msup><mml:mi>g</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msqrt></mml:mrow></mml:math></inline-formula> (Rajan and Abbott, <xref ref-type="bibr" rid="B43">2006</xref>; Kriener et al., <xref ref-type="bibr" rid="B32">2008</xref>) confining the set of eigenvalues for the larger weight. The small filled gray circle and the triangle show the effective eigenvalues <italic>L</italic> of the averaged systems for small and large weight, respectively.</p></caption>
<graphic xlink:href="fncom-07-00131-g0003.tif"/>
</fig>
</sec>
<sec>
<title>2.6. Fourier back transformation</title>
<p>Although the cross spectral matrices (8) and (11) for both dynamics look similar in the Fourier domain, the procedures for back transformation differ in detail. In both cases, the Fourier integral along the real &#x003C9;-axis can be extended to a closed integration contour by a semi-circle with infinite radius centered at 0 in the appropriately chosen half-plane. The half-plane needs to be selected such that the contribution of the integration along the semi-circle vanishes. By employing the residue theorem (Bronstein et al., <xref ref-type="bibr" rid="B6">1999</xref>) the integral can be replaced by a sum over residua of the poles encircled by the contour. For a general covariance matrix we only need to calculate <bold>c</bold>(<italic>t</italic>) for <italic>t</italic> &#x02265; 0, as for <italic>t</italic> &#x0003C; 0 the solution can be found by symmetry <bold>c</bold>(<italic>t</italic>) &#x0003D; <bold>c</bold><sup><italic>T</italic></sup>(&#x02212;<italic>t</italic>).</p>
<p>For input noise it is possible to close the contour in the upper half-plane where the integrand <bold>C</bold>(&#x003C9;)<italic>e</italic><sup><italic>i</italic>&#x003C9;<italic>t</italic></sup> vanishes for |&#x003C9;| &#x02192; &#x0221E; for all <italic>t</italic> &#x0003E; 0, as |<italic>C</italic><sub><italic>ij</italic></sub>(&#x003C9;)| decays as |&#x003C9;|<sup>&#x02212;2</sup>. This can be seen from (8), because the highest order of <italic>H</italic><sup>&#x02212;1</sup><sub><italic>d</italic></sub> &#x0221D; &#x003C9; appearing in det(<italic>H</italic><sup>&#x02212;1</sup><sub><italic>d</italic></sub> &#x02212; <bold>w</bold>) is equal to the dimensionality <italic>N</italic> of <bold>w</bold> (<italic>N</italic> &#x0003D; 2 for <bold>M</bold>), and in det(adjugate matrix <italic>ij</italic> of <italic>H</italic><sup>&#x02212;1</sup><sub><italic>d</italic></sub> &#x02212; <bold>w</bold>) it is <italic>N</italic> &#x02212; 1 (<italic>i</italic> &#x0003D; <italic>j</italic>) or <italic>N</italic> &#x02212; 2 (<italic>i</italic> &#x02260; <italic>j</italic>). So |(<italic>H</italic><sup>&#x02212;1</sup><sub><italic>d</italic></sub> &#x02212; <bold>w</bold>)<sup>&#x02212;1</sup>| is proportional to |&#x003C9;|<sup>&#x02212;1</sup>|<italic>e</italic><sup>&#x02212; <italic>i</italic>&#x003C9;<italic>d</italic></sup>| and |<bold>C</bold>(&#x003C9;)|&#x0221D;|&#x003C9;|<sup>&#x02212;2</sup> for large |&#x003C9;|.</p>
<p>For the case of output noise (11) <bold>C</bold>(&#x003C9;) can be obtained from the <bold>C</bold>(&#x003C9;) for input noise (8) multiplied with (<italic>H</italic><sub><italic>d</italic></sub>(&#x003C9;)<italic>H</italic><sub><italic>d</italic></sub>(&#x02212;&#x003C9;))<sup>&#x02212;1</sup> &#x0007E; |&#x003C9;|<sup>2</sup> for large |&#x003C9;|. The multiplication with this factor changes the asymptotic behavior of the integrand, which therefore contains terms converging to a constant value and terms decaying like |&#x003C9;|<sup>&#x02212;1</sup> for |&#x003C9;| &#x02192; &#x0221E;. These terms result in non-vanishing integrals over the semicircle in the upper half-plane and have to be considered separately. To this end we rewrite (11) as
<disp-formula id="E16"><label>(16)</label><mml:math id="M25"><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>C</mml:mi></mml:mstyle><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>D</mml:mi></mml:mstyle><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>D</mml:mi></mml:mstyle></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:mtd><mml:mtd><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mn>16</mml:mn></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:math></disp-formula>
and find the constant term <bold>D</bold> which turns into a &#x003B4;-function in the time domain. The first term in the second line of (16) decays like |&#x003C9;|<sup>&#x02212;2</sup> and can be transformed just as <bold>C</bold>(&#x003C9;) for input noise closing the contour in the upper half-plane. The second and third term are the transposed complex conjugates of each other, because of the dependence of <italic>H</italic> on &#x02212;&#x003C9; instead of &#x003C9;, and require a special consideration. Multiplied by <italic>e</italic><sup><italic>i</italic>&#x003C9;<italic>t</italic></sup> under the Fourier integral, the first term is proportional to <italic>H</italic><sub><italic>d</italic></sub><italic>e</italic><sup><italic>i</italic>&#x003C9;<italic>t</italic></sup> &#x0007E; &#x003C9;<sup>&#x02212;1</sup><italic>e</italic><sup><italic>i</italic>&#x003C9;(<italic>t</italic> &#x02212; <italic>d</italic>)</sup> and vanishes faster than |&#x003C9;|<sup>&#x02212;1</sup> for large |&#x003C9;| in the upper half-plane for <italic>t</italic> &#x0003E; <italic>d</italic> and in the lower half plane for <italic>t</italic> &#x0003C; <italic>d</italic>. For the second term the half planes are interchanged. The application of the residue theorem requires closing the integration contour in the half-plane where the integral over the semi-circle vanishes faster than |&#x003C9;|<sup>&#x02212;1</sup>. For <bold>w</bold> &#x0003D; <bold>M</bold> and in the general case of a stable dynamics all poles of the first term are in the upper half-plane <graphic xlink:href="fncom-07-00131-i0010.tif"/>(<italic>z</italic><sub><italic>k</italic></sub>(<italic>L</italic><sub><italic>j</italic></sub>)) &#x0003E; 0, and have no contribution to <bold>c</bold>(<italic>t</italic>) for <italic>t</italic> &#x0003C; <italic>d</italic>. For the second term the same is true for <italic>t</italic> &#x0003E; &#x02212;<italic>d</italic>; these terms correspond to the jumps of <bold>c</bold>(<italic>t</italic>) after one delay, caused by the effect of the sending neuron arriving at the other neurons in the system after one synaptic delay. These terms correspond to the response of the system to the impulse of the sending neuron &#x02013; hence we call them &#x0201C;echo terms&#x0201D; in the following (Helias et al., <xref ref-type="bibr" rid="B27">2013</xref>). The presence of such discontinuous jumps at time points <italic>d</italic> and &#x02212;<italic>d</italic> in the case of output noise is reflected in the convolution of <italic>h</italic><bold>w</bold> with <bold>D</bold> in the time domain in (37). For input noise the absence of discontinuities can be inferred from the absence of such terms in (33), where the derivative of the correlation function is equal to the sum of finite terms. The first summand in (16) corresponds to the covariance evoked by fluctuations propagating through the system originating from the same neuron and we call it &#x0201C;correlated input term&#x0201D;. In the system with input noise a similar separation into effective echo and correlated input terms can be performed. We obtain the correlated input term as the covariance in an auxiliary population without outgoing connections and echo terms as the difference between the full covariance between neurons within the network and the correlated input term.</p>
</sec>
<sec>
<title>2.7. Explicit expression for the population averaged cross covariance in the time domain</title>
<p>We obtain the population averaged cross spectrum in a recurrent random network of Ornstein&#x02013;Uhlenbeck processes (OUP) with input noise by inserting the averaged connectivity matrix <bold>w</bold> &#x0003D; <bold>M</bold> (14) into (8). The explicit expression for the covariance function follows by taking into account all (both) eigenvalues of <bold>M</bold> with values 0 and <italic>L</italic> &#x0003D; <italic>Kw</italic>(1 &#x02212; &#x003B3;<italic>g</italic>). The detailed derivation of the results presented in this section are documented in &#x0201C;Calculation of the Population Averaged Cross Covariance in Time Domain&#x0201D;. The expression for the cross spectrum (8) takes the form
<disp-formula id="E17"><label>(17)</label><mml:math id="M26"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>C</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:mo>+</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:mo>+</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
where we introduced <italic>f</italic>(&#x003C9;) &#x0003D; (<italic>H</italic><sub><italic>d</italic></sub>(&#x003C9;)<sup>&#x02212;1</sup> &#x02212; <italic>L</italic>)<sup>&#x02212;1</sup> as a short hand. Sorting the terms by their dependence on &#x003C9;, introducing the functions &#x003A6;<sub>1</sub>(&#x003C9;),&#x02026;,&#x003A6;<sub>4</sub>(&#x003C9;) for this dependence, and &#x003C6;<sub>1</sub>(<italic>t</italic>),&#x02026;,&#x003C6;<sub>4</sub>(<italic>t</italic>) for the corresponding functions in the time domain, the covariance in the time domain <inline-formula><mml:math id="M27"><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:mn>2</mml:mn><mml:mi>&#x003C0;</mml:mi></mml:mrow></mml:mfrac><mml:mstyle displaystyle='true'><mml:mrow><mml:msubsup><mml:mo>&#x0222B;</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x0221E;</mml:mi></mml:mrow><mml:mrow><mml:mo>+</mml:mo><mml:mi>&#x0221E;</mml:mi></mml:mrow></mml:msubsup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>C</mml:mi></mml:mstyle></mml:mrow></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:msup><mml:mi>d</mml:mi><mml:mi>&#x003C9;</mml:mi></mml:mrow></mml:math></inline-formula> takes the form
<disp-formula id="E18"><mml:math id="M28"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>3</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:msup><mml:mi>K</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:msup><mml:mi>w</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>4</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>The previous expression is valid for arbitrary <bold>D</bold>. In simulations presented in this article we consider identical marginal input statistics for all neurons. In this case the averaged activities for excitatory and inhibitory neurons are the same, so we can insert the special form of <bold>D</bold> given in (15), which results in
<disp-formula id="E19"><label>(18)</label><mml:math id="M29"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow><mml:mi>N</mml:mi></mml:mfrac><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mn>0</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>0</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:msup><mml:mi>&#x003B3;</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:mfrac><mml:mrow><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow><mml:mi>N</mml:mi></mml:mfrac><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>g</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mi>&#x003B3;</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mfrac><mml:mrow><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow><mml:mi>N</mml:mi></mml:mfrac><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mi>&#x003B3;</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>3</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:mfrac><mml:mrow><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow><mml:mi>N</mml:mi></mml:mfrac><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>K</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:msup><mml:mi>w</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:msup><mml:mi>g</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:mtd><mml:mtd><mml:mi>g</mml:mi></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mi>g</mml:mi></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mi>&#x003B3;</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>4</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>The time-dependent functions &#x003C6;<sub>1</sub>,&#x02026;,&#x003C6;<sub>4</sub> are the same in both cases. Using the residue theorem <inline-formula><mml:math id="M30"><mml:mrow><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:mn>2</mml:mn><mml:mi>&#x003C0;</mml:mi></mml:mrow></mml:mfrac><mml:mstyle displaystyle='true'><mml:mrow><mml:msubsup><mml:mo>&#x0222B;</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x0221E;</mml:mi></mml:mrow><mml:mrow><mml:mo>+</mml:mo><mml:mi>&#x0221E;</mml:mi></mml:mrow></mml:msubsup><mml:mrow><mml:msub><mml:mi>&#x003A6;</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:mrow></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:msup><mml:mi>d</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mo>=</mml:mo><mml:mi>i</mml:mi><mml:mstyle displaystyle='true'><mml:msub><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>z</mml:mi><mml:mo>&#x02208;</mml:mo><mml:mtext>poles&#x000A0;of&#x000A0;</mml:mtext><mml:msub><mml:mi>&#x003A6;</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:msub><mml:mrow><mml:mtext>Res</mml:mtext></mml:mrow></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>&#x003A6;</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>,</mml:mo><mml:mi>z</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>z</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:msup></mml:mrow></mml:math></inline-formula> for <italic>t</italic> &#x02265; 0 they can be expressed as a sum over the poles <italic>z</italic><sub><italic>k</italic></sub>(<italic>L</italic>) given by (12) and the pole <inline-formula><mml:math id="M31"><mml:mrow><mml:mi>z</mml:mi><mml:mo>=</mml:mo><mml:mfrac><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac></mml:mrow></mml:math></inline-formula> of <italic>H</italic><sub><italic>d</italic></sub>(&#x003C9;). At &#x003C9; &#x0003D; <italic>z</italic><sub><italic>k</italic></sub>(<italic>L</italic>) the residue of <italic>f</italic>(&#x003C9;) is Res(<italic>f</italic>, &#x003C9; &#x0003D; <italic>z</italic><sub><italic>k</italic></sub>(<italic>L</italic>)) &#x0003D; (<italic>idL</italic> &#x0002B; <italic>i</italic>&#x003C4;<italic>e</italic><sup><italic>i</italic>&#x003C9;<italic>d</italic></sup>)<sup>&#x02212;1</sup>, the residue of <italic>H</italic><sub><italic>d</italic></sub>(&#x003C9;) at <inline-formula><mml:math id="M32"><mml:mrow><mml:mi>z</mml:mi><mml:mo>=</mml:mo><mml:mfrac><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac></mml:mrow></mml:math></inline-formula> is <inline-formula><mml:math id="M33"><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mfrac><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mo>/</mml:mo><mml:mi>&#x003C4;</mml:mi></mml:mrow></mml:msup></mml:mrow></mml:math></inline-formula>, so that the explicit forms of &#x003C6;<sub>1</sub>,&#x02026;,&#x003C6;<sub>4</sub> follow as
<disp-formula id="E20"><label>(19)</label><mml:math id="M34"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:munder><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi><mml:mo>=</mml:mo><mml:msub><mml:mi>z</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>L</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:munder><mml:mi>i</mml:mi></mml:mstyle><mml:mtext>Res</mml:mtext><mml:mo stretchy='false'>(</mml:mo><mml:mi>f</mml:mi><mml:mo>,</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:munder><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi><mml:mo>=</mml:mo><mml:msub><mml:mi>z</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>L</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:munder><mml:mi>i</mml:mi></mml:mstyle><mml:mtext>Res</mml:mtext><mml:mo stretchy='false'>(</mml:mo><mml:mi>f</mml:mi><mml:mo>,</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:mfrac><mml:mrow><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>d</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>/</mml:mo><mml:mi>&#x003C4;</mml:mi></mml:mrow></mml:msup></mml:mrow><mml:mi>&#x003C4;</mml:mi></mml:mfrac><mml:mi>f</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mfrac><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mi>f</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mfrac><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>3</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:munder><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi><mml:mo>=</mml:mo><mml:msub><mml:mi>z</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>L</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:munder><mml:mi>i</mml:mi></mml:mstyle><mml:mtext>Res</mml:mtext><mml:mo stretchy='false'>(</mml:mo><mml:mi>f</mml:mi><mml:mo>,</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>4</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:munder><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi><mml:mo>=</mml:mo><mml:msub><mml:mi>z</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>L</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:munder><mml:mi>i</mml:mi></mml:mstyle><mml:mtext>Res</mml:mtext><mml:mo stretchy='false'>(</mml:mo><mml:mi>f</mml:mi><mml:mo>,</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:mfrac><mml:mrow><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>t</mml:mi><mml:mo>/</mml:mo><mml:mi>&#x003C4;</mml:mi></mml:mrow></mml:msup></mml:mrow><mml:mrow><mml:mn>2</mml:mn><mml:mi>&#x003C4;</mml:mi></mml:mrow></mml:mfrac><mml:mi>f</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mfrac><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mi>f</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mfrac><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>The corresponding expression for <bold>C</bold>(&#x003C9;) for output noise is obtained by multiplying (17) with <italic>H</italic><sup>&#x02212;1</sup><sub><italic>d</italic></sub>(&#x003C9;)<italic>H</italic><sup>&#x02212;1</sup><sub><italic>d</italic></sub>(&#x02212;&#x003C9;) &#x0003D; (1 &#x0002B; &#x003C9;<sup>2</sup>&#x003C4;<sup>2</sup>)
<disp-formula id="E21"><label>(20)</label><mml:math id="M35"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mtext>&#x0200B;&#x0200B;</mml:mtext><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>C</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:msubsup><mml:mi>H</mml:mi><mml:mi>d</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msubsup><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msubsup><mml:mi>H</mml:mi><mml:mi>d</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msubsup><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>&#x000D7;</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:mo>+</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:mo>+</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
which, after Fourier transform, provides the expression for <bold>c</bold>(<italic>t</italic>) in the time domain for <italic>t</italic> &#x02265; 0
<disp-formula id="E22"><label>(21)</label><mml:math id="M36"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi><mml:mi>D</mml:mi></mml:mstyle><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi><mml:mi>D</mml:mi></mml:mstyle><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>0</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mi>&#x003B4;</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msup><mml:mi>K</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:msup><mml:mi>w</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mfrac><mml:mrow><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow><mml:mi>N</mml:mi></mml:mfrac><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo>+</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:msup><mml:mi>g</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mn>1</mml:mn></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mfrac><mml:mrow><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow><mml:mi>N</mml:mi></mml:mfrac><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>g</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>g</mml:mi></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>0</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mfrac><mml:mrow><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow><mml:mi>N</mml:mi></mml:mfrac><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mn>0</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>0</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:msup><mml:mi>&#x003B3;</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mi>&#x003B4;</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>As in (18), the first line holds for arbitrary <bold>D</bold>, and the second for <bold>D</bold> given by (15), valid if the firing rates are homogeneous. &#x003C6;<sub>1</sub> is defined as before, and
<disp-formula id="E23"><label>(22)</label><mml:math id="M37"><mml:mrow><mml:msub><mml:mi>&#x003C6;</mml:mi><mml:mn>0</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mi>&#x003B8;</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mstyle displaystyle='true'><mml:munder><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi><mml:mo>=</mml:mo><mml:msub><mml:mi>z</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>L</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:munder><mml:mrow><mml:msup><mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mi>d</mml:mi><mml:mi>L</mml:mi><mml:mo>+</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>d</mml:mi></mml:mrow></mml:msup></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mrow></mml:mstyle><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:msup></mml:mrow></mml:math></disp-formula>
vanishes for <italic>t</italic> &#x0003C; <italic>d</italic>. All matrix elements of the first term in (21) are identical. Therefore all elements of <bold>c</bold>(<italic>t</italic>) are equal for 0 &#x0003C; |<italic>t</italic>| &#x0003C; <italic>d</italic>. Both rows of the matrix in front of &#x003C6;<sub>0</sub> are identical, so for <italic>t</italic> &#x0003E; 0 the off diagonal term <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub> coincides with <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub> and <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> with <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> and vice versa for <italic>t</italic> &#x0003C; 0.</p>
<p>As an illustration we show the functions &#x003C6;<sub>0</sub>,&#x02026;,&#x003C6;<sub>4</sub> for one set of parameters in Figure <xref ref-type="fig" rid="F4">4</xref>. The left panels <bold>(A,C)</bold> correspond to contributions to the covariance caused by common input to a pair of neurons, the right panels <bold>(B,D)</bold> to terms due to the effect of one of the neurons&#x00027; activities on the remaining network (echo terms). The upper panels <bold>(A,B)</bold> belong to the model with input noise, the lower panel <bold>(C,D)</bold> to the one with output noise.</p>
<fig id="F4" position="float">
<label>Figure 4</label>
<caption><p><bold>Functions &#x003C6;<sub>0</sub> (D), &#x003C6;<sub>1</sub> (C), &#x003C6;<sub>2</sub>, &#x003C6;<sub>3</sub> (B), &#x003C6;<sub>4</sub> (A) introduced in (19) and (22) for decomposition of covariance <bold>c</bold>(<italic>t</italic>)</bold>. In <bold>(B)</bold> &#x003C6;<sub>3</sub>(&#x02212;<italic>t</italic>) is shown in gray and &#x003C6;<sub>2</sub>(<italic>t</italic>) in black. The two functions are continuations of each other, joint at <italic>t</italic> &#x0003D; 0. Both functions appear in the echo term for input noise. The function &#x003C6;<sub>0</sub> in <bold>(D)</bold> describing the corresponding echo term in the case of output noise is shifted to be aligned with the function in <bold>(B)</bold> to facilitate the comparison of <bold>(B,D)</bold>. Parameters in all panels are <italic>d</italic> &#x0003D; 3 ms, &#x003C4; &#x0003D; 10 ms, <italic>L</italic> &#x0003D; &#x02212;1.72.</p></caption>
<graphic xlink:href="fncom-07-00131-g0004.tif"/>
</fig>
<p>For the rate dynamics with output noise, the term with &#x003C6;<sub>1</sub> in (21) (shown in Figure <xref ref-type="fig" rid="F4">4C</xref>) is symmetric and describes the common input covariance and the term with &#x003C6;<sub>0</sub> (shown in Figure <xref ref-type="fig" rid="F4">4D</xref>) is the echo part of the covariance. For the rate dynamics with input noise (18) the term containing &#x003C6;<sub>4</sub> (shown in Figure <xref ref-type="fig" rid="F4">4A</xref>) is caused by common input and is hence also symmetric, the terms with &#x003C6;<sub>2</sub> and &#x003C6;<sub>3</sub> (shown in Figure <xref ref-type="fig" rid="F4">4B</xref>) correspond to the echo part and have hence their peak outside the origin. The second echo term in (18) is equal to the first one transposed and with opposite sign of the time argument, so we show &#x003C6;<sub>2</sub>(<italic>t</italic>) and &#x003C6;<sub>3</sub>(&#x02212;<italic>t</italic>) together in one panel in Figure <xref ref-type="fig" rid="F4">4B</xref>. Note that for input noise, the term with &#x003C6;<sub>1</sub> describes the autocovariance, which corresponds to the term with the &#x003B4;-function in case of output noise.</p>
<p>The solution (18) is visualized in Figure <xref ref-type="fig" rid="F6">6</xref>, the solution (21) in Figure <xref ref-type="fig" rid="F7">7</xref>, and the decomposition into common input and echo parts is also shown and compared to direct simulations in Figure <xref ref-type="fig" rid="F8">8</xref>.</p>
</sec>
</sec>
<sec>
<title>3. Binary neurons</title>
<p>In the following sections we study, in turn, the binary neuron model, the Hawkes model and the LIF model and show how they can be mapped to one of the two OUPs; either the one with input or the one with output noise, so that the explicit solutions (18) and (21) for the covariances derived in the previous section can be applied. In the present section, we start with the binary neuron model (Ginzburg and Sompolinsky, <xref ref-type="bibr" rid="B23">1994</xref>; Buice et al., <xref ref-type="bibr" rid="B9">2009</xref>).</p>
<p>Following Ginzburg and Sompolinsky (<xref ref-type="bibr" rid="B23">1994</xref>) the state of the network of <italic>N</italic> binary model neurons is described by a binary vector <bold>n</bold> &#x02208; {0, 1}<sup><italic>N</italic></sup> and each neuron is updated at independently drawn time points with exponentially distributed intervals of mean duration &#x003C4;. This stochastic update constitutes a source of noise in the system. Given the <italic>i</italic>-th neuron is updated, the probability to end in the up-state (<italic>n</italic><sub><italic>i</italic></sub> &#x0003D; 1) is determined by the gain function <italic>F</italic><sub><italic>i</italic></sub>(<bold>n</bold>) which depends on the activity <bold>n</bold> of all other neurons. The probability to end in the down state (<italic>n</italic><sub><italic>i</italic></sub> &#x0003D; 0) is 1 &#x02212; <italic>F</italic><sub><italic>i</italic></sub>(<bold>n</bold>). Here we implemented the binary model in the NEST simulator (Gewaltig and Diesmann, <xref ref-type="bibr" rid="B20">2007</xref>) as described in &#x0201C;Implementation of Binary Neurons in a Spiking Simulator~Code&#x0201D;. Such systems have been considered earlier (Ginzburg and Sompolinsky, <xref ref-type="bibr" rid="B23">1994</xref>; Buice et al., <xref ref-type="bibr" rid="B9">2009</xref>), and here we follow the notation employed in the latter work. In the following we collect results that have been derived in these works and refer the reader to these publications for the details of the derivations. The zero-time lag covariance function is defined as <italic>c</italic><sub><italic>ij</italic></sub>(<italic>t</italic>) &#x0003D; &#x02329;<italic>n</italic><sub><italic>i</italic></sub>(<italic>t</italic>)<italic>n</italic><sub><italic>j</italic></sub>(<italic>t</italic>)&#x0232A; &#x02212; <italic>a</italic><sub><italic>i</italic></sub>(<italic>t</italic>)<italic>a</italic><sub><italic>j</italic></sub>(<italic>t</italic>), with the expectation value &#x02329;&#x0232A; taken over different realizations of the stochastic dynamics. Here <bold>a</bold>(<italic>t</italic>) &#x0003D; (<italic>a</italic><sub>1</sub>(<italic>t</italic>),&#x02026;,<italic>a</italic><sub><italic>N</italic></sub>(<italic>t</italic>))<sup><italic>T</italic></sup> is the vector of mean activities <italic>a</italic><sub><italic>i</italic></sub>(<italic>t</italic>) &#x0003D; &#x02329;<italic>n</italic><sub><italic>i</italic></sub>(<italic>t</italic>)&#x0232A;. <italic>c</italic><sub><italic>ij</italic></sub>(<italic>t</italic>) fulfills the differential equation
<disp-formula id="E24"><mml:math id="M38"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mi>&#x003C4;</mml:mi><mml:mfrac><mml:mi>d</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:msub><mml:mi>c</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mn>2</mml:mn><mml:msub><mml:mi>c</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mo>&#x02329;</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>n</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>a</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:msub><mml:mi>F</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>n</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:mo>&#x02329;</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>n</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>a</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:msub><mml:mi>F</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>n</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>In the stationary state, the covariance therefore fulfills
<disp-formula id="E25"><label>(23)</label><mml:math id="M39"><mml:mrow><mml:msub><mml:mi>c</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mn>2</mml:mn></mml:mfrac><mml:mo>&#x02329;</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>n</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>a</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:msub><mml:mi>F</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>n</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:mo>+</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mn>2</mml:mn></mml:mfrac><mml:mo>&#x02329;</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>n</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>a</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:msub><mml:mi>F</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>n</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>The time lagged covariance <italic>c</italic><sub><italic>ij</italic></sub>(<italic>t, s</italic>) &#x0003D; &#x02329;<italic>n</italic><sub><italic>i</italic></sub>(<italic>t</italic>)<italic>n</italic><sub><italic>j</italic></sub>(<italic>s</italic>)&#x0232A; &#x02212; <italic>a</italic><sub><italic>i</italic></sub>(<italic>t</italic>)<italic>a</italic><sub><italic>j</italic></sub>(<italic>s</italic>) fulfills for <italic>t</italic> &#x0003E; <italic>s</italic> the differential equation
<disp-formula id="E26"><label>(24)</label><mml:math id="M40"><mml:mrow><mml:mi>&#x003C4;</mml:mi><mml:mfrac><mml:mi>d</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:msub><mml:mi>c</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>c</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mo>&#x02329;</mml:mo><mml:msub><mml:mi>F</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>n</mml:mi></mml:mstyle><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>n</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>a</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>This equation is also true for <italic>i</italic> &#x0003D; <italic>j</italic>, the autocovariance. The term &#x02329;<italic>F</italic><sub><italic>i</italic></sub>(<bold>n</bold>, <italic>t</italic>)(<italic>n</italic><sub><italic>j</italic></sub>(<italic>s</italic>) &#x02212; <italic>a</italic><sub><italic>j</italic></sub>(<italic>s</italic>))&#x0232A; has a simple interpretation: it measures the influence of a fluctuation of neuron <italic>j</italic> at time <italic>s</italic> around its mean value on the gain of neuron <italic>i</italic> at time <italic>t</italic> (Ginzburg and Sompolinsky, <xref ref-type="bibr" rid="B23">1994</xref>). We now assume a particular form for the coupling between neurons
<disp-formula id="E27"><label>(25)</label><mml:math id="M41"><mml:mrow><mml:msub><mml:mi>F</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>n</mml:mi></mml:mstyle><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mi>&#x003D5;</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>J</mml:mi></mml:mstyle><mml:mi>i</mml:mi></mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>n</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mi>&#x003D5;</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mstyle displaystyle='true'><mml:munderover><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:munderover><mml:mrow><mml:msub><mml:mi>J</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>k</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mstyle><mml:msub><mml:mi>n</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
where <bold>J</bold><sub><italic>i</italic></sub> is the vector of incoming synaptic weights into neuron <italic>i</italic> and &#x003D5; is a non-linear gain function. Assuming that the fluctuations of the total input <bold>J</bold><sub><italic>i</italic></sub><bold>n</bold> into the <italic>i</italic>-th neuron are sufficiently small to allow a linearization of the gain function &#x003D5;, we obtain the Taylor expansion
<disp-formula id="E28"><mml:math id="M42"><mml:mrow><mml:msub><mml:mi>F</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>n</mml:mi></mml:mstyle><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:msub><mml:mi>F</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>a</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msup><mml:mi>&#x003D5;</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>J</mml:mi></mml:mstyle><mml:mi>i</mml:mi></mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>a</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>J</mml:mi></mml:mstyle><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>n</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>a</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
where
<disp-formula id="E29"><label>(26)</label><mml:math id="M43"><mml:mrow><mml:msup><mml:mi>&#x003D5;</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>J</mml:mi></mml:mstyle><mml:mi>i</mml:mi></mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>a</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:math></disp-formula>
is the slope of the gain function at the point of mean input.</p>
<p>Up to this point the treatment of the system is identical to the work of Ginzburg and Sompolinsky (<xref ref-type="bibr" rid="B23">1994</xref>). Now we present an alternative approach for the linearization which takes into account the effect of fluctuations in the input. For sufficiently asynchronous network states, the fluctuations in the input <bold>J</bold><sub><italic>i</italic></sub><bold>n</bold>(<italic>t</italic> &#x02212; <italic>d</italic>) to neuron <italic>i</italic> can be approximated by a Gaussian distribution <graphic xlink:href="fncom-07-00131-i0004.tif"/>(&#x003BC;, &#x003C3;). In the following we consider a homogeneous random network with fixed in-degree as described in &#x0201C;Population-averaged covariances&#x0201D;. As each neuron receives the same number <italic>K</italic> of excitatory and &#x003B3;<italic>K</italic> inhibitory synapses, the marginal statistics of the summed input to each neuron is identical. The mean input to a neuron then is &#x003BC; &#x0003D; <italic>KJ</italic>(1 &#x02212; &#x003B3;<italic>g</italic>)<italic>a</italic>, where <italic>a</italic> is the mean activity of a neuron in the network. If correlations are small, the variance of this input signal distribution can be approximated as the sum of the variances of the individual contributions from the incoming signals, resulting in &#x003C3;<sup>2</sup> &#x0003D; <italic>KJ</italic><sup>2</sup>(1 &#x0002B; &#x003B3;<italic>g</italic><sup>2</sup>)<italic>a</italic>(1 &#x02212; <italic>a</italic>), where we used the fact that the variance of a binary variable with mean <italic>a</italic> is <italic>a</italic>(1 &#x02212; <italic>a</italic>). This results from a direct calculation: since <italic>n</italic> &#x02208; {0, 1}, <italic>n</italic><sup>2</sup> &#x0003D; <italic>n</italic>, so that the variance is &#x02329;<italic>n</italic><sup>2</sup>&#x0232A; &#x02212; &#x02329;<italic>n</italic>&#x0232A;<sup>2</sup> &#x0003D; &#x02329;<italic>n</italic>&#x0232A; &#x02212; &#x02329;<italic>n</italic>&#x0232A;<sup>2</sup> &#x0003D; <italic>a</italic>(1 &#x02212; <italic>a</italic>). Averaging the slope &#x003D5;&#x02032; of the gain function over the distribution of the input variable results in the averaged slope
<graphic xlink:href="fncom-07-00131-i0005.tif"/></p>
<p>The two alternative methods of linearization of &#x003D5; are illustrated in Figure <xref ref-type="fig" rid="F5">5</xref>. In the given example, the linearization procedure taking into account the fluctuations of the input signal results in a smaller effective slope &#x02329;&#x003D5;&#x02032;&#x0232A; than taking the slope &#x003D5;&#x02032;(<italic>a</italic>) at the mean activity <italic>a</italic> near its maximum. Averaging the slope &#x02329;&#x003D5;&#x02032;&#x0232A; over this distribution fits simulation results better than &#x003D5;&#x02032;(<italic>a</italic>) calculated at the mean of <italic>a</italic>, as shown in Figure <xref ref-type="fig" rid="F6">6</xref>.</p>
<fig id="F5" position="float">
<label>Figure 5</label>
<caption><p><bold>Alternative linearizations of the binary neuron model</bold>. The black curve represents the non-linear gain function <inline-formula><mml:math id="M44"><mml:mrow><mml:mi>&#x003D5;</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>x</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mn>2</mml:mn></mml:mfrac><mml:mo>+</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mn>2</mml:mn></mml:mfrac></mml:mrow></mml:math></inline-formula> tanh(&#x003B2;<italic>x</italic>). The dashed gray line is its tangent at the mean input value (denoted by the diagonal cross). The solid curve is the slope &#x02329;&#x003D5;&#x02032;&#x0232A; averaged over the distribution of the fluctuating input (27). This distribution estimated from direct simulation is presented by black dots, the corresponding theoretical prediction of a normal distribution <graphic xlink:href="fncom-07-00131-i0004.tif"/>(&#x003BC;,&#x003C3;) (27) is shown as the light gray curve.</p></caption>
<graphic xlink:href="fncom-07-00131-g0005.tif"/>
</fig>
<fig id="F6" position="float">
<label>Figure 6</label>
<caption><p><bold>Binary model neuron corresponds to OUP model with input noise</bold>. Autocovariance <bold>(A)</bold>, crosscovarince <bold>(B)</bold>, and autocovariance of population averaged activity <bold>(C,D)</bold>, for binary neurons (dots) and rate model with input noise (crosses). <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub>, <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> and <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> are shown in black, gray, and light gray. Corresponding theoretical predictions (18) in <bold>(C,D)</bold>, (34) in <bold>(A)</bold>, their difference in <bold>(C)</bold> are plotted as light gray curves throughout. Dashed curve in <bold>(C)</bold> represents the theoretical prediction using the linearization with the slope at the mean activity (26), the solid curve shows the results for the slope averaged over Gaussian distributed input fluctuations (27). The spread of the simulation results for binary neurons in <bold>(C)</bold> is due to different realizations of the random connectivity. <bold>(E,F)</bold> are the same as <bold>(A,B)</bold> but for the presence of a synaptic delay <italic>d</italic> &#x0003D; 10 ms instead of <italic>d</italic> &#x0003D; 0.1 ms.</p></caption>
<graphic xlink:href="fncom-07-00131-g0006.tif"/>
</fig>
<p>The finite slope of the non-linear gain function can be understood as resulting from the combination of a hard threshold with an intrinsic local source of noise. The inverse strength of this noise determines the slope parameter &#x003B2; (Ginzburg and Sompolinsky, <xref ref-type="bibr" rid="B23">1994</xref>). In this sense, the network model contains two sources of noise, the explicit local noise, quantified by &#x003B2; and the fluctuating synaptic input interpreted as self-generated noise on the network level, quantified by &#x003C3;. Even in the absence of local noise (&#x003B2; &#x02192; &#x0221E;), the above mentioned linearization is applicable and yields a finite effective slope &#x02329;&#x003D5;&#x02032;&#x0232A; (27). In the latter case the resulting effective synaptic weight is independent of the original synapse strength (Grytskyy et al., <xref ref-type="bibr" rid="B24">2013</xref>).</p>
<p>We now extend the classical treatment of covariances in binary networks (Ginzburg and Sompolinsky, <xref ref-type="bibr" rid="B23">1994</xref>) by synaptic conduction delays. In (25) <italic>F</italic><sub><italic>i</italic></sub>(<bold>n</bold>, <italic>t</italic>) must therefore be understood as a functional acting on the function <bold>n</bold>(<italic>t</italic>&#x02032;) for <italic>t</italic>&#x02032; &#x02208; [&#x02212;&#x0221E;, <italic>t</italic>], so that also synaptic connections with time delay <italic>d</italic> can be realized. We define an effective weight vector to absorb the gain factor as <bold>w</bold><sub><italic>i</italic></sub> &#x0003D; &#x003B2;<sub><italic>i</italic></sub><bold>J</bold><sub><italic>i</italic></sub>, with either &#x003B2;<sub><italic>i</italic></sub> &#x0003D; &#x003D5;&#x02032;(&#x003BC;) or &#x003B2;<sub><italic>i</italic></sub> &#x0003D; &#x02329;&#x003D5;&#x02032;&#x0232A; depending on the linearization procedure, and expand the right hand side of (24) to obtain
<disp-formula id="E30"><mml:math id="M45"><mml:mrow><mml:mo>&#x02329;</mml:mo><mml:msub><mml:mi>F</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>n</mml:mi></mml:mstyle><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>n</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>a</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:munderover><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:munderover><mml:mrow><mml:msub><mml:mi>w</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>k</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mstyle><mml:msub><mml:mi>c</mml:mi><mml:mrow><mml:mi>k</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo>,</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>Thus the cross-covariance fulfills the matrix delay differential equation
<disp-formula id="E31"><label>(28)</label><mml:math id="M46"><mml:mrow><mml:mi>&#x003C4;</mml:mi><mml:mfrac><mml:mi>d</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo>,</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>This differential equation is valid for <italic>t</italic> &#x0003E; <italic>s</italic>. For the stationary solution, the differential equation only depends on the relative timing <italic>u</italic> &#x0003D; <italic>t</italic> &#x02212; <italic>s</italic>
<disp-formula id="E32"><label>(29)</label><mml:math id="M47"><mml:mrow><mml:mi>&#x003C4;</mml:mi><mml:mfrac><mml:mi>d</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mi>u</mml:mi></mml:mrow></mml:mfrac><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>u</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>u</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>u</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>The same linearization applied to (23) results in the boundary condition for the solution of the previous equation
<disp-formula id="E33"><label>(30)</label><mml:math id="M48"><mml:mrow><mml:mn>2</mml:mn><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mi>T</mml:mi></mml:msup></mml:mrow></mml:math></disp-formula>
or, if we split <bold>c</bold> into its diagonal and its off-diagonal parts <bold>c</bold><sub><italic>a</italic></sub> and <bold>c</bold><sub>&#x02260;</sub>
<disp-formula id="E34"><label>(31)</label><mml:math id="M49"><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mn>2</mml:mn><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo>&#x02260;</mml:mo></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo>&#x02260;</mml:mo></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo>&#x02260;</mml:mo></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mi>T</mml:mi></mml:msup><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>O</mml:mi></mml:mstyle></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;with&#x000A0;</mml:mtext><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>O</mml:mi></mml:mstyle><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mi>a</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mi>a</mml:mi></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mi>T</mml:mi></mml:msup><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:math></disp-formula></p>
<p>In the following section we use this representation to demonstrate the equivalence of the covariance structure of binary networks to the solution for OUP with input noise.</p>
<sec>
<title>3.1. Equivalence of binary neurons and ornstein&#x02013;uhlenbeck processes</title>
<p>In the following subsection we show that the same Equations (29) and (31) for binary neurons also hold for the Ornstein-Uhlenbeck process (OUP) with input noise. In doing so here we also extend the existing framework of OUP (Risken, <xref ref-type="bibr" rid="B45">1996</xref>) to synaptic conduction delays <italic>d</italic>. A network of such processes is described by
<disp-formula id="E35"><label>(32)</label><mml:math id="M50"><mml:mrow><mml:mi>&#x003C4;</mml:mi><mml:mfrac><mml:mi>d</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
where <bold>x</bold> is a vector of pairwise uncorrelated white noise with &#x02329;<bold>x</bold>(<italic>t</italic>)&#x0232A;<sub><italic>x</italic></sub> &#x0003D; 0 and &#x02329;<italic>x</italic><sub><italic>i</italic></sub>(<italic>t</italic>)<italic>x</italic><sub><italic>j</italic></sub>(<italic>t</italic> &#x0002B; <italic>t</italic>&#x02032;)&#x0232A;<sub><italic>x</italic></sub> &#x0003D; &#x003B4;<sub><italic>ij</italic></sub>&#x003B4;(<italic>t</italic>&#x02032;)&#x003C1;<sup>2</sup>. With the help of the Green&#x00027;s function <italic>G</italic> satisfying <inline-formula><mml:math id="M51"><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mfrac><mml:mi>d</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:mo>+</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mi>G</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mi>&#x003B4;</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:math></inline-formula>, namely <inline-formula><mml:math id="M52"><mml:mrow><mml:mi>G</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>&#x003C4;</mml:mi></mml:mfrac><mml:mi>&#x003B8;</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>t</mml:mi><mml:mo>/</mml:mo><mml:mi>&#x003C4;</mml:mi></mml:mrow></mml:msup></mml:mrow></mml:math></inline-formula>, we obtain the solution of Equation (32) as
<disp-formula id="E36"><mml:math id="M53"><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mi>G</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle displaystyle='true'><mml:mrow><mml:msubsup><mml:mo>&#x0222B;</mml:mo><mml:mn>0</mml:mn><mml:mi>t</mml:mi></mml:msubsup><mml:mi>G</mml:mi></mml:mrow></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mi>d</mml:mi><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>The equation for the fluctuations &#x003B4;<bold>r</bold>(<italic>t</italic>) &#x0003D; <bold>r</bold>(<italic>t</italic>) &#x02212; &#x02329;<bold>r</bold>(<italic>t</italic>)&#x0232A;<sub><italic>x</italic></sub> around the expectation value
<disp-formula id="E37"><mml:math id="M54"><mml:mrow><mml:mi>&#x003B4;</mml:mi><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:mrow><mml:msubsup><mml:mo>&#x0222B;</mml:mo><mml:mn>0</mml:mn><mml:mi>t</mml:mi></mml:msubsup><mml:mi>G</mml:mi></mml:mrow></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>&#x003B4;</mml:mi><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mi>d</mml:mi><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup></mml:mrow></mml:math></disp-formula>
coincides with the noisy rate model with input noise (4) with delay <italic>d</italic> and convolution kernel <italic>h</italic> &#x0003D; <italic>G</italic>. In the next step we investigate the covariance matrix <italic>c</italic><sub><italic>ij</italic></sub>(<italic>t, s</italic>) &#x0003D; &#x02329;&#x003B4;<italic>r</italic><sub><italic>i</italic></sub>(<italic>t</italic> &#x0002B; <italic>s</italic>)&#x003B4;<italic>r</italic><sub><italic>j</italic></sub>(<italic>t</italic>)&#x0232A;<sub><italic>x</italic></sub> to show for which choice of parameters the covariance matrices for the binary model and the OUP with input noise coincide. To this end we derive the differential equation with respect to the time lag <italic>s</italic> for positive lags <italic>s</italic> &#x0003E; 0
<disp-formula id="E38"><label>(33)</label><mml:math id="M55"><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mi>&#x003C4;</mml:mi><mml:mfrac><mml:mi>d</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mi>s</mml:mi></mml:mrow></mml:mfrac><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:msub><mml:mrow><mml:mo>&#x02329;</mml:mo> <mml:mrow><mml:mi>&#x003C4;</mml:mi><mml:mfrac><mml:mi>d</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mi>s</mml:mi></mml:mrow></mml:mfrac><mml:mi>&#x003B4;</mml:mi><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mi>&#x003B4;</mml:mi><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow> <mml:mo>&#x0232A;</mml:mo></mml:mrow><mml:mi>x</mml:mi></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msub><mml:mrow><mml:mo>&#x02329;</mml:mo><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>&#x003B4;</mml:mi><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B4;</mml:mi><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mi>&#x003B4;</mml:mi><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mo>&#x0232A;</mml:mo></mml:mrow><mml:mi>x</mml:mi></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mi>s</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:math></disp-formula>
where we used &#x02329;<bold>x</bold>(<italic>t</italic> &#x0002B; <italic>s</italic>))&#x003B4;<bold>r</bold>(<italic>t</italic>)&#x0232A;<sub><italic>x</italic></sub> &#x0003D; 0, because the noise is realized independently for each time step and the system is causal. Equation (33) is identical to the differential equation satisfied by the covariance matrix (28) for binary neurons (Ginzburg and Sompolinsky, <xref ref-type="bibr" rid="B23">1994</xref>). To determine the initial condition of (33) we need to take the limit <bold>c</bold>(<italic>t</italic>, 0) &#x0003D; lim<sub><italic>s</italic> &#x02192; &#x0002B;0</sub><bold>c</bold>(<italic>t, s</italic>). This initial condition can be obtained as the stationary solution of the following differential equation
<disp-formula id="E39"><mml:math id="M56"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mi>&#x003C4;</mml:mi><mml:mfrac><mml:mi>d</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:munder><mml:mrow><mml:mi>lim</mml:mi></mml:mrow><mml:mrow><mml:mi>s</mml:mi><mml:mo>&#x02192;</mml:mo><mml:mo>+</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mrow><mml:mo>&#x02329;</mml:mo> <mml:mrow><mml:mi>&#x003C4;</mml:mi><mml:mfrac><mml:mi>d</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:mi>&#x003B4;</mml:mi><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mi>&#x003B4;</mml:mi><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow> <mml:mo>&#x0232A;</mml:mo></mml:mrow></mml:mrow><mml:mi>x</mml:mi></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mrow><mml:mrow><mml:mo>&#x02329;</mml:mo> <mml:mrow><mml:mi>&#x003B4;</mml:mi><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mfrac><mml:mi>d</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:mi>&#x003B4;</mml:mi><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow> <mml:mo>&#x0232A;</mml:mo></mml:mrow></mml:mrow><mml:mi>x</mml:mi></mml:msub></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:munder><mml:mrow><mml:mi>lim</mml:mi></mml:mrow><mml:mrow><mml:mi>s</mml:mi><mml:mo>&#x02192;</mml:mo><mml:mo>+</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder><mml:mrow><mml:mo>(</mml:mo> <mml:mrow><mml:msub><mml:mrow><mml:mo>&#x02329;</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>&#x003B4;</mml:mi><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B4;</mml:mi><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mi>&#x003B4;</mml:mi><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo></mml:mrow><mml:mi>x</mml:mi></mml:msub></mml:mrow></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mrow> <mml:mrow><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:msub><mml:mrow><mml:mo>&#x02329;</mml:mo><mml:mi>&#x003B4;</mml:mi><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003B4;</mml:mi><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B4;</mml:mi><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo></mml:mrow><mml:mi>x</mml:mi></mml:msub></mml:mrow> <mml:mo>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mn>2</mml:mn><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>,</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo>,</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>Here we used that &#x02329;<bold>x</bold>(<italic>t</italic> &#x0002B; <italic>s</italic>)&#x003B4;<bold>r</bold><sup><italic>T</italic></sup>(<italic>t</italic>)&#x0232A; vanishes due to independent noise realizations and causality and
<disp-formula id="E40"><mml:math id="M57"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mo>=</mml:mo><mml:munder><mml:mrow><mml:mi>lim</mml:mi></mml:mrow><mml:mrow><mml:mi>s</mml:mi><mml:mo>&#x02192;</mml:mo><mml:mo>+</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder><mml:msub><mml:mrow><mml:mo>&#x02329;</mml:mo><mml:mrow><mml:mi>&#x003B4;</mml:mi><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mo>&#x0232A;</mml:mo></mml:mrow><mml:mi>x</mml:mi></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:munder><mml:mrow><mml:mi>lim</mml:mi></mml:mrow><mml:mrow><mml:mi>s</mml:mi><mml:mo>&#x02192;</mml:mo><mml:mo>+</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mtext>&#x02009;</mml:mtext><mml:mi>s</mml:mi><mml:mo>&#x0003C;</mml:mo><mml:mi>d</mml:mi></mml:mrow></mml:munder><mml:mstyle displaystyle='true'><mml:mrow><mml:msubsup><mml:mo>&#x0222B;</mml:mo><mml:mn>0</mml:mn><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi></mml:mrow></mml:msubsup><mml:mi>G</mml:mi></mml:mrow></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:munder><mml:munder><mml:mrow><mml:msub><mml:mrow><mml:mo>&#x02329;</mml:mo><mml:mi>&#x003B4;</mml:mi><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo></mml:mrow><mml:mi>x</mml:mi></mml:msub></mml:mrow><mml:mo stretchy='true'>&#x0FE38;</mml:mo></mml:munder><mml:mrow><mml:mo>=</mml:mo><mml:mn>0</mml:mn><mml:mtext>&#x000A0;causality</mml:mtext></mml:mrow></mml:munder><mml:mo>+</mml:mo><mml:munder><mml:munder><mml:mrow><mml:msub><mml:mrow><mml:mo>&#x02329;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo></mml:mrow><mml:mi>x</mml:mi></mml:msub></mml:mrow><mml:mo stretchy='true'>&#x0FE38;</mml:mo></mml:munder><mml:mrow><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:mi>&#x003B4;</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow></mml:munder><mml:mo stretchy='false'>)</mml:mo><mml:mi>d</mml:mi><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:munder><mml:mrow><mml:mi>lim</mml:mi></mml:mrow><mml:mrow><mml:mi>s</mml:mi><mml:mo>&#x02192;</mml:mo><mml:mo>+</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mtext>&#x02009;</mml:mtext><mml:mi>s</mml:mi><mml:mo>&#x0003C;</mml:mo><mml:mi>d</mml:mi></mml:mrow></mml:munder><mml:mstyle displaystyle='true'><mml:mrow><mml:msubsup><mml:mo>&#x0222B;</mml:mo><mml:mn>0</mml:mn><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi></mml:mrow></mml:msubsup><mml:mi>G</mml:mi></mml:mrow></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:mi>&#x003B4;</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mi>d</mml:mi><mml:msup><mml:mi>t</mml:mi><mml:mo>&#x02032;</mml:mo></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:munder><mml:mrow><mml:mi>lim</mml:mi></mml:mrow><mml:mrow><mml:mi>s</mml:mi><mml:mo>&#x02192;</mml:mo><mml:mo>+</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mtext>&#x02009;</mml:mtext><mml:mi>s</mml:mi><mml:mo>&#x0003C;</mml:mo><mml:mi>d</mml:mi></mml:mrow></mml:munder><mml:mi>G</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>s</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>&#x003C4;</mml:mi></mml:mfrac><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:msup><mml:mi>&#x003C1;</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>In the stationary state, <bold>c</bold> only depends on the time lag <italic>s</italic> and is independent of the first time argument <italic>t</italic>, which, with the symmetry <bold>c</bold>(&#x02212;<italic>d</italic>)<sup><italic>T</italic></sup> &#x0003D; <bold>c</bold>(<italic>d</italic>) yields the additional condition for the solution of (33)
<disp-formula id="E41"><mml:math id="M58"><mml:mrow><mml:mn>2</mml:mn><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mi>T</mml:mi></mml:msup><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle></mml:mrow></mml:math></disp-formula>
or, if <bold>c</bold> is split in diagonal and off-diagonal parts <bold>c</bold><sub><italic>a</italic></sub> and <bold>c</bold><sub>&#x02260;</sub>, respectively,
<disp-formula id="E42"><mml:math id="M59"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mn>2</mml:mn><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo>&#x02260;</mml:mo></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo>&#x02260;</mml:mo></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo>&#x02260;</mml:mo></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mi>T</mml:mi></mml:msup><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>O</mml:mi></mml:mstyle></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>2</mml:mn><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mi>a</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo>&#x02260;</mml:mo></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo>&#x02260;</mml:mo></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mi>T</mml:mi></mml:msup><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
with <bold>O</bold> &#x0003D; <bold>w</bold><bold>c</bold><sub><italic>a</italic></sub>(&#x02212;<italic>d</italic>) &#x0002B; (<bold>w</bold><bold>c</bold><sub><italic>a</italic></sub>(&#x02212;<italic>d</italic>))<sup><italic>T</italic></sup>. In the equation for the autocovariance <bold>c</bold><sub><italic>a</italic></sub> the first two terms are contributions due to the cross covariance. In the state of asynchronous network activity with <italic>c</italic><sub><italic>ij</italic></sub> &#x0007E; <italic>N</italic><sup>&#x02212;1</sup> for <italic>i</italic>&#x02260; <italic>j</italic> these terms are typically negligible in comparison to the third term because &#x02211;<sub><italic>k</italic></sub> <italic>w</italic><sub><italic>ik</italic></sub><italic>c</italic><sub><italic>ki</italic></sub> &#x0007E; <italic>wKN</italic><sup>&#x02212;1</sup> &#x0003D; <italic>pw</italic>, which is typically smaller than 1 for small effective weights <italic>w</italic> &#x0003C; 1 and small connection probabilities <italic>p</italic> &#x000AB; 1. In this approximation with (33) the temporal shape of the autocovariance function is exponentially decaying with time constant &#x003C4;. With <bold>c</bold><sub><italic>a</italic></sub>(0) &#x02248; <bold>D</bold>/2 the approximate solution for the autocovariance is
<disp-formula id="E43"><label>(34)</label><mml:math id="M60"><mml:mrow><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mi>a</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mn>2</mml:mn></mml:mfrac><mml:mi>exp</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mfrac><mml:mrow><mml:mo>&#x0007C;</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x0007C;</mml:mo></mml:mrow><mml:mi>&#x003C4;</mml:mi></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mtext>&#x0200B;</mml:mtext><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>The cross covariance then satisfies the initial condition
<disp-formula id="E44"><mml:math id="M61"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mn>2</mml:mn><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo>&#x02260;</mml:mo></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo>&#x02260;</mml:mo></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo>&#x02260;</mml:mo></mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mi>T</mml:mi></mml:msup><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>O</mml:mi></mml:mstyle></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>O</mml:mi></mml:mstyle><mml:mo>=</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>D</mml:mi></mml:mstyle><mml:mo>/</mml:mo><mml:mn>2</mml:mn><mml:mo>+</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>D</mml:mi></mml:mstyle><mml:mo>/</mml:mo><mml:mn>2</mml:mn></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mi>T</mml:mi></mml:msup><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
which coincides with (31) for binary neurons if the diagonal matrix containing the zero time autocorrelations <bold>c</bold><sub><italic>a</italic></sub>(0) for binary neurons is equal to <bold>D</bold>/2, i.e., if the amplitude of the input noise &#x003C1;<sup>2</sup> &#x0003D; 2&#x003C4; <italic>a</italic>(1 &#x02212; <italic>a</italic>) and the effective linear coupling satisfies <bold>w</bold><sub><italic>i</italic></sub> &#x0003D; &#x003B2;<sub><italic>i</italic></sub><bold>J</bold><sub><italic>i</italic></sub>. Figure <xref ref-type="fig" rid="F6">6</xref> shows simulation results for population averaged covariance functions in binary networks and in networks of OUPs with input noise where the parameters of the OUP network are chosen according to the requirements derived above. The theoretical results (18) agree well with the direct simulations of both systems. For comparison, both methods of linearization, as explained above, are shown. The linearization procedure which takes into account the noise on the input side of the non-linear gain function results in a more accurate prediction. Moreover, the results derived here extend the classical theory (Ginzburg and Sompolinsky, <xref ref-type="bibr" rid="B23">1994</xref>) by considering synaptic conduction delays. Figure <xref ref-type="fig" rid="F8">8</xref> shows the decomposition of the covariance structure for a non-zero delay <italic>d</italic> &#x0003D; 3 ms. For details of the implementation see &#x0201C;Implementation of binary neurons in a spiking simulator code&#x0201D;. The explicit effect of introducing delays into the system, such as the appearance of oscillations in the time dependent covariance, is presented in <bold>(E,F)</bold> of Figure <xref ref-type="fig" rid="F6">6</xref>, differing from <bold>(A,B)</bold> of this figure, respectively, only in the delay (<italic>d</italic> &#x0003D; 10 ms for <bold>(E,F)</bold>, <italic>d</italic> &#x0003D; 0.1 ms for <bold>(A,B)</bold>).</p>
</sec>
</sec>
<sec>
<title>4. Hawkes processes</title>
<p>In the following section we show that to linear order the covariance functions in networks of Hawkes processes (Hawkes, <xref ref-type="bibr" rid="B25">1971</xref>) are equivalent to those in the linear rate network with output noise. Hawkes processes generate spikes randomly with a time density given by <bold>r</bold>(<italic>t</italic>), where neuron <italic>i</italic> generates spikes at a rate <italic>r</italic><sub><italic>i</italic></sub>(<italic>t</italic>), realized independently within each infinitesimal time step. Arriving spike trains <bold>s</bold> influence <bold>r</bold> according to
<disp-formula id="E45"><label>(35)</label><mml:math id="M62"><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mi>&#x003BD;</mml:mi><mml:mo>+</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>h</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo>&#x02217;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>J</mml:mi><mml:mi>s</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
with the connectivity matrix <bold>J</bold> and the kernel function <italic>h</italic><sub><italic>d</italic></sub> including the delay. Here &#x003BD; is a constant base rate of spike emission assumed to be equal for each neuron. Here we employ the implementation of the Hawkes model in the NEST simulator (Gewaltig and Diesmann, <xref ref-type="bibr" rid="B20">2007</xref>). The implementation is described in &#x0201C;Implementation of Hawkes neurons in a spiking simulator code&#x0201D;.</p>
<p>Given neuron <italic>j</italic> spiked at time <italic>u</italic> &#x02264; <italic>t</italic>, the probability of a spike in the interval [<italic>t, t</italic>&#x0002B;&#x003B4;<italic>t</italic>) for neuron <italic>i</italic> is 1 if <italic>i</italic> &#x0003D; <italic>j</italic>, <italic>u</italic> &#x0003D; <italic>t</italic> (the neuron spikes synchronously with itself) and <italic>r</italic><sub><italic>i</italic></sub>(<italic>t</italic>)&#x003B4;<italic>t</italic> &#x0002B; <italic>o</italic>(&#x003B4;<italic>t</italic><sup>2</sup>) otherwise. Considering the system in the stationary state with the time averaged activity <overline><bold>r</bold></overline> &#x0003D; &#x02329;<bold>s</bold>(<italic>t</italic>)&#x0232A; we obtain a convolution equation for time lags &#x003C4; &#x02265; 0 for the covariance matrix with the entry <italic>c</italic><sub><italic>ij</italic></sub>(&#x003C4;) for the covariance between spike trains of neurons <italic>i</italic> and <italic>j</italic>
<disp-formula id="E46"><label>(36)</label><mml:math id="M63"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mo>&#x02329;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>s</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>s</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mo>&#x02329;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>s</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:mo>&#x02329;</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>s</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mo>&#x02329;</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003B4;</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>s</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mover accent='true'><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='true'>&#x000AF;</mml:mo></mml:mover><mml:msup><mml:mover accent='true'><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='true'>&#x000AF;</mml:mo></mml:mover><mml:mi>T</mml:mi></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mo>&#x02329;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>s</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mover accent='true'><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='true'>&#x000AF;</mml:mo></mml:mover><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:mo>+</mml:mo><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mrow><mml:mover accent='true'><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='true'>&#x000AF;</mml:mo></mml:mover></mml:mrow></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mo>&#x02329;</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003BD;</mml:mi><mml:mo>+</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>h</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo>&#x02217;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>J</mml:mi><mml:mi>s</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>s</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mover accent='true'><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='true'>&#x000AF;</mml:mo></mml:mover><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:mo>+</mml:mo><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mrow><mml:mover accent='true'><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='true'>&#x000AF;</mml:mo></mml:mover></mml:mrow></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msub><mml:mi>h</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo>&#x02217;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>J</mml:mi></mml:mstyle><mml:mo>&#x02329;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>s</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>s</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mover accent='true'><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='true'>&#x000AF;</mml:mo></mml:mover><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:mo>+</mml:mo><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mrow><mml:mover accent='true'><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='true'>&#x000AF;</mml:mo></mml:mover></mml:mrow></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>h</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo>&#x02217;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>J</mml:mi><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mrow><mml:mover accent='true'><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='true'>&#x000AF;</mml:mo></mml:mover></mml:mrow></mml:msub><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
with the diagonal matrix <bold>D</bold><sub><overline><bold>r</bold></overline></sub> &#x0003D; &#x003B4;(&#x003C4;)diag(<overline><bold>r</bold></overline>), which has been derived earlier (Hawkes, <xref ref-type="bibr" rid="B25">1971</xref>). If the rates of all neurons are equal, <overline><bold>r</bold></overline><sub><italic>i</italic></sub> &#x0003D; <overline><italic>r</italic></overline>, all entries in the diagonal matrix are the same, <bold>D</bold><sub><overline><italic>r</italic></overline></sub> &#x0003D; &#x003B4;(&#x003C4;)<bold>1</bold><overline><italic>r</italic></overline>. In the subsequent section we demonstrate that the same convolution Equation (36) holds for the linear rate with output noise.</p>
<sec>
<title>4.1. Convolution equation for linear noisy rate neurons</title>
<p>For the linear rate model with output noise we use Equation (3) for time lags &#x003C4; &#x0003E; 0 to obtain a convolution equation for the covariance matrix of the output signal vector <bold>y</bold> &#x0003D; <bold>r</bold> &#x0002B; <bold>x</bold> as
<disp-formula id="E47"><label>(37)</label><mml:math id="M64"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mo>&#x02329;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>y</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>y</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mover accent='true'><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='true'>&#x000AF;</mml:mo></mml:mover><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mo>&#x02329;</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>h</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo>&#x02217;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>y</mml:mi></mml:mstyle><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>y</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mover accent='true'><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='true'>&#x000AF;</mml:mo></mml:mover><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>h</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo>&#x02217;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mo>&#x02329;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mover accent='true'><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='true'>&#x000AF;</mml:mo></mml:mover><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo><mml:mo>+</mml:mo><mml:mo>&#x02329;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>x</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0232A;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>h</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo>&#x02217;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
where we utilized that due to causality the random noise signal generated at <italic>t</italic>&#x0002B;&#x003C4; has no influence on <bold>r</bold>(<italic>t</italic>), so the respective correlation vanishes. <bold>D</bold> is the covariance of the noise as in (11), <italic>D</italic><sub><italic>ij</italic></sub>(&#x003C4;) &#x0003D; &#x02329;<italic>x</italic><sub><italic>i</italic></sub>(<italic>t</italic>)<italic>x</italic><sub><italic>j</italic></sub>(<italic>t</italic> &#x0002B; &#x003C4;)&#x0232A; &#x0003D; &#x003B4;<sub><italic>ij</italic></sub>&#x003B4;(&#x003C4;)&#x003C1;<sup>2</sup>. If &#x003C1; is chosen such that &#x003C1;<sup>2</sup> coincides with the averaged activity <overline><italic>r</italic></overline> in a network of Hawkes neurons and the connection matrix <bold>w</bold> is identical to <bold>J</bold> of the Hawkes network, the Equations (36) and (37) are identical. Therefore the cross spectrum of both systems is given by (11).</p>
</sec>
<sec>
<title>4.2. Non-linear self-consistent rate in rectifying hawkes networks</title>
<p>The convolution Equation (36) for the covariance matrix of Hawkes neurons is exact if no element of <bold>r</bold> is negative, which is particularly the case for a network of only excitatory neurons. Especially in networks including inhibitory couplings, the intensity <italic>r</italic><sub><italic>i</italic></sub> of neuron <italic>i</italic> may assume negative values. A neuron with <italic>r</italic><sub><italic>i</italic></sub> &#x0003C; 0 does not emit spikes, so the instantaneous rate is given by &#x003BB;<sub><italic>i</italic></sub> &#x0003D; [<italic>r</italic><sub><italic>i</italic></sub>(<italic>t</italic>)]<sub>&#x0002B;</sub> &#x0003D; &#x003B8;(<italic>r</italic><sub><italic>i</italic></sub>(<italic>t</italic>)) <italic>r</italic><sub><italic>i</italic></sub>(<italic>t</italic>), with the Heaviside function &#x003B8;. We now take into account this effective nonlinearity &#x02013;the rectification of the Hawkes model neuron&#x02013; in a similar manner as we already used to linearize binary neurons. If the network is in the regime of low spike rates, the fluctuations in the input of each neuron due to the Poissonian arrival of spikes are large compared to the fluctuations due to the time varying intensities <bold>r</bold>(<italic>t</italic>). Considering the same homogeneous network structure as described in &#x0201C;Population-averaged covariances,&#x0201D; the input statistics is identical for each cell <italic>i</italic>, so the mean activity &#x003BB;<sub>0</sub> &#x0003D; &#x02329;&#x003BB;<sub><italic>i</italic></sub>&#x0232A; is the same for all neurons <italic>i</italic>. The superposition of the synaptic inputs to neuron <italic>i</italic> cause an instantaneous intensity <italic>r</italic><sub><italic>i</italic></sub> that follows approximately a Gaussian distribution <graphic xlink:href="fncom-07-00131-i0004.tif"/>(&#x003BC;,&#x003C3;,<italic>r</italic><sub><italic>i</italic></sub>) with mean &#x003BC; &#x0003D; &#x02329;<italic>r</italic>&#x0232A; &#x0003D; &#x003BD;&#x0002B;&#x003BB;<sub>0</sub><italic>KJ</italic>(1 &#x02212; <italic>g</italic>&#x003B3;) and standard deviation <inline-formula><mml:math id="M65"><mml:mrow><mml:mi>&#x003C3;</mml:mi><mml:mo>=</mml:mo><mml:msqrt><mml:mrow><mml:mo>&#x02329;</mml:mo><mml:msup><mml:mi>r</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>&#x0232A;</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mrow><mml:mo>&#x02329;</mml:mo><mml:mi>r</mml:mi><mml:mo>&#x0232A;</mml:mo></mml:mrow><mml:mn>2</mml:mn></mml:msup></mml:mrow></mml:msqrt><mml:mo>=</mml:mo><mml:mi>J</mml:mi><mml:msqrt><mml:mrow><mml:mfrac><mml:mrow><mml:msub><mml:mi>&#x003BB;</mml:mi><mml:mn>0</mml:mn></mml:msub></mml:mrow><mml:mrow><mml:mn>2</mml:mn><mml:mi>&#x003C4;</mml:mi></mml:mrow></mml:mfrac><mml:mi>K</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo>+</mml:mo><mml:msup><mml:mi>g</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mi>&#x003B3;</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msqrt></mml:mrow></mml:math></inline-formula>. These expressions hold for the exponential kernel (5) due to Campbell&#x00027;s theorem (Papoulis and Pillai, <xref ref-type="bibr" rid="B37">2002</xref>), because of the stochastic Poisson-like arrival of incoming spikes, where the standard deviation of the spike count is proportional to the square root of the intensity &#x003BB;<sub>0</sub>. The rate &#x003BB;<sub>0</sub> is accessible by explicit integration over the Gaussian probability density as
<graphic xlink:href="fncom-07-00131-i0006.tif"/></p>
<p>This equation needs to be solved self-consistently (numerically or graphically) to determine the rate in the network, as the right hand side depends on the rate &#x003BB;<sub>0</sub> itself through &#x003BC; and &#x003C3;. Rewritten as
<disp-formula id="E48"><label>(38)</label><mml:math id="M66"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:msub><mml:mi>&#x003BB;</mml:mi><mml:mn>0</mml:mn></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mi>&#x003C3;</mml:mi><mml:mrow><mml:msqrt><mml:mrow><mml:mn>2</mml:mn><mml:mi>&#x003C0;</mml:mi></mml:mrow></mml:msqrt></mml:mrow></mml:mfrac><mml:mi>exp</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mfrac><mml:mrow><mml:msup><mml:mi>&#x003BC;</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow><mml:mrow><mml:mn>2</mml:mn><mml:msup><mml:mi>&#x003C3;</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>+</mml:mo><mml:mi>&#x003BC;</mml:mi><mml:msub><mml:mi>P</mml:mi><mml:mrow><mml:mi>&#x003BC;</mml:mi><mml:mo>,</mml:mo><mml:mi>&#x003C3;</mml:mi></mml:mrow></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>r</mml:mi><mml:mo>&#x0003E;</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mi>P</mml:mi><mml:mrow><mml:mi>&#x003BC;</mml:mi><mml:mo>,</mml:mo><mml:mi>&#x003C3;</mml:mi></mml:mrow></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>r</mml:mi><mml:mo>&#x0003E;</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mn>2</mml:mn></mml:mfrac><mml:mo>&#x02212;</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mn>2</mml:mn></mml:mfrac><mml:mtext>erf</mml:mtext><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mfrac><mml:mi>&#x003BC;</mml:mi><mml:mrow><mml:msqrt><mml:mn>2</mml:mn></mml:msqrt><mml:mi>&#x003C3;</mml:mi></mml:mrow></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p><italic>P</italic><sub>&#x003BC;,&#x003C3;</sub>(<italic>r</italic> &#x0003E; 0) is the probability that the intensity of a neuron is above threshold and therefore contributes to the transmission of a small fluctuation in the input. A neuron for which <italic>r</italic> &#x0003C; 0 acts as if it was absent. Hence we can treat the network with rectifying neurons completely analogous to the case of linear Hawkes processes, but multiply the synaptic weight <italic>J</italic> or &#x02212;<italic>gJ</italic> of each neuron with <italic>P</italic><sub>&#x003BC;,&#x003C3;</sub>(<italic>r</italic> &#x0003E; 0), i.e., the linearized connectivity matrix is
<disp-formula id="E49"><label>(39)</label><mml:math id="M67"><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi></mml:mstyle><mml:mo>=</mml:mo><mml:msub><mml:mi>P</mml:mi><mml:mrow><mml:mi>&#x003BC;</mml:mi><mml:mo>,</mml:mo><mml:mi>&#x003C3;</mml:mi></mml:mrow></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>r</mml:mi><mml:mo>&#x0003E;</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>J</mml:mi></mml:mstyle><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>Figure <xref ref-type="fig" rid="F7">7</xref> shows the agreement of the covariance functions obtained from direct simulation of the network of Hawkes processes and the analytical solution (21) with average firing rate &#x003BB;<sub>0</sub> determined by (38), setting the effective strength of the noise &#x003C1;<sup>2 &#x0003D; &#x003BB;<sub>0</sub></sup>, and the linearized coupling as described above. The detailed procedure for choosing the parameters in the direct simulation is described together with the implementation of the Hawkes model in &#x0201C;Implementation of Hawkes neurons in a spiking simulator code&#x0201D;.</p>
<fig id="F7" position="float">
<label>Figure 7</label>
<caption><p><bold>Covariance structure in spiking networks corresponds to OUP with output noise. (A)</bold> Autocovariance obtained by direct simulation of the LIF (black), Hawkes (gray), and OUP (light gray) models for excitatory (dots) and inhibitory neurons (crosses). <bold>(B)</bold> Covariance <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> averaged over disjoint pairs of neurons for LIF (black dots), Hawkes (gray dots), and OUP with output noise (empty circles). <bold>(C)</bold> Covariance averaged over disjoint pairs of neurons of the same type. <bold>(D)</bold> Autocovariance of the population averaged activity. Averages in <bold>(C,D)</bold> over excitatory neurons as black dots, over inhibitory neurons as gray dots. Corresponding theoretical predictions (21) are plotted as light gray curves in all panels except <bold>(A)</bold>. Light gray diagonal crosses in <bold>(A,D)</bold> denote theoretical peak positions determined by the firing rate <overline><italic>r</italic></overline> as <overline><italic>r</italic></overline>&#x00394;<italic>t</italic> (where &#x00394;<italic>t</italic> &#x0003D; 0.1 ms is the time resolution of the histogram).</p></caption>
<graphic xlink:href="fncom-07-00131-g0007.tif"/>
</fig>
</sec>
</sec>
<sec>
<title>5. Leaky integrate-and-fire neurons</title>
<p>In this section we consider a network of LIF model neurons with exponentially decaying postsynaptic currents and show its equivalence to the network of OUP with output noise, valid in the asynchronous irregular regime. A spike sent by neuron <italic>j</italic> at time <italic>t</italic> arrives at the target neuron <italic>i</italic> after the synaptic delay <italic>d</italic>, elicits a synaptic current <italic>I</italic><sub><italic>i</italic></sub> that decays with time constant &#x003C4;<sub><italic>s</italic></sub> and causes a response in the membrane potential <italic>V</italic><sub><italic>i</italic></sub> proportional to the synaptic efficacy <italic>J</italic><sub><italic>ij</italic></sub>. With the time constant &#x003C4;<sub><italic>m</italic></sub> of the membrane potential, the coupled set of differential equations governing the subthreshold dynamics of a single neuron <italic>i</italic> is (Fourcaud and Brunel, <xref ref-type="bibr" rid="B16">2002</xref>)
<disp-formula id="E50"><label>(40)</label><mml:math id="M68"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:msub><mml:mi>&#x003C4;</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mfrac><mml:mrow><mml:mi>d</mml:mi><mml:msub><mml:mi>V</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:mi>d</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:mo>=</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>V</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>I</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;</mml:mtext><mml:msub><mml:mi>&#x003C4;</mml:mi><mml:mi>s</mml:mi></mml:msub><mml:mfrac><mml:mrow><mml:mi>d</mml:mi><mml:msub><mml:mi>I</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:mi>d</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:mfrac><mml:mo>=</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>I</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>&#x003C4;</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mstyle displaystyle='true'><mml:munderover><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn><mml:mo>,</mml:mo><mml:mi>j</mml:mi></mml:mrow><mml:mi>N</mml:mi></mml:munderover><mml:mrow><mml:msub><mml:mi>J</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mstyle><mml:msub><mml:mi>s</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
where the membrane resistance was absorbed into the definitions of <italic>J</italic><sub><italic>ij</italic></sub> and <italic>I</italic><sub><italic>i</italic></sub>. If <italic>V</italic><sub><italic>i</italic></sub> reaches the threshold <italic>V</italic><sub>&#x003B8;</sub> at time point <italic>t</italic><sup><italic>i</italic></sup><sub><italic>k</italic></sub> the neuron emits an action potential and the membrane potential is reset to <italic>V</italic><sub><italic>r</italic></sub>, where it is clamped for the refractory time &#x003C4;<sub><italic>r</italic></sub>. The spiking activity of neuron <italic>i</italic> is described by this sequence of action potentials, the spike train <italic>s</italic><sub><italic>i</italic></sub>(<italic>t</italic>) &#x0003D; &#x02211;<sub><italic>k</italic></sub>&#x003B4;(<italic>t</italic> &#x02212; <italic>t</italic><sup><italic>i</italic></sup><sub><italic>k</italic></sub>). The dynamics of a single neuron is deterministic, but in network states of asynchronous, irregular activity and in the presence of external Poisson inputs to the network, the summed input to each cell can well be approximated as white noise (Brunel, <xref ref-type="bibr" rid="B7">2000</xref>) with first moment &#x003BC;<sub><italic>i</italic></sub> &#x0003D; &#x003C4;<sub><italic>m</italic></sub>&#x02211;<sub><italic>j</italic></sub><italic>J</italic><sub><italic>ij</italic></sub><italic>r</italic><sub><italic>j</italic></sub> and second moment &#x003C3;<sup>2</sup><sub><italic>i</italic></sub> &#x0003D; &#x003C4;<sub><italic>m</italic></sub>&#x02211;<sub><italic>j</italic></sub><italic>J</italic><sup>2</sup><sub><italic>ij</italic></sub><italic>r</italic><sub><italic>j</italic></sub>, where <italic>r</italic><sub><italic>j</italic></sub> is the stationary firing rate of neuron <italic>j</italic>. The stationary firing rate of neuron <italic>i</italic> is then given by Fourcaud and Brunel (<xref ref-type="bibr" rid="B16">2002</xref>)
<disp-formula id="E51"><label>(41)</label><mml:math id="M69"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:msubsup><mml:mi>r</mml:mi><mml:mi>i</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:msub><mml:mi>&#x003C4;</mml:mi><mml:mi>r</mml:mi></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>&#x003C4;</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:msqrt><mml:mi>&#x003C0;</mml:mi></mml:msqrt><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mi>F</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>y</mml:mi><mml:mi>&#x003B8;</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>F</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>y</mml:mi><mml:mi>r</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>y</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:msup><mml:mi>y</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo>+</mml:mo><mml:mtext>erf</mml:mtext><mml:mo stretchy='false'>(</mml:mo><mml:mi>y</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mtext>&#x02003;</mml:mtext><mml:mi>F</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>y</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:msup><mml:mstyle mathsize='140%' displaystyle='true'><mml:mo>&#x0222B;</mml:mo></mml:mstyle><mml:mi>y</mml:mi></mml:msup><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>y</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mi>d</mml:mi><mml:mi>y</mml:mi></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>with&#x000A0;</mml:mtext><mml:msub><mml:mi>y</mml:mi><mml:mrow><mml:mi>&#x003B8;</mml:mi><mml:mo>,</mml:mo><mml:mi>r</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:msub><mml:mi>V</mml:mi><mml:mrow><mml:mi>&#x003B8;</mml:mi><mml:mo>,</mml:mo><mml:mi>r</mml:mi></mml:mrow></mml:msub><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>&#x003BC;</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>&#x003C3;</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:mfrac><mml:mo>+</mml:mo><mml:mfrac><mml:mi>&#x003B1;</mml:mi><mml:mn>2</mml:mn></mml:mfrac><mml:msqrt><mml:mrow><mml:mfrac><mml:mrow><mml:msub><mml:mi>&#x003C4;</mml:mi><mml:mi>s</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>&#x003C4;</mml:mi><mml:mi>m</mml:mi></mml:msub></mml:mrow></mml:mfrac></mml:mrow></mml:msqrt><mml:mtext>&#x02003;</mml:mtext><mml:mi>&#x003B1;</mml:mi><mml:mo>=</mml:mo><mml:msqrt><mml:mn>2</mml:mn></mml:msqrt><mml:mrow><mml:mo>|</mml:mo> <mml:mrow><mml:mi>&#x003B6;</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mfrac><mml:mn>1</mml:mn><mml:mn>2</mml:mn></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow> <mml:mo>|</mml:mo></mml:mrow><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
with Riemann&#x00027;s zeta function &#x003B6;. The response of the LIF neuron to the injection of an additional spike into afferent <italic>j</italic> determines the impulse response <italic>w</italic><sub><italic>ij</italic></sub><italic>h</italic>(<italic>t</italic>) of the system. The time integral <italic>w</italic><sub><italic>ij</italic></sub> &#x0003D; <italic>w</italic><sub><italic>ij</italic></sub> &#x0222B;<sup>&#x0221E;</sup><sub>0</sub><italic>h</italic>(<italic>t</italic>)<italic>dt</italic> is the DC-susceptibility, which can formally be written as the derivative of the stationary firing rate by the rate of the afferent <italic>r</italic><sub><italic>j</italic></sub>, which, evaluated by help of (41), yields (Helias et al., <xref ref-type="bibr" rid="B27">2013</xref>, Results and App. A)
<disp-formula id="E52"><label>(42)</label><mml:math id="M70"><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:msub><mml:mi>w</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mo>&#x02202;</mml:mo><mml:msub><mml:mi>r</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:mo>&#x02202;</mml:mo><mml:msub><mml:mi>r</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow></mml:mfrac><mml:mo>=</mml:mo><mml:mi>&#x003B1;</mml:mi><mml:msub><mml:mi>J</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>+</mml:mo><mml:mi>&#x003B2;</mml:mi><mml:msubsup><mml:mi>J</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mn>2</mml:mn></mml:msubsup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>with&#x000A0;</mml:mtext><mml:mi>&#x003B1;</mml:mi><mml:mo>=</mml:mo><mml:msqrt><mml:mi>&#x003C0;</mml:mi></mml:msqrt><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:msub><mml:mi>&#x003C4;</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:msub><mml:mi>r</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mn>2</mml:mn></mml:msup><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:msub><mml:mi>&#x003C3;</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:mfrac><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>y</mml:mi><mml:mi>&#x003B8;</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>y</mml:mi><mml:mi>r</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;and&#x000A0;</mml:mtext><mml:mi>&#x003B2;</mml:mi><mml:mo>=</mml:mo><mml:msqrt><mml:mi>&#x003C0;</mml:mi></mml:msqrt><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:msub><mml:mi>&#x003C4;</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:msub><mml:mi>r</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mn>2</mml:mn></mml:msup><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:mn>2</mml:mn><mml:msubsup><mml:mi>&#x003C3;</mml:mi><mml:mi>i</mml:mi><mml:mn>2</mml:mn></mml:msubsup></mml:mrow></mml:mfrac><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>y</mml:mi><mml:mi>&#x003B8;</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mfrac><mml:mrow><mml:msub><mml:mi>V</mml:mi><mml:mi>&#x003B8;</mml:mi></mml:msub><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>&#x003BC;</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>&#x003C3;</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:mfrac><mml:mo>&#x02212;</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>y</mml:mi><mml:mi>r</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mfrac><mml:mrow><mml:msub><mml:mi>V</mml:mi><mml:mi>r</mml:mi></mml:msub><mml:mo>&#x02212;</mml:mo><mml:msub><mml:mi>&#x003BC;</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>&#x003C3;</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:math></disp-formula></p>
<p>In the strongly fluctuation-driven regime, the temporal behavior of the kernel <italic>h</italic> is dominated by a single exponential decay, whose time constant can be determined empirically. In a homogeneous random network the firing rates of all neurons are identical <italic>r</italic><sub><italic>i</italic></sub> &#x0003D; <overline><italic>r</italic></overline> and follow from the numerical solution of the self-consistency Equation (41). Approximating the autocovariance function of a single spike train by a &#x003B4;-peak scaled by the rate <overline><italic>r</italic></overline>&#x003B4;(<italic>t</italic>), one obtains for the covariance function <bold>c</bold> between pairs of spike trains the same convolution Equation (36) as for Hawkes neurons (Helias et al., <xref ref-type="bibr" rid="B27">2013</xref>, cf. equation 5). As shown in &#x0201C;Convolution equation for linear noisy rate neurons&#x0201D; this convolution equation coincides with that of a linear rate model with output noise (37), where the diagonal elements of <bold>D</bold> are chosen to agree to the average spike rate &#x003C1;<sup>2</sup> &#x0003D; <overline><italic>r</italic></overline>. The good agreement of the analytical cross covariance functions (21) for the OUP with output noise and direct simulation results for LIF are shown in Figure <xref ref-type="fig" rid="F7">7</xref>.</p>
</sec>
<sec sec-type="discussion" id="s2">
<title>6. Discussion</title>
<p>In this work we describe the path to a unified theoretical view on pairwise correlations in recurrent networks. We consider binary neuron models, LIF models, and linear point process models. These models containing a non-linearity (spiking threshold in spiking models, non-linear sigmoidal gain function in binary neurons, strictly positive rates in Hawkes processes) are linearized, taking into account the distribution of the fluctuating input.</p>
<p>The work presents results for several neuron models: We derive analytical expressions for delay-coupled OUP with input and with output noise, we extend the analytical treatment for stochastic binary neurons to the presence of synaptic delays, present a method that takes into account network-generated noise to determine the effective gain function, extend the theory of Hawkes processes to the existence of delays and inhibition, and present in Equation (12) a condition for the onset of global oscillations caused by delayed feedback, generalized to feedback pathways through different eigenvalues of the connectivity.</p>
<p>Some results qualitatively extend the existing theory (delays, inhibition), others improve the accuracy of existing theories (linearization including fluctuations). More importantly, our approach enables us to demonstrate the equivalence of each of these models after linear approximation to a linear model with fluctuating continuous variables. The fact that linear perturbation theory leads to effective linear equations is of course not surprising, but the analytical procedure firstly enables a mapping between models that conserves quantitative results and secondly allows us to uncover common structures underlying the emergence of correlated activity in recurrent networks. For the commonly appearing exponentially decaying response kernel function, these rate models coincide with the OUP (OUP, Uhlenbeck and Ornstein, <xref ref-type="bibr" rid="B55">1930</xref>; Risken, <xref ref-type="bibr" rid="B45">1996</xref>). We find that the considered models form two groups, which, in linear approximation merely differ by a matrix valued factor scaling the noise and in the choice of variables interpreted as neural activity. The difference between these two groups corresponds to the location of the noise: spiking models&#x02014;LIF models and Hawkes models&#x02014;belong to the class with noise on the output side, added to the activity of each neuron. The non-spiking binary neuron model corresponds to an OUP where the noise is added on the input side of each neuron. The closed solution for the correlation structure of OUP holds for both classes.</p>
<p>We identify different contributions to correlations in recurrent networks: the solution for output noise is split into three terms corresponding to the &#x003B4;-peak in the autocovariance, the covariance caused by shared input, and the direct synaptic influence of stochastic fluctuations of one neuron on another&#x02013;the latter echo terms are equal to propagators acting with delays (Helias et al., <xref ref-type="bibr" rid="B27">2013</xref>). A similar splitting into echo and correlated input terms for the case of input noise is shown in Figure <xref ref-type="fig" rid="F8">8</xref>. For increasing network size <italic>N</italic> &#x02192; &#x0221E;, keeping the connection probability <italic>p</italic> fixed, so that <italic>K</italic> &#x0003D; <italic>pN</italic>, and with rescaled synaptic amplitudes <italic>J</italic> &#x0007E; 1/<inline-formula><mml:math id="M71"><mml:mrow><mml:msqrt><mml:mi>N</mml:mi></mml:msqrt></mml:mrow></mml:math></inline-formula> (van Vreeswijk and Sompolinsky, <xref ref-type="bibr" rid="B56">1996</xref>; Renart et al., <xref ref-type="bibr" rid="B44">2010</xref>) the echo terms vanish fastest. Formally this can be seen from (18): the multiplicative factor of the common covariance term &#x003C6;<sub>4</sub> does not change with <italic>N</italic> while the other coefficients decrease. So ultimately all four entries of the matrix <bold>c</bold> have the same time dependence determined by the common covariance term &#x003C6;<sub>4</sub>. In particular the covariance between excitation and inhibition <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> becomes symmetric in this limit. This finally provides a quantitative explanation of the observation made in (Renart et al., <xref ref-type="bibr" rid="B44">2010</xref>) that the time-lag between excitation and inhibition vanishes in the limit of infinitely large networks. For a different synaptic rescaling <italic>J</italic> &#x0007E; <italic>N</italic><sup>&#x02212;1</sup> while keeping &#x003C1;<sup>2</sup> constant by appropriate additional input to each neuron (see Helias et al., <xref ref-type="bibr" rid="B27">2013</xref> applied to the LIF model), all multiplicative factors decrease &#x0007E; <italic>N</italic><sup>&#x02212;1</sup> and so does the amplitude of all covariances. Hence the asymmetry of <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> does not vanish in this limit. The same results hold for the case of output noise where the term with &#x003C6;<sub>1</sub> describes the common input part of the covariance. In this case and for finite network size, <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub> coincides with <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub> and <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> with <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> for <italic>t</italic> &#x0003E; 0, having a discontinuous jump at the time of the synaptic delay <italic>t</italic> &#x0003D; <italic>d</italic>. For time lags smaller than the delay all four covariances coincide. This is due to causality, as the second neuron cannot feel the influence of a fluctuation that happened in the first neuron less than one synaptic delay before. The covariance functions for systems corresponding to an OUP with input noise contain neither discontinuities nor sharp peaks at <italic>t</italic> &#x0003D; <italic>d</italic>, but <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> and <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub> have maxima and minima near this location. This observation can be interpreted as a result of the stochastic nature of the binary model where changes in the input influence the state of the neuron only with a certain probability. So, the entries of <bold>c</bold> in this case take different values for |<italic>t</italic>| &#x0003C; <italic>d</italic> but show the tendency to approach each other with increasing |<italic>t</italic>| &#x000BB; <italic>d</italic>. This tendency increases with network size. Our analytical solutions (18) for input noise and (21) for output noise hence explain the model-class dependent differences in the shape of covariance functions.</p>
<fig id="F8" position="float">
<label>Figure 8</label>
<caption><p><bold>Different echo terms for spiking and non-spiking neurons</bold>. Binary non-spiking neurons shown in <bold>(A,C)</bold> and LIF in <bold>(B,D)</bold>. <bold>(A)</bold>,<bold>(B)</bold> Echo terms by direct influence of the neuron&#x00027;s output on the network in dependence of neuron types (in <bold>A,B</bold> <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub>,<italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub>, and <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> are plotted as black, gray dots and circles). <bold>(C,D)</bold>, Contributions to the covariance evoked by correlated and common input (black dots) measured with help of auxiliary model neurons which do not provide feedback to the network. Corresponding theoretical predictions (16) are plotted as light gray curves throughout.</p></caption>
<graphic xlink:href="fncom-07-00131-g0008.tif"/>
</fig>
<p>The two above mentioned synaptic scaling procedures are commonly termed &#x0201C;strong coupling&#x0201D; (<italic>J</italic> &#x0007E; 1/<inline-formula><mml:math id="M72"><mml:mrow><mml:msqrt><mml:mi>N</mml:mi></mml:msqrt></mml:mrow></mml:math></inline-formula>) and &#x0201C;weak coupling&#x0201D; (<italic>J</italic> &#x0007E; 1/<italic>N</italic>), respectively. The results shown in Figure <xref ref-type="fig" rid="F6">6</xref> were obtained for <italic>J</italic> &#x0003D; 2/<inline-formula><mml:math id="M73"><mml:mrow><mml:msqrt><mml:mi>N</mml:mi></mml:msqrt></mml:mrow></mml:math></inline-formula> and &#x003B2; &#x0003D; 0.5, so the number of synapses required to cause a notable effect on the gain function is 1/(&#x003B2; <italic>J</italic>) &#x0003D; <inline-formula><mml:math id="M74"><mml:mrow><mml:msqrt><mml:mi>N</mml:mi></mml:msqrt></mml:mrow></mml:math></inline-formula>, which is small compared to the number of incoming synapses <italic>pN</italic>. Hence the network is in the strong coupling regime. Also note that for infinite slope of the gain function, &#x003B2; &#x02192; &#x0221E;, the magnitude of the covariance becomes independent of the synaptic amplitude <italic>J</italic>, in agreement with the linear theory presented here. This finding can readily be understood by the linearization procedure, presented in the current work, that takes into account the network- generated fluctuations of the total input. The amplitude &#x003C3; of these fluctuations scales linearly in <italic>J</italic> and the effective susceptibility depends on <italic>J</italic>/&#x003C3; in the case &#x003B2; &#x02192; &#x0221E;, explaining the invariance (Grytskyy et al., <xref ref-type="bibr" rid="B24">2013</xref>). In the current manuscript we generalized this procedure to finite slopes &#x003B2; and to other models than the binary neuron model.</p>
<p>Our approach enables us to map results obtained for one neuron model to another, in particular we extend the theory of all considered models to capture synaptic conduction delays, and devise a simpler way to obtain solutions for systems considered earlier (Ginzburg and Sompolinsky, <xref ref-type="bibr" rid="B23">1994</xref>). Our derivation of covariances in spiking networks does not rely on the advanced Wiener-Hopf method (Hazewinkel, <xref ref-type="bibr" rid="B26">2002</xref>), as earlier derivations (Hawkes, <xref ref-type="bibr" rid="B25">1971</xref>; Helias et al., <xref ref-type="bibr" rid="B27">2013</xref>) do, but only employs elementary methods. Our results are applicable for general connectivity matrices, and for the purpose of comparison with simulations we explicitly derive population averaged results. The averages of the dynamics of the linear rate model equations are exact for random network architectures with fixed out-degree, and approximate for fixed in-degree. Still, for non-linear models the linearization for fixed in-degree networks are simpler, because the homogeneous input statistics results in an identical linear response kernel for all cells. Finally we show that the oscillatory properties of networks of integrate-and-fire models (Brunel, <xref ref-type="bibr" rid="B7">2000</xref>; Helias et al., <xref ref-type="bibr" rid="B27">2013</xref>) are model-invariant features of all of the studied dynamics, given inhibition acts with a synaptic delay. We relate the collective oscillations to the pole structure of the cross spectrum, which also determines the power spectra of population signals such as EEG, ECoG, and the LFP.</p>
<p>The presented results provide a further step to understand the shape and to unify the description of correlations in recurrent networks. We hope that our analytical results will be useful to constrain the inverse problem of determining the synaptic connectivity given the correlation structure of neurophysiological activity measurements. Moreover the explicit expressions for covariance functions in the time domain are a necessary prerequisite to understand the evolution of synaptic amplitudes in systems with spike-timing dependent plasticity and extend the existing methods (Burkitt et al., <xref ref-type="bibr" rid="B11">2007</xref>; Gilson et al., <xref ref-type="bibr" rid="B21">2009</xref>, <xref ref-type="bibr" rid="B22">2010</xref>) to networks including inhibitory neurons and synaptic conduction delays.</p>
<sec>
<title>Conflict of interest statement</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
</sec>
</body>
<back>
<ack>
<p>We gratefully appreciate ongoing technical support by our colleagues in the NEST Initiative, especially Moritz Deger for the implementation of the Hawkes model. Binary and spiking network simulations performed with NEST (<ext-link ext-link-type="uri" xlink:href="http://www.nest-initiative.org">www.nest-initiative.org</ext-link>). Partially supported by the Helmholtz Association: HASB and portfolio theme SMHB, the J&#x000FC;lich Aachen Research Alliance (JARA), the Next-Generation Supercomputer Project of MEXT, and EU Grant 269921 (BrainScaleS).</p>
</ack>
<ref-list>
<title>References</title>
<ref id="B1">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Abeles</surname> <given-names>M.</given-names></name></person-group> (<year>1991</year>). <source>Corticonics: Neural Circuits of the Cerebral Cortex</source>. <edition>1st Edn</edition>. <publisher-loc>Cambridge</publisher-loc>: <publisher-name>Cambridge University Press</publisher-name>. <pub-id pub-id-type="doi">10.1017/CBO9780511574566</pub-id></citation>
</ref>
<ref id="B2">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Ascher</surname> <given-names>D.</given-names></name> <name><surname>Dubois</surname> <given-names>P. F.</given-names></name> <name><surname>Hinsen</surname> <given-names>K.</given-names></name> <name><surname>Hugunin</surname> <given-names>J.</given-names></name> <name><surname>Oliphant</surname> <given-names>T.</given-names></name></person-group> (<year>2001</year>). <source>An Open Source Project: Numerical Python</source>. Technical Report UCRL-MA-128569, <publisher-loc>Livermore, CA</publisher-loc>: <publisher-name>Lawrence Livermore National Laboratory</publisher-name>.</citation>
</ref>
<ref id="B3">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bi</surname> <given-names>G.-Q.</given-names></name> <name><surname>Poo</surname> <given-names>M.-M.</given-names></name></person-group> (<year>1999</year>). <article-title>Distributed synaptic modification in neural networks induced by patterned stimulation</article-title>. <source>Nature</source> <volume>401</volume>, <fpage>792</fpage>&#x02013;<lpage>796</lpage>. <pub-id pub-id-type="doi">10.1038/44573</pub-id><pub-id pub-id-type="pmid">10548104</pub-id></citation>
</ref>
<ref id="B4">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bienenstock</surname> <given-names>E.</given-names></name></person-group> (<year>1995</year>). <article-title>A model of neocortex</article-title>. <source>Network</source> <volume>6</volume>, <fpage>179</fpage>&#x02013;<lpage>224</lpage>. <pub-id pub-id-type="doi">10.1088/0954-898X/6/2/004</pub-id><pub-id pub-id-type="pmid">17271176</pub-id></citation>
</ref>
<ref id="B5">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Braitenberg</surname> <given-names>V.</given-names></name> <name><surname>Sch&#x000FC;z</surname> <given-names>A.</given-names></name></person-group> (<year>1991</year>). <source>Anatomy of the Cortex: Statistics and Geometry</source>. <publisher-loc>Berlin; Heidelberg; New York</publisher-loc>: <publisher-name>Springer-Verlag</publisher-name>.</citation>
</ref>
<ref id="B6">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Bronstein</surname> <given-names>I. N.</given-names></name> <name><surname>Semendjajew</surname> <given-names>K. A.</given-names></name> <name><surname>Musiol</surname> <given-names>G.</given-names></name> <name><surname>M&#x000FC;hlig</surname> <given-names>H.</given-names></name></person-group> (<year>1999</year>). <source>Taschenbuch der Mathematik</source>. <edition>4th Edn</edition>. <publisher-loc>Frankfurt am Main</publisher-loc>: <publisher-name>Verlag Harri Deutsch</publisher-name>.</citation>
</ref>
<ref id="B7">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Brunel</surname> <given-names>N.</given-names></name></person-group> (<year>2000</year>). <article-title>Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons</article-title>. <source>J. Comput. Neurosci</source>. <volume>8</volume>, <fpage>183</fpage>&#x02013;<lpage>208</lpage>. <pub-id pub-id-type="doi">10.1023/A:1008925309027</pub-id><pub-id pub-id-type="pmid">10809012</pub-id></citation>
</ref>
<ref id="B8">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Brunel</surname> <given-names>N.</given-names></name> <name><surname>Hakim</surname> <given-names>V.</given-names></name></person-group> (<year>1999</year>). <article-title>Fast global oscillations in networks of integrate-and-fire neurons with low firing rates</article-title>. <source>Neural Comput</source>. <volume>11</volume>, <fpage>1621</fpage>&#x02013;<lpage>1671</lpage>. <pub-id pub-id-type="doi">10.1162/089976699300016179</pub-id><pub-id pub-id-type="pmid">10490941</pub-id></citation>
</ref>
<ref id="B9">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Buice</surname> <given-names>M. A.</given-names></name> <name><surname>Cowan</surname> <given-names>J. D.</given-names></name> <name><surname>Chow</surname> <given-names>C. C.</given-names></name></person-group> (<year>2009</year>). <article-title>Systematic fluctuation expansion for neural network activity equations</article-title>. <source>Neural Comput</source>. <volume>22</volume>, <fpage>377</fpage>&#x02013;<lpage>426</lpage>. <pub-id pub-id-type="doi">10.1162/neco.2009.02-09-960</pub-id><pub-id pub-id-type="pmid">19852585</pub-id></citation>
</ref>
<ref id="B10">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Burak</surname> <given-names>Y.</given-names></name> <name><surname>Lewallen</surname> <given-names>S.</given-names></name> <name><surname>Sompolinsky</surname> <given-names>H.</given-names></name></person-group> (<year>2009</year>). <article-title>Stimulus-dependent correlations in threshold-crossing spiking neurons</article-title>. <source>Neural Comput</source>. <volume>21</volume>, <fpage>2269</fpage>&#x02013;<lpage>2308</lpage>. <pub-id pub-id-type="doi">10.1162/neco.2009.07-08-830</pub-id><pub-id pub-id-type="pmid">19409055</pub-id></citation>
</ref>
<ref id="B11">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Burkitt</surname> <given-names>A. N.</given-names></name> <name><surname>Gilson</surname> <given-names>M.</given-names></name> <name><surname>van Hemmen</surname> <given-names>J.</given-names></name></person-group> (<year>2007</year>). <article-title>Spike-timing-dependent plasticity for neurons with recurrent connections</article-title>. <source>Biol. Cybern</source>. <volume>96</volume>, <fpage>533</fpage>&#x02013;<lpage>546</lpage>. <pub-id pub-id-type="doi">10.1007/s00422-007-0148-2</pub-id><pub-id pub-id-type="pmid">17415586</pub-id></citation>
</ref>
<ref id="B12">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Buzs&#x000E1;ki</surname> <given-names>G.</given-names></name> <name><surname>Wang</surname> <given-names>X. J.</given-names></name></person-group> (<year>2012</year>). <article-title>Mechanisms of gamma oscillations</article-title>. <source>Annu. Rev. Neurosci</source>. <volume>35</volume>, <fpage>203</fpage>&#x02013;<lpage>225</lpage>. <pub-id pub-id-type="doi">10.1146/annurev-neuro-062111-150444</pub-id><pub-id pub-id-type="pmid">22443509</pub-id></citation>
</ref>
<ref id="B13">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Cohen</surname> <given-names>M. R.</given-names></name> <name><surname>Kohn</surname> <given-names>A.</given-names></name></person-group> (<year>2011</year>). <article-title>Measuring and interpreting neuronal correlations</article-title>. <source>Nat. Rev. Neurosci</source>. <volume>14</volume>, <fpage>811</fpage>&#x02013;<lpage>819</lpage>. <pub-id pub-id-type="doi">10.1038/nn.2842</pub-id><pub-id pub-id-type="pmid">21709677</pub-id></citation>
</ref>
<ref id="B14">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Corless</surname> <given-names>R. M.</given-names></name> <name><surname>Gonnet</surname> <given-names>G. H.</given-names></name> <name><surname>Hare</surname> <given-names>D. E. G.</given-names></name> <name><surname>Jeffrey</surname> <given-names>D. J.</given-names></name> <name><surname>Knuth</surname> <given-names>D. E.</given-names></name></person-group> (<year>1996</year>). <article-title>On the lambert w function</article-title>. <source>Adv. Comput. Math</source>. <volume>5</volume>, <fpage>329</fpage>&#x02013;<lpage>359</lpage>. <pub-id pub-id-type="doi">10.1007/BF02124750</pub-id></citation>
</ref>
<ref id="B15">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Diesmann</surname> <given-names>M.</given-names></name> <name><surname>Gewaltig</surname> <given-names>M.-O.</given-names></name> <name><surname>Aertsen</surname> <given-names>A.</given-names></name></person-group> (<year>1999</year>). <article-title>Stable propagation of synchronous spiking in cortical neural networks</article-title>. <source>Nature</source> <volume>402</volume>, <fpage>529</fpage>&#x02013;<lpage>533</lpage>. <pub-id pub-id-type="doi">10.1038/990101</pub-id><pub-id pub-id-type="pmid">10591212</pub-id></citation>
</ref>
<ref id="B16">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fourcaud</surname> <given-names>N.</given-names></name> <name><surname>Brunel</surname> <given-names>N.</given-names></name></person-group> (<year>2002</year>). <article-title>Dynamics of the firing probability of noisy integrate-and-fire neurons</article-title>. <source>Neural Comput</source>. <volume>14</volume>, <fpage>2057</fpage>&#x02013;<lpage>2110</lpage>. <pub-id pub-id-type="doi">10.1162/089976602320264015</pub-id><pub-id pub-id-type="pmid">12184844</pub-id></citation>
</ref>
<ref id="B17">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Galassi</surname> <given-names>M.</given-names></name> <name><surname>Davies</surname> <given-names>J.</given-names></name> <name><surname>Theiler</surname> <given-names>J.</given-names></name> <name><surname>Gough</surname> <given-names>B.</given-names></name> <name><surname>Jungman</surname> <given-names>G.</given-names></name> <name><surname>Booth</surname> <given-names>M.</given-names></name> <name><surname>Rossi</surname> <given-names>F.</given-names></name></person-group> (<year>2006</year>). <source>GNU Scientific Library Reference Manual 2nd Edn</source>. <publisher-loc>Bristol</publisher-loc>: <publisher-name>Network Theory Limited</publisher-name>.</citation>
</ref>
<ref id="B18">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Gardiner</surname> <given-names>C. W.</given-names></name></person-group> (<year>2004</year>). <source>Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences</source>, <edition>3rd Edn</edition>. Springer Series in Synergetics. <publisher-loc>Berlin</publisher-loc>: <publisher-name>Springer</publisher-name>.</citation>
</ref>
<ref id="B19">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gerstein</surname> <given-names>G. L.</given-names></name> <name><surname>Perkel</surname> <given-names>D. H.</given-names></name></person-group> (<year>1969</year>). <article-title>Simultaneously recorded trains of action potentials: analysis and functional interpretation</article-title>. <source>Science</source> <volume>881</volume>, <fpage>828</fpage>&#x02013;<lpage>830</lpage>. <pub-id pub-id-type="doi">10.1126/science.164.3881.828</pub-id><pub-id pub-id-type="pmid">5767782</pub-id></citation>
</ref>
<ref id="B20">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gewaltig</surname> <given-names>M.-O.</given-names></name> <name><surname>Diesmann</surname> <given-names>M.</given-names></name></person-group> (<year>2007</year>). <article-title>NEST (NEural Simulation Tool)</article-title>. <source>Scholarpedia</source> <volume>2</volume>, <fpage>1430</fpage>. <pub-id pub-id-type="doi">10.4249/scholarpedia.1430</pub-id></citation>
</ref>
<ref id="B21">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gilson</surname> <given-names>M.</given-names></name> <name><surname>Burkitt</surname> <given-names>A. N.</given-names></name> <name><surname>Grayden</surname> <given-names>D. B.</given-names></name> <name><surname>Thomas</surname> <given-names>D. A.</given-names></name> <name><surname>van Hemmen</surname> <given-names>J. L.</given-names></name></person-group> (<year>2009</year>). <article-title>Emergence of network structure due to spike-timing-dependent plasticity in recurrent neuronal networks. I. Input selectivity - strengthening correlated input pathways</article-title>. <source>Biol. Cybern</source>. <volume>101</volume>, <fpage>81</fpage>&#x02013;<lpage>102</lpage>. <pub-id pub-id-type="doi">10.1007/s00422-009-0319-4</pub-id><pub-id pub-id-type="pmid">19536560</pub-id></citation>
</ref>
<ref id="B22">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gilson</surname> <given-names>M.</given-names></name> <name><surname>Burkitt</surname> <given-names>A. N.</given-names></name> <name><surname>van Hemmen</surname> <given-names>J. L.</given-names></name></person-group> (<year>2010</year>). <article-title>STDP in recurrent neuronal networks</article-title>. <source>Front. Comput. Neurosci</source>. <volume>4</volume>:<issue>23</issue>. <pub-id pub-id-type="doi">10.3389/fncom.2010.00023</pub-id><pub-id pub-id-type="pmid">20890448</pub-id></citation>
</ref>
<ref id="B23">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ginzburg</surname> <given-names>I.</given-names></name> <name><surname>Sompolinsky</surname> <given-names>H.</given-names></name></person-group> (<year>1994</year>). <article-title>Theory of correlations in stochastic neural networks</article-title>. <source>Phys. Rev. E</source> <volume>50</volume>, <fpage>3171</fpage>&#x02013;<lpage>3191</lpage>. <pub-id pub-id-type="doi">10.1103/PhysRevE.50.3171</pub-id><pub-id pub-id-type="pmid">9962363</pub-id></citation>
</ref>
<ref id="B24">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Grytskyy</surname> <given-names>D.</given-names></name> <name><surname>Tetzlaff</surname> <given-names>T.</given-names></name> <name><surname>Diesmann</surname> <given-names>M.</given-names></name> <name><surname>Helias</surname> <given-names>M.</given-names></name></person-group> (<year>2013</year>). <article-title>Invariance of covariances arises out of noise</article-title>. <source>AIP Conf. Proc</source>. <volume>1510</volume>, <fpage>258</fpage>&#x02013;<lpage>262</lpage>. <pub-id pub-id-type="doi">10.1063/1.4776531</pub-id></citation>
</ref>
<ref id="B25">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hawkes</surname> <given-names>A.</given-names></name></person-group> (<year>1971</year>). <article-title>Point spectra of some mutually exciting point process</article-title>. <source>J. R. Statist. Soc. Ser. B</source> <volume>33</volume>, <fpage>438</fpage>&#x02013;<lpage>443</lpage>.</citation>
</ref>
<ref id="B26">
<citation citation-type="book"><person-group person-group-type="editor"><name><surname>Hazewinkel</surname> <given-names>M.</given-names></name></person-group> (Ed.) (<year>2002</year>). <source>Encyclopaedia of Mathematics</source>. <publisher-loc>Dordrecht</publisher-loc>: <publisher-name>Kluwer Academic Publishers</publisher-name>.</citation>
</ref>
<ref id="B27">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Helias</surname> <given-names>M.</given-names></name> <name><surname>Tetzlaff</surname> <given-names>T.</given-names></name> <name><surname>Diesmann</surname> <given-names>M.</given-names></name></person-group> (<year>2013</year>). <article-title>Echoes in correlated neural systems</article-title>. <source>New J. Phys</source>. <volume>15</volume>:<fpage>023002</fpage>. <pub-id pub-id-type="doi">10.1088/1367-2630/15/2/023002</pub-id></citation>
</ref>
<ref id="B28">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hopfield</surname> <given-names>J. J.</given-names></name></person-group> (<year>1982</year>). <article-title>Neural networks and physical systems with emergent collective computational abilities</article-title>. <source>Proc. Natl. Acad. Sci. U.S.A</source>. <volume>79</volume>, <fpage>2554</fpage>&#x02013;<lpage>2558</lpage>. <pub-id pub-id-type="doi">10.1073/pnas.79.8.2554</pub-id><pub-id pub-id-type="pmid">6953413</pub-id></citation>
</ref>
<ref id="B29">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ito</surname> <given-names>J.</given-names></name> <name><surname>Maldonado</surname> <given-names>P.</given-names></name> <name><surname>Singer</surname> <given-names>W.</given-names></name> <name><surname>Gr&#x000FC;n</surname> <given-names>S.</given-names></name></person-group> (<year>2011</year>). <article-title>Saccade-related modulations of neuronal excitability support synchrony of visually elicited spikes</article-title>. <source>Cereb. Cortex</source> <volume>21</volume>, <fpage>2482</fpage>&#x02013;<lpage>2497</lpage>. <pub-id pub-id-type="doi">10.1093/cercor/bhr020</pub-id><pub-id pub-id-type="pmid">21459839</pub-id></citation>
</ref>
<ref id="B30">
<citation citation-type="web"><person-group person-group-type="author"><name><surname>Jones</surname> <given-names>E.</given-names></name> <name><surname>Oliphant</surname> <given-names>T.</given-names></name> <name><surname>Peterson</surname> <given-names>P.</given-names></name> <etal/></person-group>. (<year>2001</year>). <source>SciPy: Open Source Scientific Tools for Python</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="http://www.scipy.org/">http://www.scipy.org/</ext-link></citation>
</ref>
<ref id="B31">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kilavik</surname> <given-names>B. E.</given-names></name> <name><surname>Roux</surname> <given-names>S.</given-names></name> <name><surname>Ponce-Alvarez</surname> <given-names>A.</given-names></name> <name><surname>Confais</surname> <given-names>J.</given-names></name> <name><surname>Gruen</surname> <given-names>S.</given-names></name> <name><surname>Riehle</surname> <given-names>A.</given-names></name></person-group> (<year>2009</year>). <article-title>Long-term modifications in motor cortical dynamics induced by intensive practice</article-title>. <source>J. Neurosci</source>. <volume>29</volume>, <fpage>12653</fpage>&#x02013;<lpage>12663</lpage>. <pub-id pub-id-type="doi">10.1523/JNEUROSCI.1554-09.2009</pub-id><pub-id pub-id-type="pmid">19812340</pub-id></citation>
</ref>
<ref id="B32">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kriener</surname> <given-names>B.</given-names></name> <name><surname>Tetzlaff</surname> <given-names>T.</given-names></name> <name><surname>Aertsen</surname> <given-names>A.</given-names></name> <name><surname>Diesmann</surname> <given-names>M.</given-names></name> <name><surname>Rotter</surname> <given-names>S.</given-names></name></person-group> (<year>2008</year>). <article-title>Correlations and population dynamics in cortical networks</article-title>. <source>Neural Comput</source>. <volume>20</volume>, <fpage>2185</fpage>&#x02013;<lpage>2226</lpage>. <pub-id pub-id-type="doi">10.1162/neco.2008.02-07-474</pub-id><pub-id pub-id-type="pmid">18439141</pub-id></citation>
</ref>
<ref id="B33">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Markram</surname> <given-names>H.</given-names></name> <name><surname>L&#x000FC;bke</surname> <given-names>J.</given-names></name> <name><surname>Frotscher</surname> <given-names>M.</given-names></name> <name><surname>Sakmann</surname> <given-names>B.</given-names></name></person-group> (<year>1997</year>). <article-title>Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs</article-title>. <source>Science</source> <volume>275</volume>, <fpage>213</fpage>&#x02013;<lpage>215</lpage>. <pub-id pub-id-type="doi">10.1126/science.275.5297.213</pub-id><pub-id pub-id-type="pmid">8985014</pub-id></citation>
</ref>
<ref id="B34">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Moreno-Bote</surname> <given-names>R.</given-names></name> <name><surname>Parga</surname> <given-names>N.</given-names></name></person-group> (<year>2006</year>). <article-title>Auto- and crosscorrelograms for the spike response of leaky integrate-and-fire neurons with slow synapses</article-title>. <source>Phys. Rev. Lett</source>. <volume>96</volume>:<fpage>028101</fpage>. <pub-id pub-id-type="doi">10.1103/PhysRevLett.96.028101</pub-id><pub-id pub-id-type="pmid">16486646</pub-id></citation>
</ref>
<ref id="B35">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Morrison</surname> <given-names>A.</given-names></name> <name><surname>Diesmann</surname> <given-names>M.</given-names></name></person-group> (<year>2008</year>). <article-title>Maintaining causality in discrete time neuronal network simulations</article-title>, in <source>Lectures in Supercomputational Neuroscience: Dynamics in Complex Brain Networks</source>, eds <person-group person-group-type="editor"><name><surname>beim Graben</surname> <given-names>P.</given-names></name> <name><surname>Zhou</surname> <given-names>C.</given-names></name> <name><surname>Thiel</surname> <given-names>M.</given-names></name> <name><surname>Kurths</surname> <given-names>J.</given-names></name></person-group> (<publisher-name>Understanding Complex Systems, Springer</publisher-name>), <fpage>267</fpage>&#x02013;<lpage>278</lpage>.</citation>
</ref>
<ref id="B36">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Morrison</surname> <given-names>A.</given-names></name> <name><surname>Mehring</surname> <given-names>C.</given-names></name> <name><surname>Geisel</surname> <given-names>T.</given-names></name> <name><surname>Aertsen</surname> <given-names>A.</given-names></name> <name><surname>Diesmann</surname> <given-names>M.</given-names></name></person-group> (<year>2005</year>). <article-title>Advancing the boundaries of high connectivity network simulation with distributed computing</article-title>. <source>Neural Comput</source>. <volume>17</volume>, <fpage>1776</fpage>&#x02013;<lpage>1801</lpage>. <pub-id pub-id-type="doi">10.1162/0899766054026648</pub-id><pub-id pub-id-type="pmid">15969917</pub-id></citation>
</ref>
<ref id="B37">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Papoulis</surname> <given-names>A.</given-names></name> <name><surname>Pillai</surname> <given-names>S. U.</given-names></name></person-group> (<year>2002</year>). <source>Probability, Random Variables, and Stochastic Processes</source>, <edition>4th Edn</edition>. <publisher-loc>Boston, MA</publisher-loc>: <publisher-name>McGraw-Hill</publisher-name>.</citation>
</ref>
<ref id="B38">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Perkel</surname> <given-names>D. H.</given-names></name> <name><surname>Gerstein</surname> <given-names>G. L.</given-names></name> <name><surname>Moore</surname> <given-names>G. P.</given-names></name></person-group> (<year>1967</year>). <article-title>Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains</article-title>. <source>Biophys. J</source>. <volume>7</volume>, <fpage>419</fpage>&#x02013;<lpage>440</lpage>. <pub-id pub-id-type="doi">10.1016/S0006-3495(67)86597-4</pub-id><pub-id pub-id-type="pmid">4292792</pub-id></citation>
</ref>
<ref id="B39">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Pernice</surname> <given-names>V.</given-names></name> <name><surname>Staude</surname> <given-names>B.</given-names></name> <name><surname>Cardanobile</surname> <given-names>S.</given-names></name> <name><surname>Rotter</surname> <given-names>S.</given-names></name></person-group> (<year>2011</year>). <article-title>How structure determines correlations in neuronal networks</article-title>. <source>PLoS Comput. Biol</source>. <volume>7</volume>:<fpage>e1002059</fpage>. <pub-id pub-id-type="doi">10.1371/journal.pcbi.1002059</pub-id><pub-id pub-id-type="pmid">21625580</pub-id></citation>
</ref>
<ref id="B40">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Pernice</surname> <given-names>V.</given-names></name> <name><surname>Staude</surname> <given-names>B.</given-names></name> <name><surname>Cardanobile</surname> <given-names>S.</given-names></name> <name><surname>Rotter</surname> <given-names>S.</given-names></name></person-group> (<year>2012</year>). <article-title>Recurrent interactions in spiking networks with arbitrary topology</article-title>. <source>Phys. Rev. E</source> <volume>85</volume>:<fpage>031916</fpage>. <pub-id pub-id-type="doi">10.1103/PhysRevE.85.031916</pub-id><pub-id pub-id-type="pmid">22587132</pub-id></citation>
</ref>
<ref id="B41">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Plesser</surname> <given-names>H. E.</given-names></name> <name><surname>Eppler</surname> <given-names>J. M.</given-names></name> <name><surname>Morrison</surname> <given-names>A.</given-names></name> <name><surname>Diesmann</surname> <given-names>M.</given-names></name> <name><surname>Gewaltig</surname> <given-names>M.-O.</given-names></name></person-group> (<year>2007</year>). <article-title>Efficient parallel simulation of large-scale neuronal networks on clusters of multiprocessor computers</article-title>, in <source>Euro-Par 2007: Parallel Processing</source>, Vol. 4641 of <italic>Lecture Notes in Computer Science</italic>, eds <person-group person-group-type="editor"><name><surname>Kermarrec</surname> <given-names>A.-M.</given-names></name> <name><surname>Boug&#x000E9;</surname> <given-names>L.</given-names></name> <name><surname>Priol</surname> <given-names>T.</given-names></name></person-group> (<publisher-loc>Berlin</publisher-loc>: <publisher-name>Springer-Verlag</publisher-name>), <fpage>672</fpage>&#x02013;<lpage>681</lpage>.</citation>
</ref>
<ref id="B42">
<citation citation-type="web"><person-group person-group-type="author"><collab>Python Software Foundation.</collab></person-group> (<year>2008</year>). <source>The Python Programming Language</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="http://www.python.org">http://www.python.org</ext-link>.</citation>
</ref>
<ref id="B43">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rajan</surname> <given-names>K.</given-names></name> <name><surname>Abbott</surname> <given-names>L.</given-names></name></person-group> (<year>2006</year>). <article-title>Eigenvalue spectra of random matrices for neural networks</article-title>. <source>Phys. Rev. Lett</source>. <volume>97</volume>:<fpage>188104</fpage>. <pub-id pub-id-type="doi">10.1103/PhysRevLett.97.188104</pub-id><pub-id pub-id-type="pmid">17155583</pub-id></citation>
</ref>
<ref id="B44">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Renart</surname> <given-names>A.</given-names></name> <name><surname>De La Rocha</surname> <given-names>J.</given-names></name> <name><surname>Bartho</surname> <given-names>P.</given-names></name> <name><surname>Hollender</surname> <given-names>L.</given-names></name> <name><surname>Parga</surname> <given-names>N.</given-names></name> <name><surname>Reyes</surname> <given-names>A.</given-names></name> <name><surname>Harris</surname> <given-names>K. D.</given-names></name></person-group> (<year>2010</year>). <article-title>The asynchronous state in cortical cicuits</article-title>. <source>Science</source> <volume>327</volume>, <fpage>587</fpage>&#x02013;<lpage>590</lpage>. <pub-id pub-id-type="doi">10.1126/science.1179850</pub-id><pub-id pub-id-type="pmid">20110507</pub-id></citation>
</ref>
<ref id="B45">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Risken</surname> <given-names>H.</given-names></name></person-group> (<year>1996</year>). <source>The Fokker-Planck Equation</source>. <publisher-loc>Verlag; Berlin; Heidelberg</publisher-loc>: <publisher-name>Springer</publisher-name>.</citation>
</ref>
<ref id="B46">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rosenbaum</surname> <given-names>R.</given-names></name> <name><surname>Josic</surname> <given-names>K.</given-names></name></person-group> (<year>2011</year>). <article-title>Mechanisms that modulate the transfer of spiking correlations</article-title>. <source>Neural Comput</source>. <volume>23</volume>, <fpage>1261</fpage>&#x02013;<lpage>1305</lpage>. <pub-id pub-id-type="doi">10.1162/NECO_a_00116</pub-id><pub-id pub-id-type="pmid">21299426</pub-id></citation>
</ref>
<ref id="B47">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rotter</surname> <given-names>S.</given-names></name> <name><surname>Diesmann</surname> <given-names>M.</given-names></name></person-group> (<year>1999</year>). <article-title>Exact digital simulation of time-invariant linear systems with applications to neuronal modeling</article-title>. <source>Biol. Cybern</source>. <volume>81</volume>, <fpage>381</fpage>&#x02013;<lpage>402</lpage>. <pub-id pub-id-type="doi">10.1007/s004220050570</pub-id><pub-id pub-id-type="pmid">10592015</pub-id></citation>
</ref>
<ref id="B48">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Rumelhart</surname> <given-names>D. E.</given-names></name> <name><surname>McClelland</surname> <given-names>J. L.</given-names></name> <collab>the PDP Research Group</collab></person-group> (<year>1986</year>). <source>Parallel Distributed Processing, Explorations in the Microstructure of Cognition: Foundations</source>, <volume>Vol. 1</volume>. <publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>MIT Press</publisher-name>.</citation>
</ref>
<ref id="B49">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Shadlen</surname> <given-names>M. N.</given-names></name> <name><surname>Movshon</surname> <given-names>A. J.</given-names></name></person-group> (<year>1999</year>). <article-title>Synchrony unbound: A critical evaluation of the temporal binding hypothesis</article-title>. <source>Neuron</source> <volume>24</volume>, <fpage>67</fpage>&#x02013;<lpage>77</lpage>. <pub-id pub-id-type="doi">10.1016/S0896-6273(00)80822-3</pub-id><pub-id pub-id-type="pmid">10677027</pub-id></citation>
</ref>
<ref id="B50">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Singer</surname> <given-names>W.</given-names></name></person-group> (<year>1999</year>). <article-title>Neuronal synchrony: a versatile code for the definition of relations?</article-title> <source>Neuron</source> <volume>24</volume>, <fpage>49</fpage>&#x02013;<lpage>65</lpage>. <pub-id pub-id-type="doi">10.1016/S0896-6273(00)80821-1</pub-id><pub-id pub-id-type="pmid">10677026</pub-id></citation>
</ref>
<ref id="B51">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sompolinsky</surname> <given-names>H.</given-names></name> <name><surname>Yoon</surname> <given-names>H.</given-names></name> <name><surname>Kang</surname> <given-names>K.</given-names></name> <name><surname>Shamir</surname> <given-names>M.</given-names></name></person-group> (<year>2001</year>). <article-title>Population coding in neuronal systems with correlated noise</article-title>. <source>Phys. Rev. E</source> <volume>64</volume>:<fpage>51904</fpage>. <pub-id pub-id-type="doi">10.1103/PhysRevE.64.051904</pub-id><pub-id pub-id-type="pmid">11735965</pub-id></citation>
</ref>
<ref id="B52">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tchumatchenko</surname> <given-names>T.</given-names></name> <name><surname>Malyshev</surname> <given-names>A.</given-names></name> <name><surname>Geisel</surname> <given-names>T.</given-names></name> <name><surname>Volgushev</surname> <given-names>M.</given-names></name> <name><surname>Wolf</surname> <given-names>F.</given-names></name></person-group> (<year>2010</year>). <article-title>Correlations and synchrony in threshold neuron models</article-title>. <source>Phys. Rev. Lett</source>. <volume>104</volume>, <fpage>058102</fpage>. <pub-id pub-id-type="doi">10.1103/PhysRevLett.104.058102</pub-id><pub-id pub-id-type="pmid">20366796</pub-id></citation>
</ref>
<ref id="B53">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tetzlaff</surname> <given-names>T.</given-names></name> <name><surname>Helias</surname> <given-names>M.</given-names></name> <name><surname>Einevoll</surname> <given-names>G.</given-names></name> <name><surname>Diesmann</surname> <given-names>M.</given-names></name></person-group> (<year>2012</year>). <article-title>Decorrelation of neural-network activity by inhibitory feedback</article-title>. <source>PLoS Comput. Biol</source>. <volume>8</volume>:<fpage>e1002596</fpage>. <pub-id pub-id-type="doi">10.1371/journal.pcbi.1002596</pub-id><pub-id pub-id-type="pmid">23133368</pub-id></citation>
</ref>
<ref id="B54">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Trousdale</surname> <given-names>J.</given-names></name> <name><surname>Hu</surname> <given-names>Y.</given-names></name> <name><surname>Shea-Brown</surname> <given-names>E.</given-names></name> <name><surname>Josic</surname> <given-names>K.</given-names></name></person-group> (<year>2012</year>). <article-title>Impact of network structure and cellular response on spike time correlations</article-title>. <source>PLoS Comput. Biol</source>. <volume>8</volume>:<fpage>e1002408</fpage>. <pub-id pub-id-type="doi">10.1371/journal.pcbi.1002408</pub-id><pub-id pub-id-type="pmid">22457608</pub-id></citation>
</ref>
<ref id="B55">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Uhlenbeck</surname> <given-names>G. E.</given-names></name> <name><surname>Ornstein</surname> <given-names>L. S.</given-names></name></person-group> (<year>1930</year>). <article-title>On the theory of the brownian motion</article-title>. <source>Phys. Rev</source>. <volume>36</volume>, <fpage>823</fpage>&#x02013;<lpage>841</lpage>. reprinted in <comment><xref ref-type="bibr" rid="B59">Wax (1954)</xref></comment>. <pub-id pub-id-type="doi">10.1103/PhysRev.36.823</pub-id></citation>
</ref>
<ref id="B56">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>van Vreeswijk</surname> <given-names>C.</given-names></name> <name><surname>Sompolinsky</surname> <given-names>H.</given-names></name></person-group> (<year>1996</year>). <article-title>Chaos in neuronal networks with balanced excitatory and inhibitory activity</article-title>. <source>Science</source> <volume>274</volume>, <fpage>1724</fpage>&#x02013;<lpage>1726</lpage>. <pub-id pub-id-type="doi">10.1126/science.274.5293.1724</pub-id><pub-id pub-id-type="pmid">8939866</pub-id></citation>
</ref>
<ref id="B57">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>van Vreeswijk</surname> <given-names>C.</given-names></name> <name><surname>Sompolinsky</surname> <given-names>H.</given-names></name></person-group> (<year>1998</year>). <article-title>Chaotic balanced state in a model of cortical circuits</article-title>. <source>Neural Comput</source>. <volume>10</volume>, <fpage>1321</fpage>&#x02013;<lpage>1371</lpage>. <pub-id pub-id-type="doi">10.1162/089976698300017214</pub-id><pub-id pub-id-type="pmid">9698348</pub-id></citation>
</ref>
<ref id="B58">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>von der Malsburg</surname> <given-names>C.</given-names></name></person-group> (<year>1981</year>). <source>The Correlation Theory of Brain Function</source>. Internal report 81-2, Department of Neurobiology, Max-Planck-Institute for Biophysical Chemistry, G&#x000F6;ttingen, <publisher-loc>Germany</publisher-loc>.</citation>
</ref>
<ref id="B59">
<citation citation-type="book"><person-group person-group-type="editor"><name><surname>Wax</surname> <given-names>N.</given-names></name></person-group> (eds.). (<year>1954</year>). <source>Selected Papers on Noise and Stochastic Processes</source>. <publisher-loc>New York, NY</publisher-loc>: <publisher-name>Dover Publications</publisher-name>.</citation>
</ref>
<ref id="B60">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zohary</surname> <given-names>E.</given-names></name> <name><surname>Shadlen</surname> <given-names>M. N.</given-names></name> <name><surname>Newsome</surname> <given-names>W. T.</given-names></name></person-group> (<year>1994</year>). <article-title>Correlated neuronal discharge rate and its implications for psychophysical performance</article-title>. <source>Nature</source> <volume>370</volume>, <fpage>140</fpage>&#x02013;<lpage>143</lpage>. <pub-id pub-id-type="doi">10.1038/370140a0</pub-id><pub-id pub-id-type="pmid">8022482</pub-id></citation>
</ref>
</ref-list>
<app-group>
<app id="A1">
<title>Appendix</title>
<sec>
<title>Calculation of the population averaged cross covariance in time domain</title>
<p>We obtain the population averaged cross spectrum for the Ornstein-Uhlenbeck process with input noise by inserting the averaged connectivity matrix <bold>w</bold> &#x0003D; <bold>M</bold> (14) into (8). The two eigenvalues of <bold>M</bold> are 0 and <italic>L</italic> &#x0003D; <italic>Kw</italic>(1 &#x02212; &#x003B3;<italic>g</italic>). Taking these into account, we first rewrite the term
<disp-formula id="E53"><mml:math id="M75"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo>&#x02212;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mi>det</mml:mi><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo>&#x02212;</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi></mml:mstyle></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo>+</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mrow><mml:mi>K</mml:mi><mml:mi>w</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo>&#x02212;</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo>&#x02212;</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mrow><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo>&#x02212;</mml:mo><mml:mi>L</mml:mi></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:mrow><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:mo>+</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:mo>+</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
where we introduced <italic>f</italic>(&#x003C9;) &#x0003D; (<italic>H</italic><sub><italic>d</italic></sub>(&#x003C9;)<sup>&#x02212;1</sup> &#x02212; <italic>L</italic>)<sup>&#x02212;1</sup>. The corresponding transposed and conjugate complex term follows analogously. Hence we obtain the expression for the cross spectrum (17). The residue of <italic>f</italic>(&#x003C9;) at &#x003C9; &#x0003D; <italic>z</italic><sub><italic>k</italic></sub>(<italic>L</italic>) is
<disp-formula id="E54"><mml:math id="M76"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;Res</mml:mtext><mml:mo stretchy='false'>(</mml:mo><mml:mi>f</mml:mi><mml:mo>,</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo>=</mml:mo><mml:msub><mml:mi>z</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>L</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:munder><mml:mrow><mml:mi>lim</mml:mi></mml:mrow><mml:mrow><mml:msub><mml:mi>&#x003C9;</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>&#x02192;</mml:mo><mml:mi>&#x003C9;</mml:mi></mml:mrow></mml:munder><mml:mfrac><mml:mrow><mml:msub><mml:mi>&#x003C9;</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi></mml:mrow><mml:mrow><mml:msup><mml:mi>f</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>&#x003C9;</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:mfrac></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mover><mml:mo>=</mml:mo><mml:mrow><mml:msup><mml:mtext>l</mml:mtext><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mtext>Hopital</mml:mtext></mml:mrow></mml:mover><mml:munder><mml:mrow><mml:mi>lim</mml:mi></mml:mrow><mml:mrow><mml:msub><mml:mi>&#x003C9;</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>&#x02192;</mml:mo><mml:mi>&#x003C9;</mml:mi></mml:mrow></mml:munder><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:msup><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mi>f</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mo>&#x02032;</mml:mo></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>&#x003C9;</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:mfrac><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mi>d</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>d</mml:mi></mml:mrow></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo>+</mml:mo><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:mi>d</mml:mi><mml:mi>&#x003C9;</mml:mi></mml:mrow></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mi>d</mml:mi><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>d</mml:mi></mml:mrow></mml:msup><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo>+</mml:mo><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>&#x003C4;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>d</mml:mi></mml:mrow></mml:msup></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mi>d</mml:mi><mml:mi>L</mml:mi><mml:mo>+</mml:mo><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>d</mml:mi></mml:mrow></mml:msup></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
where in the last step we used the condition for a pole <italic>H</italic><sub><italic>d</italic></sub>(<italic>z</italic><sub><italic>k</italic></sub>)<sup>&#x02212;1</sup> &#x0003D; <italic>e</italic><sup><italic>iz</italic><sub><italic>k</italic></sub><italic>d</italic></sup>(1 &#x0002B; <italic>iz</italic><sub><italic>k</italic></sub>&#x003C4;) &#x0003D; <italic>L</italic> (see &#x0201C;Spectrum of the dynamics&#x0201D;). The residue of <italic>H</italic><sub><italic>d</italic></sub>(&#x003C9;) at <inline-formula><mml:math id="M77"><mml:mrow><mml:mi>z</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac></mml:mrow></mml:math></inline-formula> is <inline-formula><mml:math id="M78"><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mfrac><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>d</mml:mi><mml:mo>/</mml:mo><mml:mi>&#x003C4;</mml:mi></mml:mrow></mml:msup></mml:mrow></mml:math></inline-formula>. Using the residue theorem, we need to sum over all poles within the integration contour <inline-formula><mml:math id="M79"><mml:mrow><mml:mrow><mml:mo>{</mml:mo> <mml:mrow><mml:msub><mml:mi>z</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>L</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0007C;</mml:mo><mml:mi>k</mml:mi><mml:mo>&#x02208;</mml:mo><mml:mi>&#x02115;</mml:mi></mml:mrow> <mml:mo>}</mml:mo></mml:mrow><mml:mo>&#x0222A;</mml:mo><mml:mfrac><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac></mml:mrow></mml:math></inline-formula> to get the expression for <inline-formula><mml:math id="M80"><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>c</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>t</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:mn>2</mml:mn><mml:mi>&#x003C0;</mml:mi></mml:mrow></mml:mfrac><mml:mstyle displaystyle='true'><mml:mrow><mml:msubsup><mml:mo>&#x0222B;</mml:mo><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x0221E;</mml:mi></mml:mrow><mml:mrow><mml:mo>+</mml:mo><mml:mi>&#x0221E;</mml:mi></mml:mrow></mml:msubsup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>C</mml:mi></mml:mstyle></mml:mrow></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:msup><mml:mi>d</mml:mi><mml:mi>&#x003C9;</mml:mi><mml:mo>=</mml:mo><mml:mi>i</mml:mi><mml:msub><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>z</mml:mi><mml:mo>&#x02208;</mml:mo><mml:mrow><mml:mo>{</mml:mo> <mml:mrow><mml:msub><mml:mi>z</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>L</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x0007C;</mml:mo><mml:mi>k</mml:mi><mml:mo>&#x02208;</mml:mo><mml:mi>&#x02115;</mml:mi></mml:mrow> <mml:mo>}</mml:mo></mml:mrow><mml:mo>&#x0222A;</mml:mo><mml:mfrac><mml:mi>i</mml:mi><mml:mi>&#x003C4;</mml:mi></mml:mfrac></mml:mrow></mml:msub></mml:mrow></mml:math></inline-formula>Res(<bold>C</bold>(<italic>z</italic>), <italic>z</italic>)<italic>e</italic><sup><italic>izt</italic></sup> for <italic>t</italic> &#x02265; 0. Sorting (17) to obtain four matrix prefactors and remainders with different frequency dependence, &#x003A6;<sub>1</sub>(&#x003C9;) &#x0003D; <italic>f</italic>(&#x003C9;)<italic>f</italic>(&#x02212;&#x003C9;), &#x003A6;<sub>2</sub>(&#x003C9;) &#x0003D; <italic>f</italic>(&#x003C9;)<italic>f</italic>(&#x02212;&#x003C9;)<italic>H</italic><sub><italic>d</italic></sub>(&#x003C9;), &#x003A6;<sub>3</sub>(&#x003C9;) &#x0003D; &#x003A6;<sub>2</sub>(&#x02212;&#x003C9;), and &#x003A6;<sub>4</sub>(&#x003C9;) &#x0003D; <italic>f</italic>(&#x003C9;)<italic>f</italic>(&#x02212;&#x003C9;)<italic>H</italic><sub><italic>d</italic></sub>(&#x003C9;)<italic>H</italic><sub><italic>d</italic></sub>(&#x02212;&#x003C9;), we get (18). <bold>C</bold>(&#x003C9;) for output noise (20) is obtained by multiplying the expression for <bold>C</bold>(&#x003C9;) for input noise with <italic>H</italic><sup>&#x02212;1</sup><sub><italic>d</italic></sub>(&#x003C9;)<italic>H</italic><sup>&#x02212;1</sup><sub><italic>d</italic></sub>(&#x02212;&#x003C9;) &#x0003D; (1 &#x0002B; &#x003C9;<sup>2</sup>&#x003C4;<sup>2</sup>). In order to perform the back Fourier transformation one first needs to rewrite the cross spectrum in order to isolate the frequency independent term and the two terms that vanish for either <italic>t</italic> &#x0003C; <italic>d</italic> or <italic>t</italic> &#x0003E; <italic>d</italic>, as described in &#x0201C;Fourier back transformation,&#x0201D;
<disp-formula id="E55"><mml:math id="M81"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>C</mml:mi></mml:mstyle><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:mo>+</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi><mml:mi>D</mml:mi></mml:mstyle><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:mo>+</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:mo>+</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi><mml:mi>D</mml:mi></mml:mstyle></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mn>1</mml:mn></mml:mstyle><mml:mo>+</mml:mo><mml:mi>K</mml:mi><mml:mi>w</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:msub><mml:mi>H</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi><mml:mi>D</mml:mi></mml:mstyle><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi><mml:mi>D</mml:mi></mml:mstyle><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:msup><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi></mml:mstyle><mml:mi>T</mml:mi></mml:msup><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003C9;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>D</mml:mi></mml:mstyle><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
where in the last step we used <inline-formula><mml:math id="M82"><mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:mrow><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B3;</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>M</mml:mi></mml:mstyle><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:math></inline-formula>, because <bold>M</bold> is symmetric, obtaining (21). For each of the first three terms in the last expression the right integration contour needs to be chosen as described in &#x0201C;Fourier back transformation&#x0201D; on the example of the general expression (16).</p>
</sec>
<sec>
<title>Implementation of noisy rate models</title>
<p>The dynamics is propagated in time steps of duration &#x00394;<italic>t</italic> (note that in other works we use <italic>h</italic> as a symbol for the computation step size, which here is used as the symbol for the kernel). The product of the connectivity matrix with the vector of output variables at the end of the previous step <italic>i</italic> &#x02212; 1 is the vector <bold>I</bold>(<italic>t</italic><sub><italic>i</italic></sub>) of inputs at the current step <italic>i</italic>. The intrinsic time scale of the system is determined by the time constant &#x003C4;. For sufficiently small time steps &#x00394;<italic>t</italic> &#x000AB; &#x003C4; these inputs can be assumed to be time independent within one step. So we can use (3) or (4) and analytically convolve the kernel function <italic>h</italic> assuming the input to be constant over the time interval &#x00394;<italic>t</italic>. This corresponds to the method of exponential integration (Rotter and Diesmann, <xref ref-type="bibr" rid="B47">1999</xref>, see App. C.6) requiring only local knowledge of the connectivity matrix <bold>w</bold>. Note that this procedure becomes exact for &#x00394;<italic>t</italic> &#x02192; 0 and for finite &#x00394;<italic>t</italic> is an approximation. The propagation of the initial value <italic>r</italic><sub><italic>j</italic></sub>(<italic>t</italic><sub><italic>i</italic> &#x02212; 1</sub>) until the end of the time interval takes the form <italic>r</italic><sub><italic>j</italic></sub>(<italic>t</italic><sub><italic>i</italic> &#x02212; 1</sub>) <italic>e</italic><sup>&#x02212;&#x00394;<italic>t</italic>/&#x003C4;</sup> because <italic>h</italic>(<italic>t</italic><sub><italic>i</italic></sub>) &#x0003D; <italic>h</italic>(<italic>t</italic><sub><italic>i</italic> &#x02212; 1</sub>) <italic>e</italic><sup>&#x02212;&#x00394;<italic>t</italic>/&#x003C4;</sup>, so we obtain the expression <italic>r</italic><sub><italic>j</italic></sub>(<italic>t</italic><sub><italic>i</italic></sub>) at the end of the step as
<disp-formula id="E56"><label>(43)</label><mml:math id="M83"><mml:mrow><mml:msub><mml:mi>r</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>t</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x00394;</mml:mi><mml:mi>t</mml:mi><mml:mo>/</mml:mo><mml:mi>&#x003C4;</mml:mi></mml:mrow></mml:msup><mml:msub><mml:mi>r</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>t</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x00394;</mml:mi><mml:mi>t</mml:mi><mml:mo>/</mml:mo><mml:mi>&#x003C4;</mml:mi></mml:mrow></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:msub><mml:mi>I</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>t</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
where <italic>I</italic><sub><italic>j</italic></sub> denotes the input to the neuron <italic>j</italic>. For output noise the output variable of neuron <italic>j</italic> is <italic>y</italic><sub><italic>j</italic></sub> &#x0003D; <italic>r</italic><sub><italic>j</italic></sub> &#x0002B; <italic>x</italic><sub><italic>j</italic></sub>, with the locally generated additive noise <italic>x</italic><sub><italic>j</italic></sub> and hence the input is <italic>I</italic><sub><italic>j</italic></sub>(<italic>t</italic><sub><italic>i</italic></sub>) &#x0003D; (<bold>w y</bold>(<italic>t</italic><sub><italic>i</italic></sub>))<sub><italic>j</italic></sub>. In the case of input noise the output variable is <italic>r</italic><sub><italic>j</italic></sub> and the additional noise is added to the input variable, <italic>I</italic><sub><italic>j</italic></sub>(<italic>t</italic><sub><italic>i</italic></sub>) &#x0003D; (<bold>w r</bold>(<italic>t</italic><sub><italic>i</italic></sub>))<sub><italic>j</italic></sub> &#x0002B; <italic>x</italic><sub><italic>j</italic></sub>(<italic>t</italic><sub><italic>i</italic></sub>). In both cases <italic>x</italic><sub><italic>j</italic></sub> is implemented as a binary noise: in each time step, <italic>x</italic><sub><italic>j</italic></sub> is independently and randomly chosen to be 1 or &#x02212;1 with probability 0.5 multiplied with &#x003C1;/<inline-formula><mml:math id="M84"><mml:mrow><mml:msqrt><mml:mrow><mml:mi>&#x00394;</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:msqrt></mml:mrow></mml:math></inline-formula> to satisfy (2) for discretized time. Here the &#x003B4;-function is replaced by a &#x0201C;rectangle&#x0201D; function that is constant on the interval of length &#x00394;<italic>t</italic>, vanishes elsewhere, and has unit integral. The factor &#x00394;<italic>t</italic><sup>&#x02212;1</sup> in the expression for <italic>x</italic><sup>2</sup> ensures the integral to be unity. So far, the implementation assumes the synaptic delay to be zero. To implement a non-zero synaptic delay <italic>d</italic>, each object representing a neuron contains an array <italic>b</italic> of length <italic>l</italic><sub><italic>d</italic></sub> &#x0003D; <italic>d</italic>/&#x00394;<italic>t</italic> acting as a ring buffer. The input <italic>I</italic><sub><italic>j</italic></sub>(<italic>t</italic><sub><italic>i</italic></sub>) used to calculate the output rate at step <italic>i</italic> according to (43) is then taken from position <italic>i</italic> mod <italic>l</italic><sub><italic>d</italic></sub> of this array and after that replaced by the input presently received from the network, so that the new input will be used only after one delay has passed. This sequence of buffer handling can be represented as
<disp-formula id="E57"><mml:math id="M85"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:msub><mml:mi>I</mml:mi><mml:mi>j</mml:mi></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>t</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02190;</mml:mo><mml:mi>b</mml:mi><mml:mo stretchy='false'>[</mml:mo><mml:mi>i</mml:mi><mml:mtext>mod</mml:mtext><mml:msub><mml:mi>l</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>]</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mi>b</mml:mi><mml:mo stretchy='false'>[</mml:mo><mml:mi>i</mml:mi><mml:mtext>mod</mml:mtext><mml:msub><mml:mi>l</mml:mi><mml:mi>d</mml:mi></mml:msub><mml:mo stretchy='false'>]</mml:mo><mml:mo>&#x02190;</mml:mo><mml:mrow><mml:mo>{</mml:mo> <mml:mrow><mml:mtable columnalign='left'><mml:mtr columnalign='left'><mml:mtd columnalign='left'><mml:mrow><mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>r</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mi>j</mml:mi></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>x</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow></mml:mtd><mml:mtd columnalign='left'><mml:mrow><mml:mtext>for&#x000A0;input&#x000A0;noise</mml:mtext></mml:mrow></mml:mtd></mml:mtr><mml:mtr columnalign='left'><mml:mtd columnalign='left'><mml:mrow><mml:msub><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>w</mml:mi><mml:mi>y</mml:mi></mml:mstyle><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mi>j</mml:mi></mml:msub></mml:mrow></mml:mtd><mml:mtd columnalign='left'><mml:mrow><mml:mtext>for&#x000A0;output&#x000A0;noise</mml:mtext></mml:mrow></mml:mtd></mml:mtr></mml:mtable><mml:mo>.</mml:mo></mml:mrow></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>The model is implemented in Python version 2.7 (Python Software Foundation, <xref ref-type="bibr" rid="B42">2008</xref>) using numpy 1.6.1 (Ascher et al., <xref ref-type="bibr" rid="B2">2001</xref>) and scipy 0.9.0 (Jones et al., <xref ref-type="bibr" rid="B30">2001</xref>).</p>
</sec>
<sec>
<title>Implementation of binary neurons in a spiking simulator code</title>
<p>The binary neuron model is implemented in the NEST simulator, version 2.2.1 (Gewaltig and Diesmann, <xref ref-type="bibr" rid="B20">2007</xref>), which allows distributed simulation on parallel machines and handles synaptic delays in the established framework for spiking neurons (Morrison et al., <xref ref-type="bibr" rid="B36">2005</xref>). The name of the model is &#x0201C;<monospace>ginzburg_neuron</monospace>&#x0201D;. In NEST information is transmitted in form of point events, which in case of binary neurons are sent if the state of the neuron changes: one spike is sent for a down-transition and two spikes at the same time for an up-transition, so the multiplicity reflects the type of event. The logic to decode the original transitions is implemented in the function handle shown in Alg. <xref ref-type="table" rid="T2">2</xref>. If a single spike is received, the synaptic weight <italic>w</italic> is subtracted from the input buffer at the position determined by the time point of the transition and the synaptic delay. In distributed simulations a single spike with multiplicity 2 sent to another machine is handled on the receiving side as two separate events with multiplicity 1 each. In order to decode this case on the receiving machine we memorize the time (<italic>t</italic><sub>last</sub>) and origin (global id gid<sub>last</sub> of the sending neuron) of the last arrived spike. If both coincide to the spike under consideration, the sending neuron has performed an up transition 0 &#x02192; 1. We hence add twice the synaptic weight 2<italic>w</italic> to the input buffer of the target neuron, one that reflects the real change of the system state and another that compensates the subtraction of <italic>w</italic> after reception of the first spike of a pair. The algorithm relies on the fact that within NEST two spikes that are generated by one neuron at the same time point are delivered sequentially to the target neurons. This is assured, because neurons are updated one by one: The update propagates each neuron by a time step equal to the minimal delay <italic>d</italic><sub>min</sub> in the network. All spikes generated within one update step are written sequentially into the communication buffers, and finally the buffers are shipped to the other processors (Morrison et al., <xref ref-type="bibr" rid="B36">2005</xref>). Hence a pair of spikes generated by one neuron within a single update step will be delivered consecutively and will not be interspersed by spikes from other neurons with the same time stamp.</p>
<p>The model exhibits stochastic transitions (at random points in time) between two states. The transitions are governed by probabilities &#x003D5;(<italic>h</italic>). Using asynchronous update (Rumelhart et al., <xref ref-type="bibr" rid="B48">1986</xref>), in each infinitesimal interval [<italic>t, t</italic> &#x0002B; &#x003B4;<italic>t</italic>) each neuron in the network has the probability <inline-formula><mml:math id="M86"><mml:mrow><mml:mfrac><mml:mn>1</mml:mn><mml:mi>&#x003C4;</mml:mi></mml:mfrac><mml:mi>&#x003B4;</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:math></inline-formula> to be chosen for update (Hopfield, <xref ref-type="bibr" rid="B28">1982</xref>). A mathematically equivalent formulation draws the time points of update independently for all neurons. For a particular neuron, the sequence of update points has exponentially distributed intervals with mean duration &#x003C4;, i.e., it forms a Poisson process with rate &#x003C4;<sup>&#x02212;1</sup>. We employ the latter formulation to incorporate binary neuron models in the globally time-driven spiking simulator NEST (Gewaltig and Diesmann, <xref ref-type="bibr" rid="B20">2007</xref>) and constrain the points of transition to a discrete time grid &#x00394;<italic>t</italic> &#x0003D; 0.1 ms covering the interval <italic>d</italic><sub>min</sub> &#x02265; &#x00394;<italic>t</italic>. This neuron state update is implemented by the algorithm shown in Alg. <xref ref-type="table" rid="T1">1</xref>. Note that the field <italic>h</italic> is updated in steps of &#x00394;<italic>t</italic> while the activity state is updated only when the current time exceeds the next potential transition point. As the last step of the activity update we draw an exponentially distributed time interval to determine the new potential transition time. The potential transition time is represented with a higher resolution (on the order of microseconds) than &#x00394;<italic>t</italic> to avoid a systematic bias of the mean inter-update-interval. This update scheme is identical to the one used in (Hopfield, <xref ref-type="bibr" rid="B28">1982</xref>). Note that the implementation is different from the classical asynchronous update scheme (van Vreeswijk and Sompolinsky, <xref ref-type="bibr" rid="B57">1998</xref>), where in each discrete time step &#x00394;<italic>t</italic> exactly one neuron is picked at random. The mean inter-update-interval (time constant &#x003C4; in Alg. <xref ref-type="table" rid="T1">1</xref>) in the latter scheme is determined by &#x003C4; &#x0003D; &#x00394;<italic>tN</italic>, with <italic>N</italic> the number of neurons in the network. For small time steps both schemes converge so that update times follow a Poisson process.</p>
<p>At each update time point the neuron state becomes 1 with the probability given by the function &#x003D5; applied to the input at that time according to (25) and 0 with probability 1 &#x02212; &#x003D5;. The input is a function of the whole system state and is constant between spikes which indicate state changes. Each neuron therefore maintains a state variable <italic>h</italic> at each point in time holding the summed input and being updated by adding and subtracting the input read from the ring buffer <italic>b</italic> at the point readpos(t) corresponding to the current time (see Morrison et al., <xref ref-type="bibr" rid="B36">2005</xref>, for the implementation of the ring buffer, i.p. Fig 6). The ring buffer enables us to implement synaptic delays. For technical reasons this implementation requires a minimal delay of a single simulation time step (Morrison and Diesmann, <xref ref-type="bibr" rid="B35">2008</xref>). The gain function &#x003D5; applied to the input <italic>h</italic> has the form
<disp-formula id="E58"><label>(44)</label><mml:math id="M87"><mml:mrow><mml:mi>&#x003D5;</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>h</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:msub><mml:mi>c</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mi>h</mml:mi><mml:mo>+</mml:mo><mml:msub><mml:mi>c</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:mfrac><mml:mn>1</mml:mn><mml:mn>2</mml:mn></mml:mfrac><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo>+</mml:mo><mml:mi>tanh</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>c</mml:mi><mml:mn>3</mml:mn></mml:msub><mml:mo stretchy='false'>(</mml:mo><mml:mi>h</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mi>&#x003B8;</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
where throughout this manuscript we used <italic>c</italic><sub>1</sub> &#x0003D; 0, <italic>c</italic><sub>2</sub> &#x0003D; 1, and <italic>c</italic><sub>3</sub> &#x0003D; &#x003B2;, as defined in &#x0201C;Parameters of simulations&#x0201D;.</p>
<table-wrap position="float" id="T1">
<label>Algorithm 1</label>
<caption><p><bold>Update function of a binary neuron embedded in the spiking network simulator NEST</bold>.</p></caption>
<graphic xlink:href="fncom-07-00131-i0008.tif"/>
<table-wrap-foot>
<p><italic>The function</italic> readpos(t) <italic>returns a position in the ring buffer b corresponding to the current time point.</italic></p>
</table-wrap-foot>
</table-wrap>
<table-wrap position="float" id="T2">
<label>Algorithm 2</label>
<caption><p><bold>Input spike handler of a binary neuron embedded in the spiking network simulator NEST</bold>.</p></caption>
<graphic xlink:href="fncom-07-00131-i0009.tif"/>
<table-wrap-foot>
<p><italic>The simulation kernel calls the handle function for each spike event to be delivered to the neuron. A spike event is characterized by the time point of occurrence t</italic><sub>spike</sub><italic>, the synaptic delay d after which the event should reach the target, the global id</italic> gid <italic>identifying the sending neuron, and the multiplicity m &#x02265; 1, indicating the reception of multiple spike events. The function</italic> pos<italic>(t</italic><sub>spike</sub>, <italic>d, t) returns the position in the ring buffer b to which the spike is added so that it will be read at time t &#x0002B; d by the update function of the neuron, see Alg. <xref ref-type="table" rid="T1">1</xref>.</italic></p>
</table-wrap-foot>
</table-wrap>
</sec>
<sec>
<title>Implementation of hawkes neurons in a spiking simulator code</title>
<p>Hawkes neurons (Hawkes, <xref ref-type="bibr" rid="B25">1971</xref>) were introduced in the NEST simulator in version 2.2.0 (Gewaltig and Diesmann, <xref ref-type="bibr" rid="B20">2007</xref>). The name of the model is &#x0201C;<monospace>pp_psc_delta</monospace>&#x0201D;. In the following we describe the implemented neuron model in general and mention the particular choices of parameter and correspondences to the theory presented in &#x0201C;Hawkes processes&#x0201D;. The dynamics of the quasi-membrane potential <italic>u</italic> is integrated exactly within a time step &#x00394;<italic>t</italic> of the simulation (Rotter and Diesmann, <xref ref-type="bibr" rid="B47">1999</xref>), expressing the voltage <italic>u</italic>(<italic>t</italic><sub><italic>i</italic></sub>) at the end of time step <italic>i</italic> by the membrane potential at the end of the previous time step <italic>u</italic>(<italic>t</italic><sub><italic>i</italic> &#x02212; 1</sub>) as
<disp-formula id="E59"><label>(45)</label><mml:math id="M88"><mml:mrow><mml:mi>u</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>t</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x00394;</mml:mi><mml:mi>t</mml:mi><mml:mo>/</mml:mo><mml:mi>&#x003C4;</mml:mi></mml:mrow></mml:msup><mml:mi>u</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>t</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo>&#x02212;</mml:mo><mml:mi>&#x00394;</mml:mi><mml:mi>t</mml:mi><mml:mo>/</mml:mo><mml:mi>&#x003C4;</mml:mi></mml:mrow></mml:msup><mml:mo stretchy='false'>)</mml:mo><mml:msub><mml:mi>R</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:msub><mml:mi>I</mml:mi><mml:mi>e</mml:mi></mml:msub><mml:mo>+</mml:mo><mml:mi>b</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:msub><mml:mi>t</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
where <italic>I</italic><sub><italic>e</italic></sub> is a time-step wise constant input current (equal to 0 in all simulations presented in this article) and <italic>R</italic><sub><italic>m</italic></sub> &#x0003D; &#x003C4;<sub><italic>m</italic></sub>/<italic>C</italic><sub><italic>m</italic></sub> is the membrane resistance. The buffer <italic>b</italic>(<italic>t</italic><sub><italic>i</italic></sub>) contains the summed contributions of incoming spikes, multiplied by their respective synaptic weight, which have arrived at the neuron within the interval (<italic>t</italic><sub><italic>i</italic> &#x02212; 1</sub>,<italic>t</italic><sub><italic>i</italic></sub>]. <italic>b</italic> is implemented as a ring-buffer in order to handle the synaptic delay, logically similar as in &#x0201C;Implementation of noisy rate models,&#x0201D; described in detail in Morrison et al. (<xref ref-type="bibr" rid="B36">2005</xref>). The instantaneous spike emission rate is &#x003BB; &#x0003D; [<italic>c</italic><sub>1</sub><italic>u</italic> &#x0002B; <italic>c</italic><sub>2</sub><italic>e</italic><sup><italic>c</italic><sub>3</sub><italic>u</italic></sup>]<sub>&#x0002B;</sub>, where we use <italic>c</italic><sub>3</sub> &#x0003D; 0 in all simulations presented here. The quantities in the theory &#x0201C;Hawkes processes,&#x0201D; in particular in (35), are related to the parameters of the simulated model in the following way. The quantity <italic>r</italic> relates to the membrane potential <italic>u</italic> as <italic>r</italic> &#x0003D; <italic>c</italic><sub>1</sub><italic>u</italic> &#x0002B; <italic>c</italic><sub>2</sub> and the background rate &#x003BD; agrees to <italic>c</italic><sub>2</sub> &#x0003D; &#x003BD;. Hence the synaptic weight <italic>J</italic><sub><italic>ij</italic></sub> corresponds to the synaptic weight in the simulation multiplied by <italic>c</italic><sub>1</sub>. For the correspondence of the Hawkes model to the OUP with output noise of variance &#x003C1;<sup>2</sup> we use (38) to adjust the background rate &#x003BD; in order to obtain the desired rate &#x003BB;<sub>0</sub> &#x0003D; &#x003C1;<sup>2</sup> and we choose the synaptic weight <italic>J</italic> of the Hawkes model so that the linear coupling strength <italic>w</italic> of the OUP agrees to the effective linear weight given by (39). These two constraints can be fulfilled simultaneously by solving (38) and (39) by numerical iteration. The spike emission of the model is realized either with or without dead time. In this article we only used the latter. In the presence of a dead time, which is constrained to be larger than the simulation time step, at most one spike can be generated within a time step. A spike is hence emitted with the probability <italic>p</italic><sub>&#x02265;1</sub> &#x0003D; 1 &#x02212; <italic>e</italic><sup>&#x003BB;&#x00394;<italic>t</italic></sup>, where <italic>e</italic><sup>&#x003BB;&#x00394;<italic>t</italic></sup> is the probability of the complementary event (emitting 0 spikes), implemented by comparing a uniformly distributed random number to <italic>p</italic><sub>&#x02265;1</sub>. The refractory period is handled as described in Morrison et al. (<xref ref-type="bibr" rid="B36">2005</xref>). Without refractoriness, the number of emitted spikes is drawn from a Poisson distribution with parameter &#x003BB;&#x00394;<italic>t</italic>, implemented in the GNU Scientific Library (Galassi et al., <xref ref-type="bibr" rid="B17">2006</xref>). Reproducibility of the random sequences for different numbers of processes and threads is ensured by the concept of random number generators assigned to virtual processes, as described in (Plesser et al., <xref ref-type="bibr" rid="B41">2007</xref>).</p>
</sec>
<sec>
<title>Parameters of simulations</title>
<p>For all simulations we used &#x003B3; &#x0003D; 0.25 corresponding to the biologically realistic fraction of inhibitory neurons, a connectivity probability <italic>p</italic> &#x0003D; 0.1, and a simulation time step of &#x00394;<italic>t</italic> &#x0003D; 0.1 ms. For binary neurons we measured the covariance functions with a resolution of 1 ms, for all other models the resolution is 0.1 ms. Simulation time is 10, 000 ms for linear rate and for LIF neurons, 50, 000 ms for Hawkes, and 100, 000 ms for binary neurons. The covariance is obtained for a time window of &#x000B1;100 ms.</p>
<p>The parameters for simulations of the LIF model presented in Figure <xref ref-type="fig" rid="F7">7</xref> and Figure <xref ref-type="fig" rid="F8">8</xref> are <italic>J</italic> &#x0003D; 0.1 mV, &#x003C4; &#x0003D; 20 ms, &#x003C4;<sub><italic>s</italic></sub> &#x0003D; 2 ms, &#x003C4;<sub><italic>r</italic></sub> &#x0003D; 2 ms, <italic>V</italic><sub>&#x003B8;</sub> &#x0003D; 15 mV, <italic>V</italic><sub><italic>r</italic></sub> &#x0003D; 0, <italic>g</italic> &#x0003D; 6, <italic>d</italic> &#x0003D; 3 ms, <italic>N</italic> &#x0003D; 8000. The number of neurons in the corresponding networks of other models is the same. Cross covariances are measured between the summed spike trains of two disjoint populations of <italic>N</italic><sub>rec</sub> &#x0003D; 1000 neurons each. The single neuron autocovariances <italic>a</italic><sub>&#x003B1;</sub> are averaged over a subpopulation of 100 neurons. The autocovariances of the population averaged activity <inline-formula><mml:math id="M89"><mml:mrow><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:msub><mml:mi>N</mml:mi><mml:mi>&#x003B1;</mml:mi></mml:msub></mml:mrow></mml:mfrac><mml:msub><mml:mi>a</mml:mi><mml:mi>&#x003B1;</mml:mi></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>C</mml:mi><mml:mrow><mml:mi>&#x003B1;</mml:mi><mml:mi>&#x003B1;</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:math></inline-formula> for population &#x003B1; &#x02208; {<graphic xlink:href="fncom-07-00131-i0002.tif"/>,<graphic xlink:href="fncom-07-00131-i0003.tif"/>} (shown in Figure <xref ref-type="fig" rid="F7">7</xref>) are constructed from the estimated single neuron population averaged autocovariances <italic>a</italic><sub>&#x003B1;</sub> and cross covariances <italic>C</italic><sub>&#x003B1;&#x003B1;</sub>. This enables us to estimate <italic>a</italic><sub>&#x003B1;</sub> and <italic>C</italic><sub>&#x003B1;&#x003B1;</sub> from the activity of a small subpopulation and still assigns the correct relative weights to both contributions. The corresponding effective parameters describing the system dynamics are &#x003BC; &#x0003D; 15 mV, &#x003C3; &#x0003D; 10 mV, <italic>r</italic> &#x0003D; 23.6 Hz (see (40) and the following text for details).</p>
<p>The parameters of the Hawkes model and of the noisy rate model with output noise yielding quantitatively agreeing covariance functions are:
<list list-type="bullet">
<list-item><p>For simulations of the noisy rate model with output noise presented in Figure <xref ref-type="fig" rid="F7">7</xref> and Figure <xref ref-type="fig" rid="F2">2</xref> the parameters are <italic>w</italic> &#x0003D; 0.0043, <italic>g</italic> &#x02248; 5.93, &#x003C4; &#x0003D; 4.07 ms, &#x003C1;<sup>2</sup> &#x0003D; 23.6 Hz, <italic>d</italic> &#x0003D; 3 ms (see (3), (4)). In Figure <xref ref-type="fig" rid="F2">2</xref> also results for <italic>d</italic> &#x0003D; 1 ms and for input noise are shown. Signals are measured from <italic>N</italic><sub>rec</sub> &#x0003D; 500 neurons in each population to obtain <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub>, c<sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub> and from the whole population to determine <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub> and <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub>. The cross covariances <italic>C</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub> and <italic>C</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> are estimated from two disjoint subpopulations each comprising half of the neurons of the respective population.</p></list-item>
<list-item><p>For the network of Hawkes neurons presented in Figure <xref ref-type="fig" rid="F7">7</xref> we used &#x003BB;<sub>0</sub> &#x02248; 22.54 Hz (see (38)), <italic>J</italic> &#x0003D; 0.0055 mV, <italic>d</italic> &#x0003D; 3 ms, and the same <italic>g</italic> and &#x003C4; as for the noisy rate model. We measured the cross covariances in the same way as for the LIF model, but using the spike trains from sub-populations of <italic>N</italic><sub>rec</sub> &#x0003D; 2000 neurons. The autocovariances of the population averaged activity were estimated from the whole populations.</p></list-item>
</list></p>
<p>The network of binary neurons shown in Figure <xref ref-type="fig" rid="F8">8</xref> uses &#x003B8; &#x0003D; &#x02212;3.89 mV, &#x003B2; &#x0003D; 0.5 mV<sup>&#x02212;1</sup>, <italic>J</italic> &#x0003D; 0.02 mV, <italic>d</italic> &#x0003D; 3 ms (see (25), (44)), and the same <italic>g</italic> and &#x003C4; as the noisy rate model. Covariances are measured using the signals from all neurons.</p>
<p>The simulation results for the network of binary neurons presented in Figure <xref ref-type="fig" rid="F6">6</xref> uses &#x003B8; &#x0003D; &#x02212;2.5 mV, &#x003C4; &#x0003D; 10 ms, &#x003B2; &#x0003D; 0.5 mV<sup>&#x02212;1</sup>, <italic>g</italic> &#x0003D; 6, <italic>J</italic> &#x02248; 0.0447 mV, <italic>N</italic> &#x0003D; 2000 and the smallest possible value of synaptic delay is <italic>d</italic> &#x0003D; 0.1 ms equal to time resolution (the same set of parameters only with modified &#x003B2; &#x0003D; 1 mV<sup>&#x02212;1</sup> was used to create Figure <xref ref-type="fig" rid="F5">5</xref>). The cross covariances <italic>C</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub> and <italic>C</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> are estimated from two disjoint subpopulations each comprising half of the neurons of the respective population, <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> is measured between two such subpopulations. For <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/><inline-graphic xlink:href="fncom-07-00131-i0002.tif"/></sub> and <italic>c</italic><sub><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/><inline-graphic xlink:href="fncom-07-00131-i0003.tif"/></sub> we used the full populations.</p>
<p>The parameters required for a quantitative agreement with the rate model with input noise are <italic>w</italic> &#x02248; 0.011, &#x003C1; &#x02248; 2.23 <inline-formula><mml:math id="M90"><mml:mrow><mml:msqrt><mml:mrow><mml:mi>m</mml:mi><mml:mi>s</mml:mi></mml:mrow></mml:msqrt></mml:mrow></mml:math></inline-formula>. We used the same parameters in Figure <xref ref-type="fig" rid="F3">3</xref>, where additionally results for <italic>w</italic> &#x0003D; 0.018 are shown. The population sizes are the same as for the binary network. The covariances are estimated in the same way as for the rate model with output noise. Note that the definition of noisy rate models has no limitation for units of &#x003C1;<sup>2</sup>. These can be arbitrary and are chosen differently as required by the correspondence with either spiking or binary neurons.</p>
</sec>
</app>
</app-group>
</back>
</article>