<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xml:lang="EN" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Neurosci.</journal-id>
<journal-title>Frontiers in Neuroscience</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Neurosci.</abbrev-journal-title>
<issn pub-type="epub">1662-453X</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fnins.2025.1656892</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Neuroscience</subject>
<subj-group>
<subject>Original Research</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Spike-based time-domain analog weighted-sum calculation model for extremely low power VLSI implementation of multi-layer neural networks</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name><surname>Wang</surname> <given-names>Quan</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/2984376/overview"/>
<role content-type="https://credit.niso.org/contributor-roles/investigation/"/>
<role content-type="https://credit.niso.org/contributor-roles/software/"/>
<role content-type="https://credit.niso.org/contributor-roles/conceptualization/"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-review-editing/"/>
<role content-type="https://credit.niso.org/contributor-roles/visualization/"/>
<role content-type="https://credit.niso.org/contributor-roles/validation/"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-original-draft/"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Tamukoh</surname> <given-names>Hakaru</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/3185637/overview"/>
<role content-type="https://credit.niso.org/contributor-roles/project-administration/"/>
<role content-type="https://credit.niso.org/contributor-roles/resources/"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-review-editing/"/>
<role content-type="https://credit.niso.org/contributor-roles/supervision/"/>
</contrib>
<contrib contrib-type="author" corresp="yes">
<name><surname>Morie</surname> <given-names>Takashi</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
<xref ref-type="corresp" rid="c001"><sup>&#x0002A;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/3160610/overview"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-review-editing/"/>
<role content-type="https://credit.niso.org/contributor-roles/project-administration/"/>
<role content-type="https://credit.niso.org/contributor-roles/conceptualization/"/>
<role content-type="https://credit.niso.org/contributor-roles/funding-acquisition/"/>
<role content-type="https://credit.niso.org/contributor-roles/supervision/"/>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>Graduate School of Life Science and Systems Engineering, Kyushu Institute of Technology</institution>, <addr-line>Kitakyushu</addr-line>, <country>Japan</country></aff>
<aff id="aff2"><sup>2</sup><institution>Research Center for Neuromorphic AI Hardware, Kyushu Institute of Technology</institution>, <addr-line>Kitakyushu</addr-line>, <country>Japan</country></aff>
<author-notes>
<fn fn-type="edited-by"><p>Edited by: Thomas Martin McGinnity, Ulster University, United Kingdom</p></fn>
<fn fn-type="edited-by"><p>Reviewed by: A. N. M. Nafiul Islam, The Pennsylvania State University (PSU), United States</p>
<p>Zihao Xuan, The Hong Kong University of Science and Technology, Hong Kong SAR, China</p></fn>
<corresp id="c001">&#x0002A;Correspondence: Takashi Morie <email>morie&#x00040;brain.kyutech.ac.jp</email></corresp>
</author-notes>
<pub-date pub-type="epub">
<day>12</day>
<month>09</month>
<year>2025</year>
</pub-date>
<pub-date pub-type="collection">
<year>2025</year>
</pub-date>
<volume>19</volume>
<elocation-id>1656892</elocation-id>
<history>
<date date-type="received">
<day>30</day>
<month>06</month>
<year>2025</year>
</date>
<date date-type="accepted">
<day>14</day>
<month>08</month>
<year>2025</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x000A9; 2025 Wang, Tamukoh and Morie.</copyright-statement>
<copyright-year>2025</copyright-year>
<copyright-holder>Wang, Tamukoh and Morie</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p></license>
</permissions>
<abstract>
<p>In deep neural network (DNN) models, the weighted summation, or multiply-and-accumulate (MAC) operation, is an essential and heavy calculation task, which leads to high power consumption in current digital processors. The use of analog operation in complementary metal-oxide-semiconductor (CMOS) very-large-scale integration (VLSI) circuits is a promising method for achieving extremely low power-consumption operation for such calculation tasks. In this paper, a time-domain analog weighted-sum calculation model is proposed based on an integrate-and-fire-type spiking neuron model. The proposed calculation model is applied to multi-layer feedforward networks, in which weighted summations with positive and negative weights are separately performed, and two timings proportional to the positive and negative ones are produced, respectively, in each layer. The timings are then fed into the next layers without their subtraction operation. We also propose VLSI circuits to implement the proposed model. Unlike conventional analog voltage or current mode circuits, the time-domain analog circuits use transient operation in charging/discharging processes to capacitors. Since the circuits can be designed without operational amplifiers, they can operate with extremely low power consumption. We designed a proof-of-concept (PoC) CMOS circuit to verify weighted-sum operation with the same weights. Simulation results showed that the precision was above 4-bit, and the energy efficiency for the weighted-sum calculation was 237.7 Tera Operations Per Second Per Watt (TOPS/W), more than one order of magnitude higher than that in state-of-the-art digital AI processors. Our model promises to be a suitable approach for performing intensive in-memory computing (IMC) of DNNs with moderate precision very energy-efficiently while reducing the cost of analog-digital-converter (ADC) overhead.</p></abstract>
<kwd-group>
<kwd>time-domain analog computing</kwd>
<kwd>weighted sum</kwd>
<kwd>spike-based computing</kwd>
<kwd>deep neural networks</kwd>
<kwd>multi-layer perceptron</kwd>
<kwd>artificial intelligence hardware</kwd>
<kwd>AI processor</kwd>
<kwd>matrix-vector multiplication</kwd>
</kwd-group>
<counts>
<fig-count count="15"/>
<table-count count="4"/>
<equation-count count="59"/>
<ref-count count="77"/>
<page-count count="22"/>
<word-count count="13489"/>
</counts>
<custom-meta-wrap>
<custom-meta>
<meta-name>section-at-acceptance</meta-name>
<meta-value>Neuromorphic Engineering</meta-value>
</custom-meta>
</custom-meta-wrap>
</article-meta>
</front>
<body>
<sec sec-type="intro" id="s1">
<title>1 Introduction</title>
<p>Artificial neural networks (ANNs) or deep neural networks (DNNs), such as convolutional deep neural networks (CNNs) (<xref ref-type="bibr" rid="B40">LeCun et al., 2002</xref>) and fully connected multi-layer perceptrons (MLPs) (<xref ref-type="bibr" rid="B16">Cire&#x0015F;an et al., 2010</xref>), have shown excellent performance on various intelligent tasks, such as object detection and image classification (<xref ref-type="bibr" rid="B16">Cire&#x0015F;an et al., 2010</xref>; <xref ref-type="bibr" rid="B36">Krizhevsky et al., 2012</xref>; <xref ref-type="bibr" rid="B39">LeCun et al., 2015</xref>). However, DNNs require an enormous number of parameters and computational capability, resulting in much heavier computation and data movements, which leads to high power consumption in current digital computers, and even in highly parallel coprocessors such as graphics processing units (GPUs). To implement ANNs at edge devices such as mobile phones and personal service robots, very low power consumption operation is required.</p>
<p>In ANN models, the weighted summation, or multiply-and-accumulate (MAC) operation, is an essential and heavy calculation task (<xref ref-type="bibr" rid="B61">Shukla et al., 2019</xref>), and dedicated complementary metal-oxide-semiconductor (CMOS) very-large-scale integration (VLSI) processors have been developed to accomplish it (<xref ref-type="bibr" rid="B12">Chen et al., 2020</xref>). As an implementation approach other than digital processors, the use of analog operation in CMOS VLSI circuits is a promising method for achieving extremely low power-consumption operation for such calculation tasks (<xref ref-type="bibr" rid="B26">Hasler and Marr, 2013</xref>; <xref ref-type="bibr" rid="B19">Fick et al., 2017</xref>; <xref ref-type="bibr" rid="B45">Mahmoodi and Strukov, 2018</xref>; <xref ref-type="bibr" rid="B9">Bavandpour et al., 2020</xref>).</p>
<p>From the perspective of computing architecture, it is well known that traditional von Neumann-based architectures require the movement of weights and intermediate computing results between memory and processing units, resulting in extra latency and energy consumption, which is further aggravated in the data-intensive applications of DNNs (<xref ref-type="bibr" rid="B27">Horowitz, 2014</xref>; <xref ref-type="bibr" rid="B63">Sze et al., 2017</xref>). To reduce or eliminate power consumption and latency of data movement over memory, in-memory computing (IMC) with SRAM or memristive devices, which act as analog memory, is becoming a promising paradigm for accelerating DNNs. Analog in-memory computing (AIMC), which combines analog computation with the IMC architecture, can provide better energy efficiency by performing MACs in parallel-also known as matrix-vector multiplications (MVMs)-within the memory array in a single step (<xref ref-type="bibr" rid="B67">Verma et al., 2019</xref>; <xref ref-type="bibr" rid="B66">Valavi et al., 2019</xref>; <xref ref-type="bibr" rid="B31">Jiang et al., 2020</xref>; <xref ref-type="bibr" rid="B28">Jia et al., 2021a</xref>; <xref ref-type="bibr" rid="B54">Prezioso et al., 2015</xref>; <xref ref-type="bibr" rid="B60">Shafiee et al., 2016</xref>; <xref ref-type="bibr" rid="B65">Tsai et al., 2018</xref>; <xref ref-type="bibr" rid="B33">Khaddam-Aljameh et al., 2022</xref>). Typically, an AIMC core executes analog MVMs for a single ANN layer, multiplying the stationary weight matrix stored in the core with the activation vector applied at its input. SRAM-based implementations cannot hold the whole weight of larger networks fully on-chip because of their large areas for storing multi-bit weights (<xref ref-type="bibr" rid="B29">Jia et al., 2021b</xref>). As a result, off-chip weight buffers are needed to store network weights and transfer them partially to AIMC cores, which further reduces energy efficiency. Additionally, SRAM&#x00027;s volatility means that on-chip weights are lost when power is turned off. Analog non-volatile memory (NVM) technologies, such as resistive memory and flash memory, offer multiple bits per device, high density, and non-volatility, making it possible to store entire network weights on-chip. This has led to an emerging trend in NVM-based analog AI systems, where an increasing number of AIMC cores or tiles are deployed to efficiently conduct inference tasks (<xref ref-type="bibr" rid="B68">Wan et al., 2022</xref>; <xref ref-type="bibr" rid="B20">Fick et al., 2022</xref>; <xref ref-type="bibr" rid="B38">Le Gallo et al., 2023</xref>; <xref ref-type="bibr" rid="B6">Ambrogio et al., 2023</xref>).</p>
<p>Despite the exciting opportunities for energy-efficient ANN processing, AIMC-based AI systems also present unique challenges that must be addressed to realize their full potential. Beyond the limitations in computation accuracy, a critical challenge is the additional need for digital-to-analog converters (DACs) and analog-to-digital converters (ADCs) to transfer intermediate data between layers or tiles in digital form and to interface with digital peripheral circuits when MAC computations are performed in the current or voltage domain. These converters significantly limit the energy efficiency and scalability of AIMC systems (<xref ref-type="bibr" rid="B60">Shafiee et al., 2016</xref>; <xref ref-type="bibr" rid="B46">Marinella et al., 2018</xref>; <xref ref-type="bibr" rid="B65">Tsai et al., 2018</xref>; <xref ref-type="bibr" rid="B67">Verma et al., 2019</xref>; <xref ref-type="bibr" rid="B28">Jia et al., 2021a</xref>). To mitigate or overcome the limitations of DACs and ADCs, AIMC cores are increasingly adopting time-domain (TD) computing for MAC operations and interfacing with digital systems through digital-to-time converters (DTCs) and time-to-digital converters (TDCs) (<xref ref-type="bibr" rid="B8">Bavandpour et al., 2019a</xref>,<xref ref-type="bibr" rid="B10">b</xref>, <xref ref-type="bibr" rid="B11">2021</xref>; <xref ref-type="bibr" rid="B76">Yang et al., 2019</xref>; <xref ref-type="bibr" rid="B21">Freye et al., 2022</xref>; <xref ref-type="bibr" rid="B73">Wu et al., 2022</xref>; <xref ref-type="bibr" rid="B5">Al Maharmeh et al., 2023</xref>; <xref ref-type="bibr" rid="B14">Choi et al., 2023</xref>). TD computing offers better technology scaling than voltage- and current-domain approaches (<xref ref-type="bibr" rid="B4">Al Maharmeh et al., 2020</xref>; <xref ref-type="bibr" rid="B22">Freye et al., 2024</xref>; <xref ref-type="bibr" rid="B3">Al Maharmeh et al., 2024</xref>), while DTCs and TDCs are generally more energy- and area-efficient than DACs and ADCs (<xref ref-type="bibr" rid="B13">Chen et al., 2022</xref>; <xref ref-type="bibr" rid="B33">Khaddam-Aljameh et al., 2022</xref>). Most TD schemes represent data using pulse-width modulation (PWM) or delay. In multi-core TD AIMC systems, time-based communication-where pulses are transmitted directly from tile to tile or layer to layer in an analog manner-is emerging as another key trend (<xref ref-type="bibr" rid="B41">Lim et al., 2020</xref>; <xref ref-type="bibr" rid="B52">Narayanan et al., 2021</xref>; <xref ref-type="bibr" rid="B30">Jiang et al., 2022</xref>; <xref ref-type="bibr" rid="B59">Seo et al., 2022</xref>; <xref ref-type="bibr" rid="B50">N&#x000E4;gele et al., 2023</xref>; <xref ref-type="bibr" rid="B6">Ambrogio et al., 2023</xref>). Because of the elimination of most DTCs and TDCs, system energy efficiency can be further improved. However, the TD analog computation is inherently susceptible to analog non-idealities, such as process, voltage, and temperature (PVT) variations, limiting the computation precision. Regarding accuracy, it is now widely recognized that 4-8 bits provide stable inference performance for most mainstream applications (<xref ref-type="bibr" rid="B25">Gupta et al., 2015</xref>; <xref ref-type="bibr" rid="B47">McKinstry et al., 2018</xref>; <xref ref-type="bibr" rid="B15">Choi et al., 2018</xref>), and this level of precision can typically be achieved through carefully designed analog circuits.</p>
<p>Successful DNNs are based on the second-generation artificial neuron model, which processes real-valued data and utilizes nonlinear activation functions (<xref ref-type="bibr" rid="B55">Roy et al., 2019</xref>). Most DNN architectures require signed computations. Since the rectified linear unit (ReLU) activation function (<xref ref-type="bibr" rid="B51">Nair and Hinton, 2010</xref>) is commonly used and the inputs of the first layer can be normalized as non-negative, IMC architectures have primarily been explored for two-quadrant MACs. However, to accommodate a wider range of applications, the AIMC system needs to support four-quadrant MACs (<xref ref-type="bibr" rid="B33">Khaddam-Aljameh et al., 2022</xref>; <xref ref-type="bibr" rid="B38">Le Gallo et al., 2023</xref>; <xref ref-type="bibr" rid="B6">Ambrogio et al., 2023</xref>; <xref ref-type="bibr" rid="B37">Le Gallo et al., 2024</xref>).</p>
<p>The time-domain weighted-sum calculation model was initially proposed based on the third-generation neuron model-spiking neurons inspired by the behavior of biological neurons (<xref ref-type="bibr" rid="B42">Maass, 1997a</xref>,<xref ref-type="bibr" rid="B43">b</xref>, <xref ref-type="bibr" rid="B44">1999</xref>), to implement real-valued MACs in ANNs. In the model, inputs and outputs are encoded as spike timings, and weights are represented by the rising slope of the post-synaptic potential (PSP).</p>
<p>Subsequent research has simplified and expanded this model under the assumption of operation in analog circuits with transient states (<xref ref-type="bibr" rid="B48">Morie et al., 2016</xref>, <xref ref-type="bibr" rid="B49">2010</xref>; <xref ref-type="bibr" rid="B64">Tohara et al., 2016</xref>). However, they have been limited to one-quadrant weighted-sum models where all weights for a single neuron share the same sign, and these studies did not address how to apply their models to neural networks. The proposed analog circuit, consisting of multiple input resistive elements and a capacitor (an RC circuit), enables extremely low-power operation, with energy consumption potentially reduced to the order of 0.1 fJ per operation. Throughout this paper, we refer to this VLSI implementation approach as &#x0201C;time-domain analog computing with transient states (TACT).&#x0201D; Unlike conventional weighted-sum operations in analog voltage or current modes, the TACT approach is well-suited for achieving much lower power consumption in CMOS VLSI implementations of ANNs. In this work, we extend the model to address the above-described challenges associated with NVM-based AIMC AI systems. Our primary contributions are summarized as follows.</p>
<list list-type="simple">
<list-item><p>1) We extend the time-domain one-quadrant MAC calculation model to four-quadrant one, where signed inputs are encoded using a differential pair of spikes, and signed weights are implemented through a dummy weights scheme. The output is represented by a pair of spikes, with their timing difference proportional to the MAC result, enabled by the added dummy weights. Since both inputs and outputs are encoded in a timing format, the AIMC core can be seamlessly integrated with efficient DTCs and TDCs.</p></list-item>
<list-item><p>2) We theoretically demonstrate how MAC output spikes can be directly transferred to the next layer, potentially eliminating the need for DTCs and TDCs between tiles. Additionally, we provide a clear explanation of the challenges associated with spike transfer in this process.</p></list-item>
<list-item><p>3) We propose two sets of analog circuits, each consisting of multiple input resistive elements and a capacitor (an RC circuit), to implement the four-quadrant MAC computation. Additionally, we describe a proof-of-concept (PoC) CMOS circuit equivalent to the RC circuit, with a preliminary estimation suggesting that the energy efficiency could reach hundreds of TOPS/W (Tera Operations Per Second Per Watt) and the precision could be four bit or higher.</p></list-item>
<list-item><p>4) We propose architectures for an analog NVM-based AIMC core and system. In the core, signed weights can be implemented using either a complementary scheme or a differential scheme, depending on how dummy weights are introduced. At the system level, tile-to-tile communication is achieved through spike-based transmission in an analog manner.</p></list-item>
</list></sec>
<sec id="s2">
<title>2 Spike-based time-domain weighted-sum calculation model</title>
<sec>
<title>2.1 Time-domain weighted-sum calculation with same-signed weights</title>
<p>A simple spiking neuron model, also known as an integrate-and-fire-type (IF) neuron model, is shown in <xref ref-type="fig" rid="F1">Figure 1</xref> (<xref ref-type="bibr" rid="B44">Maass, 1999</xref>). In this model, a neuron receives spike pulses via synapses. A spike pulse only indicates the input timing, and its pulse width and amplitude do not affect the following processing. A spike generates a temporal voltage change, which is called a post-synaptic potential (PSP), and the internal potential of the <italic>n</italic>-th neuron, <italic>V</italic><sub><italic>n</italic></sub>(<italic>t</italic>), is equal to the spatiotemporal summation of all PSPs. When <italic>V</italic><sub><italic>n</italic></sub>(<italic>t</italic>) reaches the firing threshold &#x003B8;, the neuron outputs a spike, and <italic>V</italic><sub><italic>n</italic></sub>(<italic>t</italic>) then settles back to the steady state.</p>
<fig position="float" id="F1">
<label>Figure 1</label>
<caption><p>IF neuron model for weighted-sum operation: schematic of the model and weighted-sum operation using the rise timing of PSPs.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0001.tif">
<alt-text>Diagram illustrating a spiking neuron model. Shows a circle with labeled segments: input spikes i1, i2, i3, potential Vn, firing threshold &#x003B8;, output spike in, and PSPs P1, P2, P3. Features graphs of input spikes i(t), their PSPs P(t), cumulative potential Vn(t), and firing threshold &#x003B8;. Time markers t1, t2, t3, tv indicate event sequences, with Tin, Tv for intervals. Illustrates neuron firing dynamics and thresholds.</alt-text>
</graphic>
</fig>
<p>Based on the model proposed in <xref ref-type="bibr" rid="B42">Maass (1997a)</xref>, a simplified weighted-sum operation model using IF neurons is proposed. The time span <italic>T</italic><sub><italic>in</italic></sub> is defined, during which only one spike is fed from each neuron, and it is assumed that a PSP generated by a spike from neuron <italic>i</italic> increases linearly with slope <italic>k</italic><sub><italic>i</italic></sub> from the timing of the spike input, <italic>t</italic><sub><italic>i</italic></sub>, as shown in <xref ref-type="fig" rid="F1">Figure 1</xref>.</p>
<p>A required weighted-sum operation is that normalized variables <italic>x</italic><sub><italic>i</italic></sub> (0 &#x02264; <italic>x</italic><sub><italic>i</italic></sub> &#x02264; 1, <italic>i</italic> &#x0003D; 1, 2, &#x022EF;&#x000A0;, <italic>N</italic>) are multiplied by weight coefficients <italic>a</italic><sub><italic>i</italic></sub>, and the multiplication results are summed regarding <italic>i</italic>, where <italic>N</italic> is the number of inputs. This weighted-sum operation can be performed using the rise timing of PSPs in the IF neuron model. Input spike timing <italic>t</italic><sub><italic>i</italic></sub> is determined based on <italic>x</italic><sub><italic>i</italic></sub> using the following relation:</p>
<disp-formula id="E1"><label>(1)</label><mml:math id="M1"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>-</mml:mo><mml:msub><mml:mrow><mml:mi>x</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E2"><label>(2)</label><mml:math id="M2"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>x</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>-</mml:mo><mml:mfrac><mml:mrow><mml:msub><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Coefficients <italic>a</italic><sub><italic>i</italic></sub> are transformed into the PSP slopes <italic>k</italic><sub><italic>i</italic></sub>:</p>
<disp-formula id="E3"><label>(3)</label><mml:math id="M3"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mi>&#x003BB;</mml:mi><mml:msub><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>where &#x003BB; is a positive constant. If the firing time of the neuron is defined as <italic>t</italic><sub>&#x003BD;</sub>, we easily obtain the equation</p>
<disp-formula id="E4"><label>(4)</label><mml:math id="M4"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munderover></mml:mstyle><mml:msub><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow></mml:msub><mml:mo>-</mml:mo><mml:msub><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mi>&#x003B8;</mml:mi><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>If we define the following parameters:</p>
<disp-formula id="E5"><label>(5)</label><mml:math id="M5"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>&#x003B2;</mml:mi><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munderover></mml:mstyle><mml:msub><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>we obtain</p>
<disp-formula id="E6"><label>(6)</label><mml:math id="M6"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munderover></mml:mstyle><mml:msub><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>&#x000B7;</mml:mo><mml:msub><mml:mrow><mml:mi>x</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mfrac><mml:mrow><mml:mi>&#x003B8;</mml:mi><mml:mo>/</mml:mo><mml:mi>&#x003BB;</mml:mi><mml:mo>&#x0002B;</mml:mo><mml:mi>&#x003B2;</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>-</mml:mo><mml:msub><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E7"><label>(7)</label><mml:math id="M7"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mfrac><mml:mrow><mml:mi>&#x003B8;</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BB;</mml:mi><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mo>&#x0002B;</mml:mo><mml:mi>&#x003B2;</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>-</mml:mo><mml:mfrac><mml:mrow><mml:msub><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Here, we assume that all the weights in the calculation have the same sign, i.e., <italic>a</italic><sub><italic>i</italic></sub>&#x02265;0 or <italic>a</italic><sub><italic>i</italic></sub> &#x02264; 0 for all <italic>i</italic>. This is different from the previous similar work in which only the sum of weights is restricted to be positive for firing (<xref ref-type="bibr" rid="B77">Zhang et al., 2021</xref>). When all inputs are minimum (&#x02200;<italic>i x</italic><sub><italic>i</italic></sub> &#x0003D; 0), the left side of <xref ref-type="disp-formula" rid="E6">Equation 6</xref> is zero. Then, the output timing <italic>t</italic><sub>&#x003BD;</sub> is given by</p>
<disp-formula id="E8"><label>(8)</label><mml:math id="M8"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mi>m</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mi>&#x003B8;</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BB;</mml:mi><mml:mi>&#x003B2;</mml:mi></mml:mrow></mml:mfrac><mml:mo>&#x0002B;</mml:mo><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>On the other hand, when all inputs are maximum (&#x02200;<italic>i x</italic><sub><italic>i</italic></sub> &#x0003D; 1), the left side of <xref ref-type="disp-formula" rid="E6">Equation 6</xref> is &#x003B2;, and the output timing <italic>t</italic><sub>&#x003BD;</sub> is given by</p>
<disp-formula id="E9"><label>(9)</label><mml:math id="M9"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mi>m</mml:mi><mml:mi>a</mml:mi><mml:mi>x</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mi>&#x003B8;</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BB;</mml:mi><mml:mi>&#x003B2;</mml:mi></mml:mrow></mml:mfrac><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>The time span during which <italic>t</italic><sub>&#x003BD;</sub> can be output is <inline-formula><mml:math id="M10"><mml:mrow><mml:mo>[</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mi>m</mml:mi><mml:mi>a</mml:mi><mml:mi>x</mml:mi></mml:mrow></mml:msubsup><mml:mo>,</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mi>m</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msubsup></mml:mrow><mml:mo>]</mml:mo></mml:mrow></mml:math></inline-formula>, and its interval is</p>
<disp-formula id="E10"><label>(10)</label><mml:math id="M11"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>o</mml:mi><mml:mi>u</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:msub><mml:mo>&#x02261;</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mi>m</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mi>m</mml:mi><mml:mi>a</mml:mi><mml:mi>x</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Thus, the time span of output spikes is the same as that of input spikes, <italic>T</italic><sub><italic>in</italic></sub>.</p>
<p>In this model, since the normalization of the sum of <italic>a</italic><sub><italic>i</italic></sub> (&#x003B2; &#x0003D; 1) is not required [unlike in the previous work (<xref ref-type="bibr" rid="B42">Maass, 1997a</xref>, <xref ref-type="bibr" rid="B44">1999</xref>; <xref ref-type="bibr" rid="B64">Tohara et al., 2016</xref>)], the calculation process becomes much simpler. When implementing the time-domain weighted-sum operation, setting the threshold potential &#x003B8; properly is the key to making the operation work appropriately. As shown in <xref ref-type="fig" rid="F1">Figure 1</xref>, the earliest output spike timing has to be later than the latest input spike timing <italic>T</italic><sub><italic>in</italic></sub>; that is, <inline-formula><mml:math id="M12"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mi>m</mml:mi><mml:mi>a</mml:mi><mml:mi>x</mml:mi></mml:mrow></mml:msubsup><mml:mo>&#x02265;</mml:mo><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula>. Thus,</p>
<disp-formula id="E11"><label>(11)</label><mml:math id="M13"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>&#x003B8;</mml:mi><mml:mo>&#x02265;</mml:mo><mml:mi>&#x003BB;</mml:mi><mml:mi>&#x003B2;</mml:mi><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Also, we can rewrite <xref ref-type="disp-formula" rid="E11">Equation 11</xref> as</p>
<disp-formula id="E12"><label>(12)</label><mml:math id="M14"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>&#x003B8;</mml:mi></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mi>&#x003BB;</mml:mi><mml:mi>&#x003B2;</mml:mi><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>&#x0002B;</mml:mo><mml:mi>&#x003B4;</mml:mi><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E13"><label>(13)</label><mml:math id="M15"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>&#x003B4;</mml:mi></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mi>&#x003F5;</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>&#x003BB;</mml:mi><mml:mi>&#x003B2;</mml:mi><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>where &#x003F5;&#x02265;0 is an arbitrarily small value. By substituting <xref ref-type="disp-formula" rid="E12">Equations 12</xref>, <xref ref-type="disp-formula" rid="E13">13</xref> into <xref ref-type="disp-formula" rid="E8">Equations 8</xref>, <xref ref-type="disp-formula" rid="E9">9</xref>, we obtain</p>
<disp-formula id="E14"><label>(14)</label><mml:math id="M16"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mi>m</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>2</mml:mn><mml:mo>&#x0002B;</mml:mo><mml:mi>&#x003F5;</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E15"><label>(15)</label><mml:math id="M17"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mi>m</mml:mi><mml:mi>a</mml:mi><mml:mi>x</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x0002B;</mml:mo><mml:mi>&#x003F5;</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>where &#x003F5;<italic>T</italic><sub><italic>in</italic></sub> is considered as a time slot between input and output timing spans, as shown in <xref ref-type="fig" rid="F1">Figure 1</xref>, and &#x003F5; determines the length of the slot.</p></sec>
<sec>
<title>2.2 Time-domain weighted-sum calculation with different-signed weights</title>
<p>We propose a time-domain weighted-sum calculation model with two spiking neurons, one for all the positive weights and the other for all the negative ones. We apply <xref ref-type="disp-formula" rid="E6">Equation 6</xref> to each neuron, and the two results are summed as the final result of the original weighted sum. Here, we show the details of the model.</p>
<p>Let <inline-formula><mml:math id="M18"><mml:msubsup><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M19"><mml:msubsup><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> indicate the positive and negative weights, respectively. We define</p>
<disp-formula id="E16"><label>(16)</label><mml:math id="M20"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msup><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:munderover></mml:mstyle><mml:msubsup><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02265;</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mtext>&#x02003;</mml:mtext><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msup><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:munderover></mml:mstyle><mml:msubsup><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02264;</mml:mo><mml:mn>0</mml:mn><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Where <italic>N</italic><sup>&#x0002B;</sup> and <italic>N</italic><sup>&#x02212;</sup> are the numbers of positive and negative weights, respectively:</p>
<disp-formula id="E17"><label>(17)</label><mml:math id="M21"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>N</mml:mi><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msup><mml:mo>&#x0002B;</mml:mo><mml:msup><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msup><mml:mo>,</mml:mo><mml:mtext>&#x02003;</mml:mtext><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munderover></mml:mstyle><mml:msub><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msup><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:munderover></mml:mstyle><mml:msubsup><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x0002B;</mml:mo><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msup><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:munderover></mml:mstyle><mml:msubsup><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>,</mml:mo><mml:mtext>&#x02003;</mml:mtext><mml:mi>&#x003B2;</mml:mi><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msup><mml:mo>&#x0002B;</mml:mo><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msup><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Thus, assuming &#x003BB; &#x0003D; 1, <xref ref-type="disp-formula" rid="E4">Equation 4</xref> is rewritten for the positive and negative weighted-sum operations as</p>
<disp-formula id="E18"><label>(18)</label><mml:math id="M22"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msup><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:munderover></mml:mstyle><mml:msubsup><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msub><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mi>&#x003B8;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msup><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E19"><label>(19)</label><mml:math id="M23"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msup><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:munderover></mml:mstyle><mml:msubsup><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msub><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mi>&#x003B8;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msup><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>where &#x003B8;<sup>&#x0002B;</sup>(&#x0003E;0), &#x003B8;<sup>&#x02212;</sup>(&#x0003C;0), and <inline-formula><mml:math id="M24"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M25"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> indicate the threshold values and output timings for the positively and negatively weighted-sum operation, respectively. We obtain</p>
<disp-formula id="E20"><label>(20)</label><mml:math id="M26"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msup><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:munderover></mml:mstyle><mml:msubsup><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x000B7;</mml:mo><mml:msub><mml:mrow><mml:mi>x</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mfrac><mml:mrow><mml:msup><mml:mrow><mml:mi>&#x003B8;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msup><mml:mo>&#x0002B;</mml:mo><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msup><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E21"><label>(21)</label><mml:math id="M27"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msup><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:munderover></mml:mstyle><mml:msubsup><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x000B7;</mml:mo><mml:msub><mml:mrow><mml:mi>x</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mfrac><mml:mrow><mml:msup><mml:mrow><mml:mi>&#x003B8;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msup><mml:mo>&#x0002B;</mml:mo><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msup><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Therefore, we can obtain the original weighted-sum result:</p>
<disp-formula id="E22"><label>(22)</label><mml:math id="M28"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mstyle displaystyle='true'><mml:munderover><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:munderover><mml:mrow><mml:msub><mml:mi>a</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:mstyle><mml:mo>&#x000B7;</mml:mo><mml:msub><mml:mi>x</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:msubsup><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msup><mml:mi>N</mml:mi><mml:mo>+</mml:mo></mml:msup></mml:mrow></mml:msubsup><mml:mrow><mml:msubsup><mml:mi>a</mml:mi><mml:mi>i</mml:mi><mml:mo>+</mml:mo></mml:msubsup></mml:mrow></mml:mstyle><mml:mo>&#x000B7;</mml:mo><mml:msub><mml:mi>x</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>+</mml:mo><mml:mstyle displaystyle='true'><mml:msubsup><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msup><mml:mi>N</mml:mi><mml:mo>&#x02212;</mml:mo></mml:msup></mml:mrow></mml:msubsup><mml:mrow><mml:msubsup><mml:mi>a</mml:mi><mml:mi>i</mml:mi><mml:mo>&#x02212;</mml:mo></mml:msubsup></mml:mrow></mml:mstyle><mml:mo>&#x000B7;</mml:mo><mml:msub><mml:mi>x</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:msup><mml:mi>&#x003B8;</mml:mi><mml:mo>+</mml:mo></mml:msup><mml:mo>+</mml:mo><mml:msup><mml:mi>&#x003B8;</mml:mi><mml:mo>&#x02212;</mml:mo></mml:msup><mml:mo>+</mml:mo><mml:mi>&#x003B2;</mml:mi><mml:msub><mml:mi>T</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>&#x02212;</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msup><mml:mi>&#x003B2;</mml:mi><mml:mo>+</mml:mo></mml:msup><mml:msubsup><mml:mi>t</mml:mi><mml:mi>&#x003BD;</mml:mi><mml:mo>+</mml:mo></mml:msubsup><mml:mo>+</mml:mo><mml:msup><mml:mi>&#x003B2;</mml:mi><mml:mo>&#x02212;</mml:mo></mml:msup><mml:msubsup><mml:mi>t</mml:mi><mml:mi>&#x003BD;</mml:mi><mml:mo>&#x02212;</mml:mo></mml:msubsup><mml:mo stretchy='false'>)</mml:mo></mml:mrow><mml:mrow><mml:msub><mml:mi>T</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Let us define a dummy weight <italic>a</italic><sub>0</sub> as the difference between both absolute values of &#x003B2;<sup>&#x000B1;</sup>:</p>
<disp-formula id="E23"><label>(23)</label><mml:math id="M29"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mo>-</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msup><mml:mo>&#x0002B;</mml:mo><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>If &#x003B2;<sup>&#x0002B;</sup>&#x02265;&#x02212;&#x003B2;<sup>&#x02212;</sup>, then <italic>a</italic><sub>0</sub> &#x02264; 0 and this dummy weight is incorporated into the negative weight group, and vice versa. This dummy weight is related to a zero input, <italic>x</italic><sub>0</sub> &#x0003D; 0, which means <italic>t</italic><sub>0</sub> &#x0003D; <italic>T</italic><sub><italic>in</italic></sub>. By using the dummy weight, we can make the absolute values of &#x003B2;<sup>&#x000B1;</sup> identical (&#x003B2; &#x0003D; 0), and we define</p>
<disp-formula id="E24"><label>(24)</label><mml:math id="M30"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mi>o</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:mo>-</mml:mo><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msup><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Also, according to <xref ref-type="disp-formula" rid="E12">Equations 12</xref>, <xref ref-type="disp-formula" rid="E13">13</xref>, the absolute values of &#x003B8;<sup>&#x0002B;</sup> and &#x003B8;<sup>&#x02212;</sup> can be the same, and &#x003B8;<sup>&#x0002B;</sup>&#x0002B;&#x003B8;<sup>&#x02212;</sup> &#x0003D; 0. Therefore, <xref ref-type="disp-formula" rid="E22">Equation 22</xref> can be rewritten as</p>
<disp-formula id="E25"><label>(25)</label><mml:math id="M32"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munderover></mml:mstyle><mml:msub><mml:mrow><mml:mi>a</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>&#x000B7;</mml:mo><mml:msub><mml:mrow><mml:mi>x</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:msub><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mi>o</mml:mi></mml:mrow></mml:msub><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
</sec></sec>
<sec id="s3">
<title>3 Time domain neural network model</title>
<sec>
<title>3.1 Neuron model</title>
<p>The typical neuron model of ANNs is shown in <xref ref-type="fig" rid="F2">Figure 2a</xref>, which has <italic>N</italic> inputs <italic>x</italic><sub><italic>i</italic></sub> with weights <italic>w</italic><sub><italic>i</italic></sub> and a bias <italic>b</italic>;</p>
<disp-formula id="E26"><label>(26)</label><mml:math id="M33"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>y</mml:mi><mml:mo>=</mml:mo><mml:mi>f</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munderover></mml:mstyle><mml:msub><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>&#x000B7;</mml:mo><mml:msub><mml:mrow><mml:mi>x</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>&#x0002B;</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<fig position="float" id="F2">
<label>Figure 2</label>
<caption><p>Neuron model: <bold>(a)</bold> typical neuron model; <bold>(b)</bold> neuron model for time-domain weighted-sum operation with a dummy weight, <italic>w</italic><sub><italic>n</italic>&#x0002B;1</sub>; <bold>(c)</bold> neuron model for time-domain weighted-sum operation in which each synapse has two sets of inputs and weights that one set is (<italic>x</italic><sub><italic>i</italic></sub>, <italic>w</italic><sub><italic>i</italic></sub>) and the other is (0, &#x02212;<italic>w</italic><sub><italic>i</italic></sub>) or (<italic>t</italic><sub><italic>i</italic></sub>, <italic>w</italic><sub><italic>i</italic></sub>) and (<italic>T</italic><sub><italic>in</italic></sub>, &#x02212;<italic>w</italic><sub><italic>i</italic></sub>) according to <xref ref-type="disp-formula" rid="E2">Equation 2</xref>.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0002.tif">
<alt-text>Diagram showing three models: (a) a single-layer neural network with inputs, weights, and bias producing output y. (b) a perceptron model similar to (a) with an additional dummy input and equations for weight adjustments. (c) a perceptron model with absolute weight values for updates. Each model illustrates input connections, weight labels, and output functions.</alt-text>
</graphic>
</fig>
<p>where <italic>y</italic> is the output of the neuron, and <italic>f</italic> is an activation function. We can consider the bias as a weight whose input is always unity and regard the 0 index weight as the bias throughout this paper. Therefore, our time-domain weighted-sum calculation model with the dummy weight can be applied to this neuron model, as shown in <xref ref-type="fig" rid="F2">Figure 2b</xref>. According to <xref ref-type="disp-formula" rid="E25">Equation 25</xref>,</p>
<disp-formula id="E27"><label>(27)</label><mml:math id="M34"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munderover></mml:mstyle><mml:msub><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>&#x000B7;</mml:mo><mml:msub><mml:mrow><mml:mi>x</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mi>&#x003B2;</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Based on <xref ref-type="disp-formula" rid="E27">Equation 27</xref>, we propose another model, shown in <xref ref-type="fig" rid="F2">Figure 2c</xref>, in which each synapse has two sets of inputs and weights; one is (<italic>x</italic><sub><italic>i</italic></sub>, <italic>w</italic><sub><italic>i</italic></sub>) and the other is (0, &#x02212;<italic>w</italic><sub><italic>i</italic></sub>). In this model, it is not necessary to add a dummy weight because the summation of positive weights is equal to the absolute one of negative weights automatically, i.e., <inline-formula><mml:math id="M35"><mml:mi>&#x003B2;</mml:mi><mml:mo>=</mml:mo><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munderover><mml:mo>|</mml:mo><mml:mo>|</mml:mo><mml:msub><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>|</mml:mo><mml:mo>|</mml:mo></mml:math></inline-formula>.</p>
<p>As the activation function <italic>f</italic>, we often use the rectified linear unit called &#x0201C;ReLU&#x0201D; (<xref ref-type="bibr" rid="B51">Nair and Hinton, 2010</xref>), which is defined as follows:</p>
<disp-formula id="E28"><label>(28)</label><mml:math id="M36"><mml:mrow><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>x</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mi>R</mml:mi><mml:mi>e</mml:mi><mml:mi>L</mml:mi><mml:mi>U</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mi>x</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:mrow><mml:mtable columnalign='left'><mml:mtr columnalign='left'><mml:mtd columnalign='left'><mml:mi>x</mml:mi></mml:mtd><mml:mtd columnalign='left'><mml:mrow><mml:mi>i</mml:mi><mml:mi>f</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mi>x</mml:mi><mml:mo>&#x02265;</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr columnalign='left'><mml:mtd columnalign='left'><mml:mn>0</mml:mn></mml:mtd><mml:mtd columnalign='left'><mml:mrow><mml:mi>o</mml:mi><mml:mi>t</mml:mi><mml:mi>h</mml:mi><mml:mi>e</mml:mi><mml:mi>r</mml:mi><mml:mi>w</mml:mi><mml:mi>i</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:mrow><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula>
<p>We can implement the ReLU function by comparing the output timings <inline-formula><mml:math id="M37"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M38"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> in the time-domain weighted-sum calculation as follows:</p>
<disp-formula id="E29"><label>(29)</label><mml:math id="M39"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>f</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munderover></mml:mstyle><mml:msub><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>&#x000B7;</mml:mo><mml:msub><mml:mrow><mml:mi>x</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mi>R</mml:mi><mml:mi>e</mml:mi><mml:mi>L</mml:mi><mml:mi>U</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mi>&#x003B2;</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mi>&#x003B2;</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>where, if <inline-formula><mml:math id="M40"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x0003E;</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula>, the difference between the two timing values is regarded as the output transferred to neurons in the next layer, and if <inline-formula><mml:math id="M41"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x0003C;</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula>, we set <inline-formula><mml:math id="M42"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M43"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> to be identical to make the output zero because of the negative weighted-sum result. Its circuit implementation will be shown later.</p></sec>
<sec>
<title>3.2 Neural network model</title>
<p>In this section, we extend our time-domain neuron model shown in <xref ref-type="fig" rid="F2">Figure 2c</xref> to the neural network and theoretically show the intermediate timing transfer mechanisms between layers. We first apply the procedure to a two-layer MLP, which has one hidden layer and two sets of input and weight for each neuron shown in <xref ref-type="fig" rid="F3">Figure 3</xref>, as an example, and then generalize it.</p>
<fig position="float" id="F3">
<label>Figure 3</label>
<caption><p>General neural network model with two inputs and outputs for time-domain weighted-sum calculation with positive and negative weights.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0003.tif">
<alt-text>Diagram of a neural network with an input layer, hidden layer, and output layer. Nodes are represented by circles connected with directional lines. Input nodes are labeled x1 to xi. The hidden layer has nodes &#x003B8;1(n) and &#x003B8;2(n), with weights w and biases &#x003B2;(n). The output layer node &#x003B8;(p) has weights and biases w(p) and &#x003B2;(p). Connections are color-coded with corresponding equations and terms indicated in blue and black.</alt-text>
</graphic>
</fig>
<p>In this MLP, according to <xref ref-type="disp-formula" rid="E27">Equation 27</xref>, the weighted-sum result of the <italic>j</italic>-th neuron in the hidden layer labeled as <italic>n</italic> can be</p>
<disp-formula id="E30"><label>(30)</label><mml:math id="M44"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munderover></mml:mstyle><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>&#x000B7;</mml:mo><mml:msub><mml:mrow><mml:mi>x</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:msubsup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>where <inline-formula><mml:math id="M45"><mml:msubsup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munderover><mml:mo>|</mml:mo><mml:mo>|</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>|</mml:mo><mml:mo>|</mml:mo></mml:math></inline-formula>, <inline-formula><mml:math id="M46"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M47"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> are the timings generated at the <italic>j</italic>-th neuron in the <italic>n</italic>-th layer. The output <inline-formula><mml:math id="M48"><mml:msubsup><mml:mrow><mml:mi>y</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> of the <italic>k</italic>-th neuron in the output layer labeled as <italic>p</italic>(&#x0003D; <italic>n</italic>&#x0002B;1) is</p>
<disp-formula id="E31"><label>(31)</label><mml:math id="M49"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:msubsup><mml:mi>y</mml:mi><mml:mi>k</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:msubsup><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:msubsup><mml:mrow><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mi>j</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow></mml:mstyle><mml:mo>&#x000B7;</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mstyle displaystyle='true'><mml:msubsup><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:msubsup><mml:mrow><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow></mml:mstyle><mml:mo>&#x000B7;</mml:mo><mml:msub><mml:mi>x</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msubsup><mml:mi>b</mml:mi><mml:mi>k</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:msubsup><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:msubsup><mml:mrow><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mi>j</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow></mml:mstyle><mml:mo>&#x000B7;</mml:mo><mml:mi>f</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mfrac><mml:mrow><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:msub><mml:mi>T</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msubsup><mml:mi>b</mml:mi><mml:mi>k</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:msubsup><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:msubsup><mml:mrow><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mi>j</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow></mml:mstyle><mml:mo>&#x000B7;</mml:mo><mml:mfrac><mml:mrow><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:msub><mml:mi>T</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>where ReLU is used as the activation function, and the bias <inline-formula><mml:math id="M50"><mml:msubsup><mml:mrow><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> is represented in the time-domain model by</p>
<disp-formula id="E32"><label>(32)</label><mml:math id="M51"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:msubsup><mml:mi>b</mml:mi><mml:mi>k</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mn>0</mml:mn><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x000B7;</mml:mo><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mn>0</mml:mn><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x000B7;</mml:mo><mml:mfrac><mml:mrow><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mn>0</mml:mn><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:msub><mml:mi>T</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>in which <inline-formula><mml:math id="M52"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> is paired to <inline-formula><mml:math id="M53"><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula>, <inline-formula><mml:math id="M54"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:math></inline-formula> is paired to <inline-formula><mml:math id="M55"><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M56"><mml:msubsup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:math></inline-formula> as there is no input to the bias.</p>
<p>In the MLP shown in <xref ref-type="fig" rid="F3">Figure 3</xref>, we transfer the output timings <inline-formula><mml:math id="M57"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M58"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> generated in layer <italic>n</italic> to the neurons in layer <italic>p</italic> and perform the time-domain weighted-sum operation. The timings <inline-formula><mml:math id="M59"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M60"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> are assumed to be produced at the <italic>k</italic>-th neuron of layer <italic>p</italic>. We relate timing <inline-formula><mml:math id="M61"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> to weight <inline-formula><mml:math id="M62"><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M63"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> to <inline-formula><mml:math id="M64"><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula>. We also assume here <italic>N</italic> &#x0003D; 3 and that <inline-formula><mml:math id="M65"><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>1</mml:mn><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>&#x02265;</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>&#x0003C;</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>3</mml:mn><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>&#x02265;</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:msubsup><mml:mrow><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>&#x02265;</mml:mo><mml:mn>0</mml:mn></mml:math></inline-formula>, and <inline-formula><mml:math id="M66"><mml:msubsup><mml:mrow><mml:mi>&#x003B8;</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>&#x003B8;</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula>, where <inline-formula><mml:math id="M67"><mml:msubsup><mml:mrow><mml:mi>&#x003B8;</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M68"><mml:msubsup><mml:mrow><mml:mi>&#x003B8;</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> are the threshold values for positively and negatively weighted-sum operations, respectively. Thus, according to <xref ref-type="disp-formula" rid="E4">Equation 4</xref>, we can obtain</p>
<disp-formula id="E33"><label>(33)</label><mml:math id="M69"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mn>1</mml:mn><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mn>2</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mn>3</mml:mn><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mn>3</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msubsup><mml:mi>b</mml:mi><mml:mi>k</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:msubsup><mml:mi>&#x003B8;</mml:mi><mml:mi>k</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E34"><label>(34)</label><mml:math id="M70"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mn>1</mml:mn><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mn>2</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mo>+</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mn>3</mml:mn><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mn>3</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>b</mml:mi><mml:mi>k</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:msubsup><mml:mi>&#x003B8;</mml:mi><mml:mi>k</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>By adding <xref ref-type="disp-formula" rid="E33">Equation 33</xref> to <xref ref-type="disp-formula" rid="E34">Equation 34</xref> on the left and right sides, respectively, the following relationship is obtained:</p>
<disp-formula id="E35"><label>(35)</label><mml:math id="M71"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi><mml:mo>=</mml:mo><mml:mn>3</mml:mn></mml:mrow></mml:munderover></mml:mstyle><mml:mo>||</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||&#x000B7;</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi><mml:mo>=</mml:mo><mml:mn>3</mml:mn></mml:mrow></mml:munderover></mml:mstyle><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>&#x000B7;</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mn>0</mml:mn><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Thus, we can obtain the following simple expression:</p>
<disp-formula id="E36"><label>(36)</label><mml:math id="M72"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi><mml:mo>=</mml:mo><mml:mn>3</mml:mn></mml:mrow></mml:munderover></mml:mstyle><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>&#x000B7;</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x000B7;</mml:mo><mml:mstyle displaystyle="true"><mml:munderover accentunder="false" accent="false"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi><mml:mo>=</mml:mo><mml:mn>3</mml:mn></mml:mrow></mml:munderover></mml:mstyle><mml:mo>||</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Therefore, we generalize the number of neurons <italic>N</italic> &#x0003D; 3 to <italic>N</italic> again, and replace <xref ref-type="disp-formula" rid="E36">Equation 36</xref> with <xref ref-type="disp-formula" rid="E31">Equation 31</xref>. Then, the output <inline-formula><mml:math id="M73"><mml:msubsup><mml:mrow><mml:mi>y</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> in <xref ref-type="disp-formula" rid="E31">Equation 31</xref> can finally be</p>
<disp-formula id="E37"><label>(37)</label><mml:math id="M74"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:msubsup><mml:mi>y</mml:mi><mml:mi>k</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:msubsup><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:msubsup><mml:mrow><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mi>j</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow></mml:mstyle><mml:mo>&#x000B7;</mml:mo><mml:mi>R</mml:mi><mml:mi>e</mml:mi><mml:mi>L</mml:mi><mml:mi>U</mml:mi><mml:mo stretchy='false'>(</mml:mo><mml:mstyle displaystyle='true'><mml:msubsup><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:msubsup><mml:mrow><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow></mml:mstyle><mml:mo>&#x000B7;</mml:mo><mml:msub><mml:mi>x</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo><mml:msubsup><mml:mi>b</mml:mi><mml:mi>k</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x000B7;</mml:mo><mml:mstyle displaystyle='true'><mml:msubsup><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:msubsup><mml:mrow><mml:mo>&#x02016;</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mi>j</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02016;</mml:mo></mml:mrow></mml:mstyle><mml:mo>&#x000B7;</mml:mo><mml:mfrac><mml:mrow><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:msub><mml:mi>T</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>As a result, for neurons in the hidden layer <italic>n</italic>, we apply the time-domain weighted-sum operation to generate the timing <inline-formula><mml:math id="M75"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M76"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> for the positively and negatively weighted-sum calculation from the input layer, respectively. Then, these timings are directly transferred to neurons in the next layer <italic>p</italic>, and timing <inline-formula><mml:math id="M77"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M78"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> are obtained. Finally, we calculate the final outputs of the MLP using <xref ref-type="disp-formula" rid="E37">Equation 37</xref> without calculating the middle layers&#x00027; weighted-sum results using <xref ref-type="disp-formula" rid="E27">Equation 27</xref>.</p>
<p>We summarized the above mathematical time-domain operations in general MLPs graphically as shown in <xref ref-type="fig" rid="F4">Figure 4</xref>. We refer to the &#x0201C;sum of the weights&#x0201D; as the &#x0201C;sum of the weights&#x00027; absolute values&#x0201D; in the remainder of this paper. Note that the weights in the middle and output layers are replaced by the products of the original weight and the sum of the neuron&#x00027;s weights in the previous layer during the time-domain process. We indicated the sum of the new reconfigured weights by <italic>B</italic> instead of the aforementioned &#x003B2;, which indicated the sum of the original weights, as follows:</p>
<disp-formula id="E38"><label>(38)</label><mml:math id="M79"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msubsup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mstyle displaystyle="true"><mml:munder class="msub"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder></mml:mstyle><mml:mo>||</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E39"><label>(39)</label><mml:math id="M80"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:msubsup><mml:mi>B</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>2</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:munder><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder><mml:mrow><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow></mml:mstyle><mml:mo>&#x02016;</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>2</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02016;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>&#x022EF;</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E40"><label>(40)</label><mml:math id="M81"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msubsup><mml:mrow><mml:mi>B</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mstyle displaystyle="true"><mml:munder class="msub"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder></mml:mstyle><mml:msubsup><mml:mrow><mml:mi>B</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<fig position="float" id="F4">
<label>Figure 4</label>
<caption><p>Summary of the general scheme of the time-domain weighted-sum neural network model.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0004.tif">
<alt-text>Diagram of a multi-layer neural network. Shows connections between neurons labeled as 1st, ith, jth, kth, and bias. Mathematical equations describe the interactions and weights, including terms like Bi(n-1), w_{ij}{(n)}, and t_{vj}{(n)}. A small inset graph illustrates a curve, highlighting changes in values. The jth neuron-s weighted-sum result equation is shown in a shaded box at the bottom.</alt-text></graphic>
</fig>
<p>Note the bias of neuron <italic>j</italic> in layer <italic>n</italic>, whose index <italic>i</italic> &#x0003D; 0, is indicated as <inline-formula><mml:math id="M82"><mml:msubsup><mml:mrow><mml:mi>B</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula>, in which <inline-formula><mml:math id="M83"><mml:msubsup><mml:mrow><mml:mi>B</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>&#x02261;</mml:mo><mml:mn>1</mml:mn></mml:math></inline-formula>. Therefore, the new weight connected from neuron <italic>i</italic> in layer <italic>n</italic>&#x02212;1 to neuron <italic>j</italic> in layer <italic>n</italic> becomes <inline-formula><mml:math id="M84"><mml:msubsup><mml:mrow><mml:mi>B</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula>. Then we can have the original weighted-sum result of the <italic>j</italic>&#x02212;th neuron in the <italic>n</italic>&#x02212;th layer indicated by <inline-formula><mml:math id="M85"><mml:msubsup><mml:mrow><mml:mi>y</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> expressed as follows:</p>
<disp-formula id="E41"><label>(41)</label><mml:math id="M86"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msubsup><mml:mrow><mml:mi>y</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:msubsup><mml:mrow><mml:mi>B</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
</sec>
<sec>
<title>3.3 Numerical simulations of neural networks</title>
<p>We performed numerical simulations to verify our weighted-sum calculation model. First, in order to verify our model for weighted-sum calculation with different-signed weights, we conducted a simulation to perform a weighted-sum calculation with 501 pairs of inputs and weights that consisted of 249 positive and 252 negative weights. We added a dummy weight to make the sum of positive weights equal to the absolute sum of the negative ones. <xref ref-type="fig" rid="F5">Figure 5</xref> shows the simulation results of the time-domain weighted-sum calculation with a dummy weight <italic>w</italic><sub><italic>n</italic>&#x0002B;1</sub>. The results show that the weighted summation can be calculated correctly with different negative and positive firing timing inputs, each set of which are multiplied by the corresponding signed weights.</p>
<fig position="float" id="F5">
<label>Figure 5</label>
<caption><p>Simulation results for the time-domain weighted-sum calculation model applied to the neuron shown in <xref ref-type="fig" rid="F2">Figure 2b</xref>: <bold>(a)</bold> PSP of positively weighted-sum operation with 249 inputs in which <inline-formula><mml:math id="M87"><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mn>1</mml:mn><mml:mo>,</mml:mo><mml:mi>&#x003BB;</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn><mml:mo>,</mml:mo><mml:mi>&#x003F5;</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn><mml:mo>.</mml:mo><mml:mn>01</mml:mn><mml:mo>,</mml:mo><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:mn>24</mml:mn><mml:mo>.</mml:mo><mml:mn>01</mml:mn></mml:math></inline-formula>, and &#x003B8;<sup>&#x0002B;</sup> &#x0003D; 24.25. The output spike timing is <inline-formula><mml:math id="M88"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mn>1</mml:mn><mml:mo>.</mml:mo><mml:mn>6356</mml:mn></mml:math></inline-formula>. <bold>(b)</bold> PSP of negatively weighted-sum operation with 253 inputs in which <inline-formula><mml:math id="M89"><mml:msub><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn></mml:mrow></mml:msub><mml:mo>=-</mml:mo><mml:mn>0</mml:mn><mml:mo>.</mml:mo><mml:mn>06</mml:mn><mml:mo>,</mml:mo><mml:msub><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>n</mml:mi><mml:mo>&#x0002B;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>=-</mml:mo><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>819</mml:mn><mml:mo>,</mml:mo><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mn>1</mml:mn><mml:mo>,</mml:mo><mml:mi>&#x003BB;</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn><mml:mo>,,</mml:mo><mml:mi>&#x003F5;</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn><mml:mo>.</mml:mo><mml:mn>01</mml:mn><mml:mo>,</mml:mo><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msup><mml:mo>=-</mml:mo><mml:mn>24</mml:mn><mml:mo>.</mml:mo><mml:mn>01</mml:mn></mml:math></inline-formula>, and &#x003B8;<sup>&#x02212;</sup> &#x0003D; &#x02212;24.25. The output spike timing is <inline-formula><mml:math id="M90"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mn>1</mml:mn><mml:mo>.</mml:mo><mml:mn>8321</mml:mn></mml:math></inline-formula>. Thus, the result of the weighted-sum calculation is <inline-formula><mml:math id="M91"><mml:mo>||</mml:mo><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x000B1;</mml:mo></mml:mrow></mml:msup><mml:mo>||</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>/</mml:mo><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mn>4</mml:mn><mml:mo>.</mml:mo><mml:mn>718</mml:mn></mml:math></inline-formula>.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0005.tif">
<alt-text>Graph (a) shows an exponential increase of V(t), transitioning to a linear increase at tv. Graph (b) shows a similar pattern inverted, with an exponential decrease transitioning to linear. Both graphs delineate time intervals Tin and tv, and thresholds \theta, tvmax, and tvmin.</alt-text>
</graphic>
</fig>
<p>Then, we applied our model to a four-layer MLP (784-100-100-100-10) to classify the MNIST digit character set. We trained the MLP and then performed inference according to <xref ref-type="disp-formula" rid="E41">Equation 41</xref> with the obtained weights, which were either binary (<xref ref-type="bibr" rid="B17">Courbariaux et al., 2015</xref>) or floating-point values. As described above, output spike timings at each neuron in the previous layer were directly conveyed to the neurons in the next layer without obtaining weighted-sum results in the middle layers. We found that we obtained the same weighted-sum calculation results in the last layer and also the same recognition precisions in both NNs as in the numerically calculated ones.</p></sec></sec>
<sec id="s4">
<title>4 Issues about time-domain weighted-sum models toward VLSI implementation</title>
<p>We have established our time-domain weighted-sum neural network model in a general form and summarized it in <xref ref-type="fig" rid="F4">Figure 4</xref>, and conducted numerical simulations that verified the effectiveness in pre-trained ANN models in Section 3. In this section, we will discuss some issues about the model when implemented in analog VLSI circuits.</p>
<sec>
<title>4.1 Weights and biases</title>
<p>In the general time-domain weighted-sum neural network model as shown in <xref ref-type="fig" rid="F4">Figure 4</xref> and <xref ref-type="disp-formula" rid="E41">Equation 41</xref>, the weights must be reconfigured as <inline-formula><mml:math id="M92"><mml:msubsup><mml:mrow><mml:mi>B</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> in order to generate two timings whose interval is proportional to the original weighted-sum result, which results in the same recognition accuracy as the original ANN. The reconfigured weights correspond to the PSP slope in the IF neuron model. The slope will be greatly increased with the reconfiguration, which may result in very high potentials that do not satisfy the hardware system criteria. To solve this problem, we introduced a scaling factor &#x00393;<sup>(<italic>n</italic>)</sup> for the <italic>n</italic>-th layer as shown in <xref ref-type="fig" rid="F6">Figure 6</xref>, to adjust the PSP slope to a reasonable level. Note that every neuron in the same layer has the same scaling factor.</p>
<fig position="float" id="F6">
<label>Figure 6</label>
<caption><p>Weights scaling: a scaling factor &#x00393;<sup>(<italic>n</italic>)</sup>, expressed in <xref ref-type="disp-formula" rid="E52">Equation 52</xref>, is introduced for the <italic>n</italic>-th layer to adjust the reconfigured large PSP slope shown in <xref ref-type="fig" rid="F4">Figure 4</xref> to a reasonable level. After scaling, the slopes are expressed as <inline-formula><mml:math id="M93"><mml:mi>B</mml:mi><mml:msubsup><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>/</mml:mo><mml:msup><mml:mrow><mml:mi>&#x00393;</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msup></mml:math></inline-formula>, where <inline-formula><mml:math id="M94"><mml:mi>B</mml:mi><mml:msubsup><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> represents the scaled sum of weights in the previous layer <italic>n</italic>&#x02212;1 and is scaled by &#x00393;<sup>(<italic>n</italic>&#x02212;1)</sup>.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0006.tif">
<alt-text>Diagram illustrating a neural network layer interaction. It shows the relationship between layers n&#x02212;1 and n, with nodes labeled ith, jth, and 0th. Equations describe weight (wij), bias (bj(n)), and signal transformations (Bsi(n&#x02212;1), tvij). A boxed equation shows the output yj(n). Connections represent weighted relationships and processing paths, with variables represented in different colors for context.</alt-text>
</graphic>
</fig>
<p>Then the reconfigured weight becomes <inline-formula><mml:math id="M95"><mml:mi>B</mml:mi><mml:msubsup><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>/</mml:mo><mml:msup><mml:mrow><mml:mi>&#x00393;</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msup></mml:math></inline-formula>, where <inline-formula><mml:math id="M96"><mml:mi>B</mml:mi><mml:msubsup><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> represents the scaled sum of weights in the previous layer <italic>n</italic>&#x02212;1, and the scaled sum of the reconfigured weights in layer <italic>n</italic>, which is also interpreted as the total PSP slope, is expressed as</p>
<disp-formula id="E42"><label>(42)</label><mml:math id="M97"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>B</mml:mi><mml:msubsup><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mfrac><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msup><mml:mrow><mml:mi>&#x00393;</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msup></mml:mrow></mml:mfrac><mml:mstyle displaystyle="true"><mml:munder class="msub"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder></mml:mstyle><mml:mo>||</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E43"><label>(43)</label><mml:math id="M98"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mi>B</mml:mi><mml:msubsup><mml:mi>s</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>2</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:msup><mml:mi>&#x00393;</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>2</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:mfrac><mml:mstyle displaystyle='true'><mml:munder><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder><mml:mi>B</mml:mi></mml:mstyle><mml:msubsup><mml:mi>s</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02016;</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>2</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02016;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>&#x022EF;</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E44"><label>(44)</label><mml:math id="M99"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>B</mml:mi><mml:msubsup><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mfrac><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msup><mml:mrow><mml:mi>&#x00393;</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msup></mml:mrow></mml:mfrac><mml:mstyle displaystyle="true"><mml:munder class="msub"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder></mml:mstyle><mml:mi>B</mml:mi><mml:msubsup><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>so we can obtain</p>
<disp-formula id="E45"><label>(45)</label><mml:math id="M100"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>B</mml:mi><mml:msubsup><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mfrac><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msup><mml:mrow><mml:mi>&#x00393;</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msup></mml:mrow></mml:mfrac><mml:msubsup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E46"><label>(46)</label><mml:math id="M101"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mi>B</mml:mi><mml:msubsup><mml:mi>s</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>2</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:mstyle displaystyle='true'><mml:msubsup><mml:mo>&#x0220F;</mml:mo><mml:mrow><mml:mi>l</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>2</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mrow><mml:msup><mml:mi>&#x00393;</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>l</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:mstyle></mml:mrow></mml:mfrac><mml:msubsup><mml:mi>B</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>2</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>&#x022EF;</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E47"><label>(47)</label><mml:math id="M102"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>B</mml:mi><mml:msubsup><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mfrac><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mstyle displaystyle="true"><mml:msubsup><mml:mrow><mml:mo>&#x0220F;</mml:mo></mml:mrow><mml:mrow><mml:mi>l</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mstyle><mml:msup><mml:mrow><mml:mi>&#x00393;</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>l</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msup></mml:mrow></mml:mfrac><mml:msubsup><mml:mrow><mml:mi>B</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Note that the bias <inline-formula><mml:math id="M103"><mml:msubsup><mml:mrow><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> is reconfigured as</p>
<disp-formula id="E48"><label>(48)</label><mml:math id="M104"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msubsup><mml:mrow><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mi>B</mml:mi><mml:msubsup><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>/</mml:mo><mml:msup><mml:mrow><mml:mi>&#x00393;</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msup><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>where <inline-formula><mml:math id="M105"><mml:mi>B</mml:mi><mml:msubsup><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msubsup><mml:mrow><mml:mo>&#x0220F;</mml:mo></mml:mrow><mml:mrow><mml:mi>l</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:msup><mml:mrow><mml:mi>&#x00393;</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>l</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msup></mml:mrow></mml:mfrac></mml:math></inline-formula>. Accordingly, the original weighted-sum result is expressed as</p>
<disp-formula id="E49"><label>(49)</label><mml:math id="M106"><mml:mrow><mml:msubsup><mml:mi>y</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:munderover><mml:mo>&#x0220F;</mml:mo><mml:mrow><mml:mi>l</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:munderover><mml:mrow><mml:msup><mml:mi>&#x00393;</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>l</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:mstyle><mml:mfrac><mml:mrow><mml:mi>B</mml:mi><mml:msubsup><mml:mi>s</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:msub><mml:mi>T</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:msubsup><mml:mi>B</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:msub><mml:mi>T</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mo stretchy='false'>(</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>t</mml:mi><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>+</mml:mo></mml:mrow></mml:msubsup><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:math></disp-formula>
<p>From <xref ref-type="disp-formula" rid="E49">Equations 49</xref>, <xref ref-type="disp-formula" rid="E41">41</xref>, we can find that the difference between timing <inline-formula><mml:math id="M107"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M108"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> remains the same before and after the scaling operations.</p>
<p>So far we have shown the general scaling process toward the weights&#x00027; reconfiguration. Next, we will show some special cases that can simplify the weights&#x00027; reconfiguration. Suppose that</p>
<disp-formula id="E50"><label>(50)</label><mml:math id="M109"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mstyle displaystyle="true"><mml:munder class="msub"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>&#x02260;</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder></mml:mstyle><mml:mo>||</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||</mml:mo></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mstyle displaystyle="true"><mml:munder class="msub"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>&#x02260;</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder></mml:mstyle><mml:mo>||</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mn>2</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||=&#x022EF;=</mml:mo><mml:mstyle displaystyle="true"><mml:munder class="msub"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>&#x02260;</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder></mml:mstyle><mml:mo>||</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E51"><label>(51)</label><mml:math id="M110"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mo>||</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>01</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||</mml:mo></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mo>||</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>02</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||=&#x022EF;=||</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>meaning that the bias and the sum of the original weights of every neuron in layer <italic>n</italic> are equal to each other, such as in the BinaryConnect NN model (<xref ref-type="bibr" rid="B17">Courbariaux et al., 2015</xref>), whose weights and biases are binary values. Let the scaling factor &#x00393;<sup>(<italic>n</italic>)</sup> be</p>
<disp-formula id="E52"><label>(52)</label><mml:math id="M111"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:msup><mml:mi>&#x00393;</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>,</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msup><mml:mi>&#x00393;</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>2</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>,</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>&#x022EF;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msup><mml:mi>&#x00393;</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>where <inline-formula><mml:math id="M112"><mml:msubsup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>l</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> denotes the sum of the weights of the layer <italic>l</italic>(&#x0003D; 0, 1, 2&#x022EF;<italic>n</italic>) expressed as follows:</p>
<disp-formula id="E53"><label>(53)</label><mml:math id="M113"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mn>1</mml:mn><mml:mo>,</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:munder><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>&#x02260;</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder><mml:mrow><mml:mo>&#x02016;</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02016;</mml:mo></mml:mrow></mml:mstyle><mml:mo>+</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mrow><mml:mi>j</mml:mi><mml:mo>&#x02260;</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow></mml:mfrac><mml:mo>&#x02016;</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mn>0</mml:mn><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02016;</mml:mo><mml:mo>,</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>2</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:munder><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>&#x02260;</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder><mml:mrow><mml:mo>&#x02016;</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>2</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02016;</mml:mo></mml:mrow></mml:mstyle><mml:mo>+</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mrow><mml:mi>j</mml:mi><mml:mo>&#x02260;</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow></mml:mfrac><mml:mo>&#x02016;</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mn>0</mml:mn><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mn>2</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02016;</mml:mo><mml:mo>,</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>&#x022EF;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mi>j</mml:mi><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mstyle displaystyle='true'><mml:munder><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>&#x02260;</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:munder><mml:mrow><mml:mo>&#x02016;</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02016;</mml:mo></mml:mrow></mml:mstyle><mml:mo>+</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:msubsup><mml:mi>&#x003B2;</mml:mi><mml:mrow><mml:mi>j</mml:mi><mml:mo>&#x02260;</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup></mml:mrow></mml:mfrac><mml:mo>&#x02016;</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mrow><mml:mn>0</mml:mn><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy='false'>(</mml:mo><mml:mi>n</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x02016;</mml:mo><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>where <inline-formula><mml:math id="M114"><mml:msubsup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:math></inline-formula>. Note that <inline-formula><mml:math id="M115"><mml:msubsup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:msubsup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>=&#x022EF;=</mml:mo><mml:msubsup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> under the assumption of <xref ref-type="disp-formula" rid="E51">Equations 51</xref>, <xref ref-type="disp-formula" rid="E52">52</xref>. Then we can generate the desired timings using only the original weights <inline-formula><mml:math id="M116"><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>,</mml:mo><mml:mi>i</mml:mi><mml:mo>&#x02260;</mml:mo><mml:mn>0</mml:mn></mml:math></inline-formula> shown in <xref ref-type="fig" rid="F7">Figure 7a</xref>, without reconfiguring the weights (not biases included) as <inline-formula><mml:math id="M117"><mml:msubsup><mml:mrow><mml:mi>B</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>,</mml:mo><mml:mi>i</mml:mi><mml:mo>&#x02260;</mml:mo><mml:mn>0</mml:mn></mml:math></inline-formula> shown in <xref ref-type="fig" rid="F4">Figure 4</xref>. However, the bias <inline-formula><mml:math id="M118"><mml:msubsup><mml:mrow><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula>must be reconfigured as</p>
<disp-formula id="E54"><label>(54)</label><mml:math id="M119"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msubsup><mml:mrow><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msubsup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mrow></mml:mfrac><mml:mo>&#x000B7;</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<fig position="float" id="F7">
<label>Figure 7</label>
<caption><p>Derivations from the general time-domain weighted-sum process in MLPs whose weights (biases not included) involved in the time-domain process, i.e., <inline-formula><mml:math id="M120"><mml:msubsup><mml:mrow><mml:mi>B</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>,</mml:mo><mml:mi>i</mml:mi><mml:mo>&#x02260;</mml:mo><mml:mn>0</mml:mn></mml:math></inline-formula>, are scaled to the original weights, <inline-formula><mml:math id="M121"><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>,</mml:mo><mml:mi>i</mml:mi><mml:mo>&#x02260;</mml:mo><mml:mn>0</mml:mn></mml:math></inline-formula>: <bold>(a)</bold> the models with biases: the biases are scaled as <inline-formula><mml:math id="M122"><mml:mfrac><mml:mrow><mml:msubsup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:msubsup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>&#x02260;</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mrow></mml:mfrac><mml:mo>&#x000B7;</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mn>0</mml:mn><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> under the assumption of <xref ref-type="disp-formula" rid="E50">Equations 50</xref>, <xref ref-type="disp-formula" rid="E51">51</xref> accompanying the weights&#x00027; scaling operation; <bold>(b)</bold> the models without biases: we add an extra dummy weight <inline-formula><mml:math id="M123"><mml:msubsup><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msup><mml:mo>-</mml:mo><mml:munder class="msub"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:munder><mml:mo>||</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||</mml:mo></mml:math></inline-formula> whose two input timings are the same, i.e., the input is 0, for neuron <italic>j</italic> in layer <italic>n</italic> to make the sum of weights of every neuron in layer <italic>n</italic> identical, which is marked as &#x003B2;<sup>(<italic>n</italic>)</sup>.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0007.tif">
<alt-text>Diagrams (a) and (b) illustrate neural network architectures with layers labeled as (n&#x02212;1)-th, n-th, and (n&#x0002B;1)-th. Neurons and connections with weights are depicted. Diagram (a) shows connections with a focus on biases and weighted sums, while diagram (b) includes dummy weights and modified weight calculations, highlighting dj(n) adjustments. Equations and annotations explain neuron computations, targeting neural network and algorithm functions.</alt-text>
</graphic>
</fig>
<p>Accordingly, the original weighted-sum result will be</p>
<disp-formula id="E55"><label>(55)</label><mml:math id="M124"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msubsup><mml:mrow><mml:mi>y</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mstyle displaystyle="true"><mml:msubsup><mml:mrow><mml:mo>&#x0220F;</mml:mo></mml:mrow><mml:mrow><mml:mi>l</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:mstyle><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>l</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msup></mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>where &#x003B2;<sup>(<italic>l</italic>)</sup> denotes the identical value of the sum of weights among neurons in layer <italic>l</italic>.</p>
<p>In modern deep neural networks with deep layers and a large number of parameters, several experiments using models without bias demonstrated that there was an accuracy degradation of 3.9% and 4% in CIFAR10 and CIFAR100 datasets, respectively (<xref ref-type="bibr" rid="B72">Wang et al., 2019</xref>). Such degradation of less than 5% is supposed to be acceptable when deploying DNNs to resource-constrained edge devices, in which trade-offs between accuracy, latency, and energy efficiency need to be carefully considered (<xref ref-type="bibr" rid="B62">Shuvo et al., 2022</xref>; <xref ref-type="bibr" rid="B53">Ngo et al., 2025</xref>).</p>
<p>We also trained a four-layer MLP (784-100-100-100-10) with and without biases on the datasets MNIST and Fashion-MNIST and compared the results in both the floating-point and binary weight connect models, as shown in <xref ref-type="fig" rid="F8">Figure 8</xref>. The results showed that the accuracies with and without biases were comparable. Therefore, in certain cases, the bias can be removed so that the reconfiguration cost of the bias shown in <xref ref-type="disp-formula" rid="E48">Equations 48</xref>, <xref ref-type="disp-formula" rid="E54">54</xref> is saved.</p>
<fig position="float" id="F8">
<label>Figure 8</label>
<caption><p>The MLP recognition accuracy comparison between models with and without bias on <bold>(a)</bold> MNIST dataset and <bold>(b)</bold> Fashion-MNIST dateset. The MLP models are the BinaryConnect model and the general floating-point connection model.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0008.tif">
<alt-text>Two line graphs comparing accuracy over epochs for different configurations: (a) shows minimal differences between four configurations, with all nearing 95% accuracy. (b) shows more variation with accuracy around 80-90%, with floating-point weights generally performing better than BinaryConnect configurations.</alt-text>
</graphic>
</fig>
<p>We have shown a case in which the weights and biases were restricted to the condition shown in <xref ref-type="disp-formula" rid="E50">Equations 50</xref>, <xref ref-type="disp-formula" rid="E51">51</xref> so that the cost of the weights&#x00027; reconfiguration can be saved. Next, we propose a method to satisfy the restriction in <xref ref-type="disp-formula" rid="E50">Equation 50</xref> for a more general ANN model to save the weights&#x00027; reconfiguration shown in <xref ref-type="fig" rid="F7">Figure 7b</xref>. Note that we discuss the method in the model without biases for simplicity. We add a dummy weight to every neuron in layer <italic>n</italic> to make the sum of the weights identical. We regard the identical value in layer <italic>n</italic> as &#x003B2;<sup>(<italic>n</italic>)</sup>. The dummy weight to neuron <italic>j</italic> in layer <italic>n</italic>, indicated as <inline-formula><mml:math id="M125"><mml:msubsup><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula>, is allocated as:</p>
<disp-formula id="E56"><label>(56)</label><mml:math id="M126"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msubsup><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mi>&#x003B2;</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msup><mml:mo>-</mml:mo><mml:mstyle displaystyle="true"><mml:munder class="msub"><mml:mrow><mml:mo>&#x02211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:munder></mml:mstyle><mml:mo>||</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup><mml:mo>||</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Note that the input timings for the dummy weights <inline-formula><mml:math id="M127"><mml:msubsup><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M128"><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> are identical, such as <italic>T</italic><sub><italic>in</italic></sub>, meaning a 0 input.</p></sec>
<sec>
<title>4.2 Output timing difference</title>
<p>From <xref ref-type="disp-formula" rid="E40">Equations 40</xref>, <xref ref-type="disp-formula" rid="E41">41</xref>, we can find that the coefficients applied to the output timing difference are increased monotonically as the layer goes deeper. It has generally been observed that the outputs of every neuron, i.e., the weighted-sum results, converge at a certain range in a well-trained ANN. Therefore, we figure out that the timing difference decreases monotonically as the layer goes deeper. This effect essentially results from our calculating the positively signed and negatively signed weighted sums separately.</p>
<p>We demonstrate the timing difference issue by means of a case study in which we perform the time domain weighted-sum inference in a well-trained four-layer MLP (784-100-100-100-10). We collected the distributions of the output timing differences in every layer of the MLP when performing evaluation on the 10,000-sample test data in MNIST. <xref ref-type="fig" rid="F9">Figure 9a</xref> shows the distributions where the histograms show only 100 of the total 10,000 samples, but the standard deviations &#x003C3; are calculated over the total samples. The output timing differences, &#x003C3;, of the 1st, 2nd, 3rd and 4th layers are 5.89e &#x02212; 8 s, 5.91e &#x02212; 9 s, 8.42e &#x02212; 10 s, and 1.08e &#x02212; 10 s, respectively, under the assumption of <italic>T</italic><sub><italic>in</italic></sub> &#x0003D; 1 &#x003BC;s. If we assume that the resolution time step is 10 ns by taking the noise of analog circuits into account, the great majority of the timing differences in the 2nd and the subsequent layers are less than the time resolution. Therefore, such a time-domain multi-layer model cannot be implemented into analog VLSI directly. We also conducted an experiment to evaluate the noise tolerance of the above model. In the experiment, we injected noise with different standard deviations &#x003C3; to the output timing <inline-formula><mml:math id="M129"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M130"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula>, and evaluated the MLP recognition accuracy. The results are shown in <xref ref-type="fig" rid="F10">Figure 10a</xref>. We can find that the accuracy deteriorates when the noise level is around 0.5 ns near the distribution &#x003C3; of the 3rd layer. The model does not work when &#x003C3; is over 5 ns near that of the 2nd layer.</p>
<fig position="float" id="F9">
<label>Figure 9</label>
<caption><p>The distribution of the output timing difference, <inline-formula><mml:math id="M131"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula>, in every layer of the 784-100-100-100-10 four-layer MLP model <italic>T</italic><sub><italic>in</italic></sub> is assumed to be 1 &#x003BC;s: <bold>(a)</bold> the original distributions; <bold>(b)</bold> the distributions in which the timing differences are amplified with a gain of 10.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0009.tif">
<alt-text>Comparison of histograms (a) and (b) showing the distribution of (tv*-tv&#x0002B;) for four layers. Graphs (a) have narrower peaks and lower standard deviations compared to graphs (b). Standard deviations for layer one are 5.89 &#x000D7; 10&#x02212;8 seconds in both (a) and (b), with (b) showing wider distribution. Layer two shows 5.91 &#x000D7; 10&#x02212;9 seconds in (a) and 5.94 &#x000D7; 10&#x02212;8 seconds in (b). Layer three shows 8.42 &#x000D7; 10&#x02212;10 seconds in (a) and 8.30 &#x000D7; 10&#x02212;8 seconds in (b). Layer four shows 1.08 &#x000D7; 10&#x02212;10 seconds in (a) and 1.07 &#x000D7; 10&#x02212;7 seconds in (b).</alt-text>
</graphic>
</fig>
<fig position="float" id="F10">
<label>Figure 10</label>
<caption><p>Noise tolerance examinations in the four-layer MLP: <bold>(a)</bold> recognition accuracy of the model without timing difference amplification evaluated under different noise levels; <bold>(b)</bold> recognition accuracy under the different amplification gain conditions with the critical noise injected; <bold>(c)</bold> the output timing difference distributions in every layer under different amplifying gain conditions.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0010.tif">
<alt-text>Three graphs are presented: (a) shows accuracy versus noise, where accuracy decreases as noise increases; (b) compares accuracy against gain for different noise levels, showing varying trends based on noise; (c) depicts a relation between time difference, noise, and gain across four layers, each represented by a different color, indicating an increase with gain.</alt-text>
</graphic>
</fig>
<p>In order to solve the problem of decreasing output timing difference, we can introduce an amplification component into our model, which can amplify the timing difference just before the timings are transferred to the next layer. To examine the effectiveness of this amplification function, we performed experiments in which we amplified the timing difference with different gains and evaluated the recognition accuracy with the critical noise injected. The results are shown in <xref ref-type="fig" rid="F10">Figure 10b</xref>. We also plotted the output timing difference distributions in every layer under different amplifying gain conditions. <xref ref-type="fig" rid="F10">Figure 10c</xref> shows the distribution standard deviations, and <xref ref-type="fig" rid="F9">Figure 9b</xref> shows the histograms with a gain of 10. Note that in these experiments, we simply set the same amplification gains in every layer without optimizing the gains. We found that the recognition accuracy is comparable to that in the model without noise injected if we select a gain to make the output timing difference distribution &#x003C3; of the last layer larger than the critical noise level, such as 10 ns. However, for more robustness, the distribution &#x003C3; is supposed to be much larger than the resolution time step so that the gain can be 8&#x02013;10. We verified the effectiveness of the amplification function and established the time-domain weighted-sum model with amplification components. In VLSI circuits, we can introduce a time-difference-amplifier (TDA) (<xref ref-type="bibr" rid="B1">Abas et al., 2002</xref>; <xref ref-type="bibr" rid="B7">Asada et al., 2018</xref>) component to amplify the output timing difference.</p></sec></sec>
<sec id="s5">
<title>5 Circuits and architectures for TACT-based neural networks</title>
<p>As a VLSI implementation of our time-domain weighted-sum calculation based on the TACT approach, we propose an RC circuit in which a capacitor is connected by multiple resistors, as shown in <xref ref-type="fig" rid="F11">Figure 11a</xref>. Theoretical estimations have indicated that this circuit can perform weighted-sum calculations with extremely low energy consumption (<xref ref-type="bibr" rid="B64">Tohara et al., 2016</xref>; <xref ref-type="bibr" rid="B70">Wang et al., 2016</xref>).</p>
<fig position="float" id="F11">
<label>Figure 11</label>
<caption><p>Synapse circuit: <bold>(a)</bold> step voltage input and a resistance-capacitance (R-C) circuit in which a pMOSFET acts as resistance <italic>R</italic>, and parasitic capacitance of interconnection and the gate capacitance of MOSFETs act as <italic>C</italic> in a VLSI circuit; <bold>(b)</bold> approximately linear response of the step voltage input at timing <italic>t</italic><sub><italic>i</italic></sub> with a slope determined by gate voltage <italic>V</italic><sub><italic>ki</italic></sub>. <bold>(c)</bold> an operating example for explanation of the pMOSFET synapse&#x00027;s rectification function.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0011.tif">
<alt-text>Diagram illustrating an synapse circuit. Part (a) shows a resistor ladder with three input voltages and a capacitor grounded. Part (b) displays a MOSFET connected to a capacitor, with a timing diagram for input voltage and capacitor voltage. Part (c) depicts a switch configuration with different voltage states and a table indicating threshold voltage, Vth, Vki, and theta values. Text explains that switch S2 is off due to a positive gate-source voltage.</alt-text>
</graphic>
</fig>
<p>In CMOS VLSI implementation, resistance <italic>R</italic> can be replaced by a p-type MOS field-effect transistor (pMOSFET), as shown in <xref ref-type="fig" rid="F11">Figure 11b</xref>. The approximately linear slope <italic>k</italic> is generated by capacitance <italic>C</italic> and ON resistance of a pMOSFET with a step voltage input <italic>V</italic><sub><italic>in</italic></sub>, where we use step voltages instead of spike pulses as inputs. Each resistance should have a rectification function to prevent an inverse current. The rectification function is automatically realized by the FET operation as follows. When a pMOSFET receives a step-voltage input, the terminal voltage of the input is higher than that at <italic>C</italic>, and therefore, the input-side terminal of the pMOSFET is the &#x0201C;source,&#x0201D; and the capacitor-side terminal is the &#x0201C;drain.&#x0201D; In this state, if the gate-source voltage (<italic>V</italic><sub><italic>gs</italic></sub>) of the pMOSFET is set to exceed its threshold voltage, the pMOSFET turns on, and <italic>C</italic> is charged up. On the other hand, when a pMOSFET receives no input, the terminal voltage of the input is lower than that at <italic>C</italic>, and therefore the source-drain position in the pMOSFET is reversed; i.e., the input-side terminal of the pMOSFET is &#x0201C;drain,&#x0201D; and the capacitor-side terminal is &#x0201C;source.&#x0201D; In this state, if the <italic>V</italic><sub><italic>gs</italic></sub> of the pMOSFET is set not to exceed its threshold voltage, the pMOSFET turns off, and the charges stored at <italic>C</italic> do not flow back to the input side. An operating example is shown in <xref ref-type="fig" rid="F11">Figure 11c</xref>, in which the synapse pMOSFET without input (i.e., the input voltage is 0V here), denoted as S2, is strongly off because its gate-source voltage is positive.</p>
<sec>
<title>5.1 Architectures</title>
<p>We propose a circuit architecture of a neural network based on our established weighted-sum calculation model which is suitable for our TACT approach, accommodating both positive and negative weights. The architecture is shown in <xref ref-type="fig" rid="F12">Figure 12</xref> and is composed of a crossbar synapse array acting as resistive elements, the neuron part functioning as thresholding and nonlinear activation, and the configuration part controlling synapses.</p>
<fig position="float" id="F12">
<label>Figure 12</label>
<caption><p>TACT-based MLP architecture. <bold>(a)</bold> A two-layer MLP architecture described in <xref ref-type="fig" rid="F3">Figure 3</xref> whose input layer is modeled in <xref ref-type="fig" rid="F2">Figure 2c</xref> in which each synapse has two set of inputs and weights. <bold>(b)</bold> Another type of input layer architecture of the MLP that there is one input and one weight for each synapse while a row of dummy cells with a dummy input are added.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0012.tif">
<alt-text>Diagram showing a neuromorphic computing architecture. Panel A illustrates weight control using positive and negative dendrite lines, neurons, ReLU activation, and post-processing blocks. Panel B displays a similar setup with a focus on dummy synapse cells. Both panels include weight conditions for neuron connections and axon lines.</alt-text>
</graphic>
</fig>
<p><xref ref-type="fig" rid="F12">Figure 12a</xref> shows a two-layer MLP architecture described in <xref ref-type="fig" rid="F3">Figure 3</xref> whose input layer is modeled in <xref ref-type="fig" rid="F2">Figure 2c</xref> in which each synapse has two sets of inputs and weights. Another type of input layer architecture of the MLP is shown in <xref ref-type="fig" rid="F12">Figure 12b</xref>, as described in <xref ref-type="fig" rid="F2">Figure 2b</xref>.</p>
<p>In <xref ref-type="fig" rid="F12">Figure 12a</xref>, there are two inputs for each synapse circuit, which are <italic>t</italic><sub><italic>i</italic></sub> as the signal input and <italic>t</italic><sub><italic>dmy</italic></sub> as a dummy input in the first layer, and <inline-formula><mml:math id="M132"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M133"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> in the subsequent layers. Pairs of positive and negative timings are directly connected to the next layer without subtracting the negatively signed weighted results from the positively signed weighted results according to the theory explained in Section 3.2 and <xref ref-type="fig" rid="F4">Figure 4</xref>. In the synapse array, the horizontal and vertical lines are referred to as &#x0201C;axons&#x0201D; of the previous neurons and &#x0201C;dendrites&#x0201D; of the post neurons, respectively. Suppose that each axon line has <italic>M</italic> synapse circuits, and each dendrite line receives <italic>N</italic> synapse outputs. A synapse cell is designed with two resistive elements and two pairs of switches. A set of two identical resistances represents the weight value. The resistive elements are expected to be replaced by resistance-based analog memories to store the multi-bit weights (<xref ref-type="bibr" rid="B58">Sebastian et al., 2020</xref>). We can assume that the upper-side axon is for <inline-formula><mml:math id="M134"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and the other is for <inline-formula><mml:math id="M135"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula>, and the left-side dendrite is for a positive weight connection while the other is for a negative one. The two switches are exclusively controlled according to the corresponding sign of weights, which is controlled by the weight control circuit. By contrast, there is one input and one weight for each synapse in the input layer shown in <xref ref-type="fig" rid="F12">Figure 12b</xref>, while a row of dummy cells with a dummy input is added to the synapse array conceptually according to Section 3.1. The resistances (<italic>d</italic><sub><italic>i</italic></sub>) of the dummy cell are theoretically set according to <xref ref-type="disp-formula" rid="E56">Equation 56</xref>.</p>
<p>In AIMC, the most common implementation of a signed weight <italic>w</italic><sub><italic>i</italic></sub> is using a differential scheme with two subweights <inline-formula><mml:math id="M136"><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and <inline-formula><mml:math id="M137"><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> such that</p>
<disp-formula id="E57"><label>(57)</label><mml:math id="M138"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>in which one is for positive weighted-sum and another is for negative one (<xref ref-type="bibr" rid="B74">Xiao et al., 2023</xref>; <xref ref-type="bibr" rid="B2">Aguirre et al., 2024</xref>). Here, we treat the following signed weight configuration as a special differential scheme (<xref ref-type="bibr" rid="B75">Yamaguchi et al., 2020</xref>; <xref ref-type="bibr" rid="B34">Kingra et al., 2022</xref>):</p>
<disp-formula id="E58"><label>(58)</label><mml:math id="M139"><mml:mrow><mml:msub><mml:mi>w</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:mrow><mml:mtable columnalign='left'><mml:mtr columnalign='left'><mml:mtd columnalign='left'><mml:mrow><mml:msubsup><mml:mi>w</mml:mi><mml:mi>i</mml:mi><mml:mo>+</mml:mo></mml:msubsup><mml:mo>&#x02212;</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:mtd><mml:mtd columnalign='left'><mml:mrow><mml:mtext>where&#x000A0;</mml:mtext><mml:msubsup><mml:mi>w</mml:mi><mml:mi>i</mml:mi><mml:mo>&#x02212;</mml:mo></mml:msubsup><mml:mtext>&#x000A0;is&#x000A0;disabled,&#x000A0;indicating&#x000A0;</mml:mtext><mml:msub><mml:mi>w</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>&#x02265;</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr columnalign='left'><mml:mtd columnalign='left'><mml:mrow><mml:mn>0</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msubsup><mml:mi>w</mml:mi><mml:mi>i</mml:mi><mml:mo>&#x02212;</mml:mo></mml:msubsup></mml:mrow></mml:mtd><mml:mtd columnalign='left'><mml:mrow><mml:mtext>where&#x000A0;</mml:mtext><mml:msubsup><mml:mi>w</mml:mi><mml:mi>i</mml:mi><mml:mo>+</mml:mo></mml:msubsup><mml:mtext>&#x000A0;is&#x000A0;disabled,&#x000A0;indicating&#x000A0;</mml:mtext><mml:msub><mml:mi>w</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>&#x02264;</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:mrow><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula>
<p>Subtraction to obtain the final weighted-sum result is commonly performed in either differential mode or common mode. In differential mode, the operation is carried out at two separate nodes within the peripheral circuitry (<xref ref-type="bibr" rid="B24">Guo et al., 2017</xref>; <xref ref-type="bibr" rid="B32">Joshi et al., 2020</xref>; <xref ref-type="bibr" rid="B75">Yamaguchi et al., 2020</xref>; <xref ref-type="bibr" rid="B56">Sahay et al., 2020</xref>). In common mode, the subtraction is performed at a single node based on Kirchhoff&#x00027;s law, using either bipolar (<xref ref-type="bibr" rid="B68">Wan et al., 2022</xref>; <xref ref-type="bibr" rid="B2">Aguirre et al., 2024</xref>) or unipolar inputs (<xref ref-type="bibr" rid="B69">Wang et al., 2021</xref>; <xref ref-type="bibr" rid="B33">Khaddam-Aljameh et al., 2022</xref>; <xref ref-type="bibr" rid="B38">Le Gallo et al., 2023</xref>). It&#x00027;s worth noting that common-mode subtraction with unipolar inputs generally requires a bi-directional peripheral circuit, capable of providing two voltages: one higher and one lower than the common node voltage.</p>
<p>Signed inputs for 4-quadrant computation can be implemented by applying opposite polarity voltage (<xref ref-type="bibr" rid="B46">Marinella et al., 2018</xref>; <xref ref-type="bibr" rid="B37">Le Gallo et al., 2024</xref>) or using differential pairs when the input is unipolar (<xref ref-type="bibr" rid="B57">Schlottmann and Hasler, 2011</xref>; <xref ref-type="bibr" rid="B8">Bavandpour et al., 2019a</xref>). Additionally, signed computations without adopting the above two designs need multiple phase modulations, like two phases in <xref ref-type="bibr" rid="B34">Kingra et al. (2022)</xref> for 2-quadrant MAC and four phases in <xref ref-type="bibr" rid="B38">Le Gallo et al. (2023)</xref> for 4-quadrant MAC.</p>
<p>With respect to the signed weight representation in our approaches, the configuration in <xref ref-type="fig" rid="F12">Figure 12b</xref> is regarded as the special differential scheme described in the expression <xref ref-type="disp-formula" rid="E58">Equation 58</xref>, and that in <xref ref-type="fig" rid="F12">Figure 12a</xref> restricts the two subweights to be identical. We term the latter configuration as a <italic>complementary scheme</italic>, distinguishing it from the general differential scheme. With respect to the signed input representation, we adopt differential pairs as in <xref ref-type="bibr" rid="B8">Bavandpour et al. (2019a)</xref>. Our model with the complementary scheme can perform four-quadrant MAC computation in a single modulation without bipolar or bi-directional peripheral requirements.</p>
<p>The neuron part shown in <xref ref-type="fig" rid="F12">Figure 12a</xref> consists of a thresholding block, such as a comparator, a ReLU block, and a post-processing block (PPB) after the ReLU block. ReLU block with input and output timings is shown in <xref ref-type="fig" rid="F13">Figure 13a</xref>. The relationships between inputs and outputs are illustrated in <xref ref-type="fig" rid="F13">Figure 13b</xref>, in which both output timings are set identical to <inline-formula><mml:math id="M140"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> when <inline-formula><mml:math id="M141"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mo>&#x0003E;</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula>. The truth table is shown in <xref ref-type="fig" rid="F13">Figure 13c</xref> and accordingly the ReLU activation function can easily be implemented by logic gates, as shown in <xref ref-type="fig" rid="F13">Figure 13d</xref>. With such circuits, the nonlinear activation function ReLU can be implemented with low energy consumption operation. The PPB can be either a TDA circuit or a set of TDC and DTC to address the issue of timing difference shrinkage discussed in Section 4.2. The TDA is introduced to transmit the timings to the next layer directly in an analog manner, and the TDC and DTC are introduced to communicate intra-layers digitally. We leave the PPB implementation with high performance, such as high precision and low power, to be an open design problem in this paper.</p>

<fig position="float" id="F13">
<label>Figure 13</label>
<caption><p>ReLU block: <bold>(a)</bold> symbol of the block; <bold>(b)</bold> timing chart of the ReLU function; <bold>(c)</bold> true table and <bold>(d)</bold> circuit implementation by simple logic gates.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0013.tif">
<alt-text>Diagram with four panels illustrating a ReLU circuit logic. (a) ReLU circuit with input and output signals. (b) Graphs showing the signal relationships when tv&#x0002B;&#x0003C;tv- and tvi&#x0002B;&#x0003E;tvi-. (c) Truth table showing input and corresponding output. (d) Circuit diagram using logic gates with labeled inputs and outputs.</alt-text>
</graphic>
</fig>
<p>We summarize the main differences of our complementary weight approach with respect to the previous similar research (<xref ref-type="bibr" rid="B8">Bavandpour et al., 2019a</xref>; <xref ref-type="bibr" rid="B56">Sahay et al., 2020</xref>), which are also partially inspired by our model (<xref ref-type="bibr" rid="B48">Morie et al., 2016</xref>; <xref ref-type="bibr" rid="B64">Tohara et al., 2016</xref>; <xref ref-type="bibr" rid="B71">Wang et al., 2018</xref>), as follows:</p>
<list list-type="bullet">
<list-item><p>Input and output information are encoded as the timing of step voltages, rather than using a PWM scheme.</p></list-item>
<list-item><p>The response to every input step voltage in the output line is continuous until the firing threshold of the post-neuron, instead of being discrete.</p></list-item>
<list-item><p>We represent signed weights using a complementary scheme, where the two sub-weights are identical, rather than using a differential scheme.</p></list-item>
</list></sec>
<sec>
<title>5.2 Circuits</title>
<p>In order to evaluate the energy consumption and computation precision of our TACT-based circuit, we designed a PoC CMOS circuit equivalent to the RC circuit to perform the one-column (i.e., <italic>N</italic> inputs and 1 output) signed weighted-sum calculation whose synapses are in a complementary scheme. The resistive elements in the synapses are replaced by pMOSFETs. The comparator function in the neuron part is implemented by an S-R latch.</p>
<p>We propose SRAM-based synapse circuits to implement an IMC circuit for the computation shown in <xref ref-type="fig" rid="F14">Figure 14a</xref>. It consists of a 1-bit standard 6T SRAM to save the sign of a weight, a pair of pMOSFETs assigned as M1 and M2 serving as the value of the weight, and four pMOSFETs assigned as M3- M6 functioning as switches controlled by the SRAM state. M1 and M2 are identical transistors and are biased with the same gate voltage, implementing the concept of complementary dummy weight introduced in the previous chapter. They serve as current sources operating in the subthreshold saturation region, showing a high impedance. M3-M6 switch the current to the dendrite line determined by the weight&#x00027;s sign according to the diagram shown in <xref ref-type="fig" rid="F12">Figure 12a</xref>. As a PoC circuit, we implemented the BinaryConnect NN (<xref ref-type="bibr" rid="B17">Courbariaux et al., 2015</xref>) by limiting all the biases of the synapse pMOSFETs to be the same.</p>
<fig position="float" id="F14">
<label>Figure 14</label>
<caption><p>SRAM based synpase circuit and the main analog nonidealities (ANIs): <bold>(a)</bold> SRAM based synpase circuit; <bold>(b)</bold> Main ANIs of the synapse circuit <bold>(c)</bold> Output timing jitters induced by the ANIs <bold>(d)</bold> Main ANIs summary.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0014.tif">
<alt-text>Diagram composed of four parts: (a) and (b) depict electrical circuits with labeled components like transistors (M1, M2, etc.) and connections to neuron parts. (c) shows a graph plotting voltage (Vdl, Vth) versus time, illustrating jitter. (d) presents a table categorizing main ANIs (Analog Noise Injectors) such as input pulse jitter and transistor NL into stochastic and deterministic categories.</alt-text>
</graphic>
</fig>
<p>The main design parameters and simulation conditions are summarized in <xref ref-type="table" rid="T1">Table 1</xref>. We used the predictive technology model (PTM) 45 nm SPICE model for the design and simulation. Both the gate length and width of the synapse pMOSFET were 0.45 &#x003BC;m. Based on the size of the synapse pMOSFET, we estimated the parasitic capacitance of the axon line and the dendrite line based on 65 nm SRAM-based IMC circuits (<xref ref-type="bibr" rid="B35">Kneip and Bol, 2021</xref>). As a result, the parasitic capacitance of the axon line per cell, denoted as <italic>C</italic><sub><italic>al</italic></sub>, is around 0.88 fF, and the parasitic capacitance of the dendrite line per cell, denoted as <italic>C</italic><sub><italic>dl</italic></sub>, is around 0.87 fF. <italic>V</italic><sub><italic>gs</italic></sub> of the synapse pMOSFET is fixed at &#x02013;0.34 V so that one synapse current <italic>I</italic><sub><italic>s</italic></sub> is around 11.5 nA under typical conditions. We set the typical supply voltage of the synapse array and the neuron part to be 1.1 and 0.75 V, respectively. And the threshold (<italic>V</italic><sub><italic>TH</italic></sub>) of the post-neuron (i.e., the S-R latch) is around 0.4 V typically.</p>
<table-wrap position="float" id="T1">
<label>Table 1</label>
<caption><p>Simulation conditions.</p></caption>
<table frame="box" rules="all">
<thead>
<tr style="background-color:#727779;color:#ffffff">
<th valign="top" align="left"><bold>Item</bold></th>
<th valign="top" align="center"><bold>Unit</bold></th>
<th valign="top" align="center" colspan="3"><bold>Values</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Technology</td>
<td valign="top" align="center">&#x02013;</td>
<td valign="top" align="center" colspan="3">PTM 45 nm</td>
</tr> <tr>
<td valign="top" align="left">Dimension of the synapse pMOSFET</td>
<td valign="top" align="center">&#x003BC;m</td>
<td valign="top" align="center" colspan="3"><italic>L</italic> = 0.45 <italic>W</italic> = 0.45</td>
</tr> <tr>
<td valign="top" align="left">Process corner</td>
<td valign="top" align="center">&#x02013;</td>
<td valign="top" align="center">tt</td>
<td valign="top" align="center">ss</td>
<td valign="top" align="center">ff</td>
</tr> <tr>
<td valign="top" align="left">Vth of pMOSFET</td>
<td valign="top" align="center">V</td>
<td valign="top" align="center">&#x02013;0.423</td>
<td valign="top" align="center">&#x02013;0.452</td>
<td valign="top" align="center">&#x02013;0.392</td>
</tr> <tr>
<td valign="top" align="left">Temperature</td>
<td valign="top" align="center">&#x000B0;C</td>
<td valign="top" align="center">27</td>
<td valign="top" align="center">&#x02013;30</td>
<td valign="top" align="center">90</td>
</tr> <tr>
<td valign="top" align="left">Vgs of the pMOSFET</td>
<td valign="top" align="center">V</td>
<td valign="top" align="center" colspan="3">fixed to &#x02013;0.34</td>
</tr> <tr>
<td valign="top" align="left">Drain current of the pMOSFET</td>
<td valign="top" align="center">nA</td>
<td valign="top" align="center">11.5</td>
<td valign="top" align="center">4.3</td>
<td valign="top" align="center">27.8</td>
</tr></tbody>
</table>
</table-wrap>
<p>With respect to the computation precision, we set the full-scale time window (<italic>T</italic><sub><italic>in</italic></sub>) to be 640 ns, and the effective number of bit (ENOB) to be 4 bit as the design target. Then the total capacitance of the dendrite line (<italic>C</italic><sub><italic>DL</italic></sub>) for MAC computation can be obtained by</p>
<disp-formula id="E59"><label>(59)</label><mml:math id="M142"><mml:mtable class="eqnarray" columnalign="center"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>C</mml:mi></mml:mrow><mml:mrow><mml:mi>D</mml:mi><mml:mi>L</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mi>N</mml:mi><mml:msub><mml:mrow><mml:mi>I</mml:mi></mml:mrow><mml:mrow><mml:mi>s</mml:mi></mml:mrow></mml:msub><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mi>V</mml:mi></mml:mrow><mml:mrow><mml:mi>T</mml:mi><mml:mi>H</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mfrac><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p><italic>C</italic><sub><italic>DL</italic></sub> includes the total parasitic capacitance of the dendrite lines, <italic>NC</italic><sub><italic>dl</italic></sub>, the input capacitance of the post neuron, <italic>C</italic><sub><italic>i</italic></sub>, and an extra load capacitor (<italic>C</italic><sub><italic>l</italic></sub>) which is needed under the given <italic>T</italic><sub><italic>in</italic></sub> and <italic>I</italic><sub><italic>s</italic></sub> conditions.</p>
<p>Analog computation suffers from analog nonidealities (ANIs) (<xref ref-type="bibr" rid="B35">Kneip and Bol, 2021</xref>). These ANIs limit the computation precision, leading to a degradation of the inference accuracy. We sketch the main ANIs on our synapse circuit shown in <xref ref-type="fig" rid="F14">Figure 14b</xref>. All these ANIs may cause the output jitters illustrated in <xref ref-type="fig" rid="F14">Figure 14c</xref> resulting in computation errors. We summarize them in <xref ref-type="fig" rid="F14">Figure 14d</xref> classifying them by their stochastic or deterministic nature.</p>
<p>Because time domain computation is sensitive to PVT variations (<xref ref-type="bibr" rid="B59">Seo et al., 2022</xref>) and local device mismatch is dominant than the intrinsic noise (<xref ref-type="bibr" rid="B35">Kneip and Bol, 2021</xref>; <xref ref-type="bibr" rid="B23">Gonugondla et al., 2021</xref>), we mainly considered the local mismatch and the PVT variations here.</p>
<p>We conducted Monte Carlo simulations to evaluate the errors induced by the local mismatches. A set of Monte Carlo simulation waveforms is shown in <xref ref-type="fig" rid="F15">Figure 15a</xref>, and the standard deviations (&#x003C3;) vs. the number of the inputs, <italic>N</italic>, are shown in <xref ref-type="fig" rid="F15">Figure 15b</xref>. The results showed that the &#x003C3; scales roughly as <inline-formula><mml:math id="M143"><mml:mfrac><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:msqrt><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:msqrt></mml:mrow></mml:mfrac></mml:math></inline-formula> leading to higher precision for larger <italic>N</italic> (<xref ref-type="bibr" rid="B8">Bavandpour et al., 2019a</xref>).</p>
<fig position="float" id="F15">
<label>Figure 15</label>
<caption><p>TACT-based time-domain weighted-sum simulation results: <bold>(a)</bold> operational waveforms in a Monte Carlo simulation; <bold>(b)</bold> Monte Carlo simulation results of the positive or negative dendrite line firing timings <inline-formula><mml:math id="M144"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003C5;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mo>/</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003C5;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and their difference <inline-formula><mml:math id="M145"><mml:mi>&#x00394;</mml:mi><mml:msub><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003C5;</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003C5;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003C5;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> vs. the number of MAC&#x00027;s inputs (<italic>N</italic><sub><italic>in</italic></sub>); <bold>(c)</bold> comparison of the simulated PVT variations between <inline-formula><mml:math id="M146"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003C5;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mo>/</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003C5;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> and &#x00394;<italic>t</italic><sub>&#x003C5;</sub>; <bold>(d)</bold> simulated weighted-sum linearily and its PVT variation in which the max peak-to-peak (&#x00394;<italic>t</italic><sub>&#x003C5;<italic>pp</italic></sub>) is 8.33 ns in the case of <italic>N</italic><sub><italic>in</italic></sub> &#x0003D; 50; <bold>(e)</bold> the distribution of the 50-inputs weighted-sum results (&#x00394;<italic>t</italic><sub>&#x003C5;</sub>) which is well divided into 16 levels based on the predefined 4-bit output time resolution.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fnins-19-1656892-g0015.tif">
<alt-text>(a) A graph illustrating neuron voltage against time in microseconds, with input spikes and various lines representing voltage changes. (b) A line graph showing standard deviation of timing differences versus number of inputs. (c) A box plot displaying firing timings with three color-coded series: &#x00394;tv, tv-, and tv&#x0002B;. (d) A scatter plot of simulated versus expected timing differences with an enlarged section detailing a small fluctuation box plot. (e) A frequency histogram of simulated firing timing differences in nanoseconds, with peaks at regular intervals, each colored differently.</alt-text>
</graphic>
</fig>
<p>Our approach is expected to have good immunity to the PVT variations. The error induced by PVT variation in both the synapse array and neuron part is common to both positive and negative dendrite lines, and thus can be canceled from the point of view of the timing difference. To verify the PVT variation tolerance, we conducted a simulation for <italic>N</italic> &#x0003D; 50 with process and temperature (PT) corners shown in <xref ref-type="table" rid="T1">Table 1</xref>, and changed the supply voltage of the neuron part to 0.65, 0.75, and 0.85 V. We supposed that the gate voltage of the synapse pMOSFET changed along with the supply voltage of the synapse part, and thus the <italic>V</italic><sub><italic>gs</italic></sub> of it is fixed. One MAC PVT simulation result is shown in <xref ref-type="fig" rid="F15">Figure 15c</xref>. The synapse current <italic>I</italic><sub><italic>s</italic></sub> changed largely across the PT variations shown in <xref ref-type="table" rid="T1">Table 1</xref>, resulting in large variation of the single dendrite line output timings <inline-formula><mml:math id="M147"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup><mml:mo>/</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula>. However, the variation of the difference between them was much smaller. We also checked the weighted-sum linearity against the ideal 16 levels with 1<italic>LSB</italic> &#x0003D; 40 ns under the PVT simulations. The simulated linearity result is shown in <xref ref-type="fig" rid="F15">Figure 15d</xref>, indicating good linearity with the max peak-to-peak variation of 8.33 ns. Finally, we incorporated the PVT and Monte Carlo simulation results into a distribution shown in <xref ref-type="fig" rid="F15">Figure 15e</xref>, indicating that the ENOB = 4 was well achieved. We summarized the potentially achievable MAC computation precision with the number of inputs increased to 256, as shown in <xref ref-type="table" rid="T2">Table 2</xref>.</p>
<table-wrap position="float" id="T2">
<label>Table 2</label>
<caption><p>The MAC computation precision.</p></caption>
<table frame="box" rules="all">
<thead>
<tr style="background-color:#727779;color:#ffffff">
<th valign="top" align="left"><bold>Item</bold></th>
<th valign="top" align="center"><bold>Unit</bold></th>
<th valign="top" align="center"><bold>Simulation results</bold></th>
<th valign="top" align="center" colspan="2"><bold>Estimation results</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Number of inputs</td>
<td valign="top" align="center">&#x02013;</td>
<td valign="top" align="center">50</td>
<td valign="top" align="center">100</td>
<td valign="top" align="center">256</td>
</tr> <tr>
<td valign="top" align="left">Full-scale time window (Tin)</td>
<td valign="top" align="center">ns</td>
<td valign="top" align="center" colspan="3">640</td>
</tr> <tr>
<td valign="top" align="left">Peak-to-peak of <inline-formula><mml:math id="M148"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula> caused by PVT</td>
<td valign="top" align="center">ns</td>
<td valign="top" align="center">8.33</td>
<td valign="top" align="center">5.89</td>
<td valign="top" align="center">1.4</td>
</tr> <tr>
<td valign="top" align="left">Standard deviation (&#x003C3;) of <inline-formula><mml:math id="M149"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003BD;</mml:mi></mml:mrow><mml:mrow><mml:mo>&#x0002B;</mml:mo></mml:mrow></mml:msubsup></mml:math></inline-formula></td>
</tr>
<tr>
<td valign="top" align="left">caused by mismatch</td>
<td valign="top" align="center">ns</td>
<td valign="top" align="center">3.61</td>
<td valign="top" align="center"><sup>&#x0002A;</sup>3.12</td>
<td valign="top" align="center">1.99</td>
</tr> <tr>
<td valign="top" align="left">ENOB computed by 1 &#x003C3;</td>
<td valign="top" align="center">bit</td>
<td valign="top" align="center">5.3</td>
<td valign="top" align="center">5.7</td>
<td valign="top" align="center">6.3</td>
</tr> <tr>
<td valign="top" align="left">ENOB computed by 3 &#x003C3;</td>
<td valign="top" align="center">bit</td>
<td valign="top" align="center">4.4</td>
<td valign="top" align="center">4.7</td>
<td valign="top" align="center">5.4</td>
</tr></tbody>
</table>
<table-wrap-foot>
<p><sup>&#x0002A;</sup>Simulation value.</p>
</table-wrap-foot>
</table-wrap>
<p>With respect to energy consumption, an input voltage charges up the parasitic capacitance of the axon line, <italic>C</italic><sub><italic>al</italic></sub>, and then charges the capacitance <italic>C</italic><sub><italic>DL</italic></sub> via synapse pMOSFET. Therefore, total energy consumption due to the dendrite line (DL) charge and discharge, <italic>E</italic><sub><italic>DL</italic></sub>, is expressed by <inline-formula><mml:math id="M150"><mml:msub><mml:mrow><mml:mi>E</mml:mi></mml:mrow><mml:mrow><mml:mi>D</mml:mi><mml:mi>L</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:msub><mml:mrow><mml:mi>C</mml:mi></mml:mrow><mml:mrow><mml:mi>D</mml:mi><mml:mi>L</mml:mi></mml:mrow></mml:msub><mml:msubsup><mml:mrow><mml:mi>V</mml:mi></mml:mrow><mml:mrow><mml:mi>T</mml:mi><mml:mi>H</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup></mml:math></inline-formula>, and it was 80.59 fJ when <italic>V</italic><sub><italic>TH</italic></sub> &#x0003D; 0.3 V and 143.27 fJ when <italic>V</italic><sub><italic>TH</italic></sub> &#x0003D; 0.4 V. And the total energy consumption per DL due to the axon line (AL) charge and discharge, <italic>E</italic><sub><italic>AL</italic></sub>, is expressed by <inline-formula><mml:math id="M151"><mml:msub><mml:mrow><mml:mi>E</mml:mi></mml:mrow><mml:mrow><mml:mi>A</mml:mi><mml:mi>L</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mi>N</mml:mi><mml:msub><mml:mrow><mml:mi>C</mml:mi></mml:mrow><mml:mrow><mml:mi>a</mml:mi><mml:mi>l</mml:mi></mml:mrow></mml:msub><mml:msubsup><mml:mrow><mml:mi>V</mml:mi></mml:mrow><mml:mrow><mml:mi>d</mml:mi><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup></mml:math></inline-formula>, and it was 53.24 fJ, where <italic>V</italic><sub><italic>dd</italic></sub> &#x0003D; 1.1 V.</p>
<p>As for the neuron part, which consists of an S-R latch and the output buffer, the energy consumption, <italic>E</italic><sub><italic>NP</italic></sub>, was about 216.91 fJ when the supply voltage was 0.75 V, and decreased to about 76.49 fJ when the supply voltage is 0.65 V.</p>
<p>As a result, overall energy consumption was 210.32 fJ per MAC with <italic>N</italic> &#x0003D; 50 when the supply voltage of the neuron part was 0.65 V. This implies that the energy efficiency is 237.74 TOPS/W (Tera-Operations Per Second per Watt). This efficiency is comparable to the state of the art of the analog MAC macros (<xref ref-type="bibr" rid="B59">Seo et al., 2022</xref>; <xref ref-type="bibr" rid="B14">Choi et al., 2023</xref>). We summarized the potentially achievable energy efficiency with the number of inputs increased to 256, as shown in <xref ref-type="table" rid="T3">Table 3</xref>.</p>
<table-wrap position="float" id="T3">
<label>Table 3</label>
<caption><p>The energy consumption breakdown and the energy efficiency of one MAC computation.</p></caption>
<table frame="box" rules="all">
<thead>
<tr style="background-color:#727779;color:#ffffff">
<th valign="top" align="left"><bold>Item</bold></th>
<th valign="top" align="center"><bold>Unit</bold></th>
<th valign="top" align="center" colspan="2"><bold>Simulation results</bold></th>
<th valign="top" align="center" colspan="2"><bold>Estimation results</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Number of inputs</td>
<td valign="top" align="center">&#x02013;</td>
<td valign="top" align="center" colspan="2">50</td>
<td valign="top" align="center">100</td>
<td valign="top" align="center">256</td>
</tr> <tr style="background-color:#dee1e1">
<td valign="top" align="left" colspan="6"><bold>Energy consumption due to the axon line charge and discharge</bold></td>
</tr> <tr>
<td valign="top" align="left">Total capacitance of the AL per DL</td>
<td valign="top" align="center">fF</td>
<td valign="top" align="center" colspan="2">44</td>
<td valign="top" align="center">88</td>
<td valign="top" align="center">225.28</td>
</tr> <tr>
<td valign="top" align="left">Supply voltage in synapse part</td>
<td valign="top" align="center">V</td>
<td valign="top" align="center" colspan="4">1.1</td>
</tr> <tr>
<td valign="top" align="left"><bold>Energy consumption of the AL per DL</bold></td>
<td valign="top" align="center">fJ</td>
<td valign="top" align="center" colspan="2">53.24</td>
<td valign="top" align="center">106.48</td>
<td valign="top" align="center">272.59</td>
</tr> <tr style="background-color:#dee1e1">
<td valign="top" align="left" colspan="6"><bold>Energy consumption due to the dendrite line charge and discharge</bold></td>
</tr> <tr>
<td valign="top" align="left">Total capacitance of one DL</td>
<td valign="top" align="center">fF</td>
<td valign="top" align="center" colspan="2">895.44</td>
<td valign="top" align="center">1788.94</td>
<td valign="top" align="center">4576.66</td>
</tr> <tr>
<td valign="top" align="left">Firing threshold of the neuron part</td>
<td valign="top" align="center">V</td>
<td valign="top" align="center">0.4</td>
<td valign="top" align="center">0.3</td>
<td valign="top" align="center">0.3</td>
<td valign="top" align="center">0.3</td>
</tr> <tr>
<td valign="top" align="left"><bold>Energy consumption of one DL</bold></td>
<td valign="top" align="center">fJ</td>
<td valign="top" align="center">143.27</td>
<td valign="top" align="center">80.59</td>
<td valign="top" align="center">161.00</td>
<td valign="top" align="center">411.90</td>
</tr> <tr style="background-color:#dee1e1">
<td valign="top" align="left" colspan="6"><bold>Energy consumption in the neuron part</bold></td>
</tr> <tr>
<td valign="top" align="left">Supply voltage in neuron part</td>
<td valign="top" align="center">V</td>
<td valign="top" align="center">0.75</td>
<td valign="top" align="center">0.65</td>
<td valign="top" align="center">0.65</td>
<td valign="top" align="center">0.65</td>
</tr> <tr>
<td valign="top" align="left"><bold>Energy consumption of one DL</bold></td>
<td valign="top" align="center">fJ</td>
<td valign="top" align="center">216.91</td>
<td valign="top" align="center">76.49</td>
<td valign="top" align="center">76.49</td>
<td valign="top" align="center">76.49</td>
</tr> <tr style="background-color:#dee1e1">
<td valign="top" align="left" colspan="6"><bold>Energy efficiency</bold></td>
</tr> <tr>
<td valign="top" align="left">Total energy consumption of one MAC</td>
<td valign="top" align="center">fJ</td>
<td valign="top" align="center">413.42</td>
<td valign="top" align="center">210.32</td>
<td valign="top" align="center">343.97</td>
<td valign="top" align="center">760.97</td>
</tr> <tr>
<td valign="top" align="left">Energy/synapse operation</td>
<td valign="top" align="center">fJ</td>
<td valign="top" align="center">8.27</td>
<td valign="top" align="center">4.21</td>
<td valign="top" align="center">3.44</td>
<td valign="top" align="center">2.97</td>
</tr> <tr>
<td valign="top" align="left"><bold>Energy efficiency</bold></td>
<td valign="top" align="center">TOPS/W</td>
<td valign="top" align="center">120.94</td>
<td valign="top" align="center">237.74</td>
<td valign="top" align="center">290.72</td>
<td valign="top" align="center">336.41</td>
</tr></tbody>
</table>
</table-wrap>
<p>Our purpose is to show the potential energy efficiency and computation precision of the TACT-based circuit, so we don&#x00027;t perform further design space exploration for optimizing the performance of the proposed circuit.</p></sec></sec>
<sec sec-type="discussion" id="s6">
<title>6 Discussion</title>
<p>We discuss some possible improvements on our PoC circuit design here.</p>
<p>In our design of the PoC circuit, we use a relatively long time window, i.e., 640 ns, to guarantee a moderate time resolution of 4&#x02013;7 bits. The single MAC operation time is 1,300 ns, which consists of 20 ns for the neuron part reset and 640 ns for the input and output window, respectively. To improve the system latency, we can utilize massively parallel MAC operations to compensate for the relatively slow single MAC computation thanks to our simple readout circuit, which is area-efficient to make one column one readout possible, like in <xref ref-type="bibr" rid="B33">Khaddam-Aljameh et al. (2022)</xref> and <xref ref-type="bibr" rid="B68">Wan et al. (2022)</xref>. At the system level, applying a pipeline scheme that uses the output timing window as the input window for subsequent computation can also be an effective approach <xref ref-type="bibr" rid="B41">Lim et al., (2020)</xref>; <xref ref-type="bibr" rid="B59">Seo et al., (2022)</xref>; <xref ref-type="bibr" rid="B6">Ambrogio et al., (2023)</xref>.</p>
<p>We also designed a relatively large capacitance for the DL, which could degrade the area efficiency of the AIMC system. The total DL capacitance, <italic>C</italic><sub><italic>DL</italic></sub>, is about 895 fF when the number of inputs is 50, as shown in <xref ref-type="table" rid="T3">Table 3</xref>. Because the wiring parasitic capacitance of the dendrite line per cell, <italic>C</italic><sub><italic>dl</italic></sub>, is around 0.87 fF, an extra load capacitor, <italic>C</italic><sub><italic>l</italic></sub>, will be about 850 fF, leading to area inefficiency in the neuron part. According to <xref ref-type="disp-formula" rid="E59">Equation 59</xref>, shortening the full-scale time window, decreasing the total integration current, or setting a higher <italic>V</italic><sub><italic>TH</italic></sub> will help minimize the capacitance to improve area efficiency. Regarding the decrease of the total integration current, we can make use of sparsity-aware optimization such as weight pruning. The sparsity, which we refer to as the ratio of zero weights to total weights, is typically 20%&#x02013;50% <xref ref-type="bibr" rid="B63">Sze et al., (2017)</xref>; <xref ref-type="bibr" rid="B18">Deng et al., (2020)</xref>. Suppose the sparsity is 40%, then <italic>C</italic><sub><italic>DL</italic></sub> will be about 540 fF. To further minimize neuron part area overhead, we can also implement capacitors using a multi-layer metal-oxide-metal (MoM) structure lying on top of transistors in the synapse cell <xref ref-type="bibr" rid="B66">Valavi et al., (2019)</xref>; <xref ref-type="bibr" rid="B59">Seo et al., (2022)</xref>. Typically, the capacitance is 1&#x02013;3 fF per cell area. By this means, <italic>C</italic><sub><italic>l</italic></sub> can be optimized to about 340 fF and such capacitance can be efficiently implemented by a MOSCAP <xref ref-type="bibr" rid="B8">Bavandpour et al., (2019a)</xref>.</p>
<p>Shortening the full-scale time window also helps minimize <italic>C</italic><sub><italic>DL</italic></sub>, but it will lead to a degradation of the computation precision. Because it involves improving the system latency and lowering the energy consumption of the neuron part, we are interested in estimating the results. When the number of inputs is increased to 256, the ENOB can be up to 5.4 bits, as shown in <xref ref-type="table" rid="T2">Table 2</xref>. If we keep the ENOB target as four bits, the full-scale time window can be shortened to about 250 ns. <italic>C</italic><sub><italic>DL</italic></sub> will be decreased to about 1,100 fF, and <italic>C</italic><sub><italic>l</italic></sub> can be minimized to a level under 100 fF, given that the weights&#x00027; sparsity is 40%, <italic>I</italic><sub><italic>s</italic></sub> is 11.5 nA, and <italic>V</italic><sub><italic>TH</italic></sub> &#x0003D; 0.4 V. Accordingly, the energy consumption of the DL, <italic>E</italic><sub><italic>DL</italic></sub>, and the neuron part, <italic>E</italic><sub><italic>NP</italic></sub>, is optimized to about 176.6 and 29.9 fJ, respectively. This indicates an energy efficiency of 534.3 TOPS/W. We compare our work with the previous AIMC designs shown in <xref ref-type="table" rid="T4">Table 4</xref>. Our work shows a favorable performance.</p>
<table-wrap position="float" id="T4">
<label>Table 4</label>
<caption><p>Performance summary and comparison with previous AIMC designs.</p></caption>
<table frame="box" rules="all">
<thead>
<tr style="background-color:#727779;color:#ffffff">
<th valign="top" align="left"><bold>Item</bold></th>
<th valign="top" align="center" colspan="2"><bold>This work</bold></th>
<th valign="top" align="center"><bold><xref ref-type="bibr" rid="B8">Bavandpour et al. (2019a)</xref></bold></th>
<th valign="top" align="center"><bold><xref ref-type="bibr" rid="B41">Lim et al. (2020)</xref></bold></th>
<th valign="top" align="center"><bold><xref ref-type="bibr" rid="B59">Seo et al. (2022)</xref></bold></th>
<th valign="top" align="center"><bold><xref ref-type="bibr" rid="B73">Wu et al. (2022)</xref></bold></th>
<th valign="top" align="center"><bold><xref ref-type="bibr" rid="B68">Wan et al. (2022)</xref></bold></th>
<th valign="top" align="center"><bold><xref ref-type="bibr" rid="B38">Le Gallo et al. (2023)</xref></bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Memory type</td>
<td valign="top" align="center" colspan="2">SRAM</td>
<td valign="top" align="center">NOR flash</td>
<td valign="top" align="center">SRAM</td>
<td valign="top" align="center">Analog memory</td>
<td valign="top" align="center">SRAM</td>
<td valign="top" align="center">RRAM</td>
<td valign="top" align="center">PCM</td>
</tr> <tr>
<td valign="top" align="left">Technology</td>
<td valign="top" align="center" colspan="2">45 nm</td>
<td valign="top" align="center">55 nm</td>
<td valign="top" align="center">28 nm</td>
<td valign="top" align="center">28 nm</td>
<td valign="top" align="center">28 nm</td>
<td valign="top" align="center">130 nm</td>
<td valign="top" align="center">14 nm</td>
</tr> <tr>
<td valign="top" align="left">Computing method</td>
<td valign="top" align="center" colspan="5">Time-domain &#x00026; charge accumulation</td>
<td valign="top" align="center">Time-domain</td>
<td valign="top" align="center">Voltage</td>
<td valign="top" align="center">Current</td>
</tr> <tr style="background-color:#dee1e1">
<td valign="top" align="left" colspan="9"><bold>MVM core size [row</bold>&#x000D7;<bold>column], precision [bit], energy efficiency (EE) [TOPS/W], computation time [ns]</bold></td>
</tr> <tr>
<td valign="top" align="left">Core size</td>
<td valign="top" align="center">256 &#x000D7; 1</td>
<td valign="top" align="center">50 &#x000D7; 1</td>
<td valign="top" align="center">100 &#x000D7; 100/ 500 &#x000D7; 500</td>
<td valign="top" align="center">27 &#x000D7; 8</td>
<td valign="top" align="center">28 &#x000D7; 28</td>
<td valign="top" align="center">64 &#x000D7; 256</td>
<td valign="top" align="center">256 &#x000D7; 256</td>
<td valign="top" align="center">256 &#x000D7; 256</td>
</tr> <tr>
<td valign="top" align="left">Input precision</td>
<td valign="top" align="center" colspan="2">Timing (ENB 4-6b)</td>
<td valign="top" align="center">Pulse width (ENB 6b)</td>
<td valign="top" align="center">Pulse width (ENB 3-7b)</td>
<td valign="top" align="center">Analog</td>
<td valign="top" align="center">4b/8b</td>
<td valign="top" align="center">4b</td>
<td valign="top" align="center">8b</td>
</tr> <tr>
<td valign="top" align="left">Weight precision</td>
<td valign="top" align="center" colspan="2">1b/Analog</td>
<td valign="top" align="center">Analog</td>
<td valign="top" align="center">4b</td>
<td valign="top" align="center">5b</td>
<td valign="top" align="center">4b/8b</td>
<td valign="top" align="center">Analog</td>
<td valign="top" align="center">Analog</td>
</tr> <tr>
<td valign="top" align="left">Output precision</td>
<td valign="top" align="center" colspan="2">Timing (ENB 4-6b)</td>
<td valign="top" align="center">Pulse width (ENB 6b)</td>
<td valign="top" align="center">Pulse width (analog)</td>
<td valign="top" align="center">Pulse width (analog)</td>
<td valign="top" align="center">Digital 14b/22b</td>
<td valign="top" align="center">Digital 6b</td>
<td valign="top" align="center">Digital 8b</td>
</tr> <tr>
<td valign="top" align="left">EE [TOPS/W]</td>
<td valign="top" align="center"><inline-formula><mml:math id="M152"><mml:mrow><mml:mstyle mathcolor="#0000ff"><mml:mtext>534.3 (4b)</mml:mtext></mml:mstyle></mml:mrow></mml:math></inline-formula></td>
<td valign="top" align="center">237.7</td>
<td valign="top" align="center">85/135</td>
<td valign="top" align="center">7.1 (system)</td>
<td valign="top" align="center">332.7</td>
<td valign="top" align="center">85&#x02013;112 (4b/4b/14b)</td>
<td valign="top" align="center">16</td>
<td valign="top" align="center">2.48</td>
</tr> <tr>
<td valign="top" align="left">Computation time [ns]</td>
<td valign="top" align="center"><inline-formula><mml:math id="M153"><mml:mrow><mml:mstyle mathcolor="#0000ff"><mml:mtext>500 (4b)</mml:mtext></mml:mstyle></mml:mrow></mml:math></inline-formula></td>
<td valign="top" align="center">1,300</td>
<td valign="top" align="center">50&#x02013;200</td>
<td valign="top" align="center">150*</td>
<td valign="top" align="center">740</td>
<td valign="top" align="center">105.6</td>
<td valign="top" align="center">4,000</td>
<td valign="top" align="center">520&#x02013;1,518</td>
</tr> <tr>
<td valign="top" align="left">Results</td>
<td valign="top" align="center">Estimated</td>
<td valign="top" align="center">Simulated</td>
<td valign="top" align="center">Simulated</td>
<td valign="top" align="center">Simulated</td>
<td valign="top" align="center">Measured</td>
<td valign="top" align="center">Measured</td>
<td valign="top" align="center">Measured</td>
<td valign="top" align="center">Measured</td>
</tr> <tr style="background-color:#dee1e1">
<td valign="top" align="left" colspan="9"><bold>Readout</bold></td>
</tr> <tr>
<td valign="top" align="left">Circuit</td>
<td valign="top" align="center" colspan="2">S-R Latch</td>
<td valign="top" align="center">TDC</td>
<td valign="top" align="center">Amp&#x00026; Comparator</td>
<td valign="top" align="center">VTC</td>
<td valign="top" align="center">TDC</td>
<td valign="top" align="center">column ADC</td>
<td valign="top" align="center">CCO-based ADC</td>
</tr> <tr>
<td valign="top" align="left">Compactness</td>
<td valign="top" align="center" colspan="2">&#x025CB;</td>
<td valign="top" align="center">&#x025CB;</td>
<td valign="top" align="center">&#x000D7;</td>
<td valign="top" align="center">&#x025CB;</td>
<td valign="top" align="center">&#x025CB;</td>
<td valign="top" align="center">&#x025CB;</td>
<td valign="top" align="center">&#x025CB;</td>
</tr> <tr>
<td valign="top" align="left">Cycles or modulation steps for one MVM computation</td>
<td valign="top" align="center" colspan="2"><inline-formula><mml:math id="M154"><mml:mrow><mml:mstyle mathcolor="#0000ff"><mml:mtext>Single</mml:mtext></mml:mstyle></mml:mrow></mml:math></inline-formula></td>
<td valign="top" align="center">Single</td>
<td valign="top" align="center">Multiple</td>
<td valign="top" align="center">Multiple</td>
<td valign="top" align="center">Multiple</td>
<td valign="top" align="center">Multiple</td>
<td valign="top" align="center">Four</td>
</tr> <tr>
<td valign="top" align="left">Processing w/o ADC</td>
<td valign="top" align="center" colspan="2">&#x025CB;</td>
<td valign="top" align="center">&#x000D7;</td>
<td valign="top" align="center">&#x025CB;</td>
<td valign="top" align="center">&#x025CB;</td>
<td valign="top" align="center">&#x000D7;</td>
<td valign="top" align="center">&#x000D7;</td>
<td valign="top" align="center">&#x000D7;</td>
</tr> <tr style="background-color:#dee1e1">
<td valign="top" align="left" colspan="9"><bold>Other features</bold></td>
</tr> <tr>
<td valign="top" align="left">Signed MAC</td>
<td valign="top" align="center" colspan="2"><inline-formula><mml:math id="M155"><mml:mrow><mml:mstyle mathcolor="#0000ff"><mml:mtext>4-quadrant</mml:mtext></mml:mstyle></mml:mrow></mml:math></inline-formula></td>
<td valign="top" align="center">2-quadrant</td>
<td valign="top" align="center">2-quadrant</td>
<td valign="top" align="center">2-quadrant</td>
<td valign="top" align="center">4-quadrant</td>
<td valign="top" align="center">4-quadrant</td>
<td valign="top" align="center">4-quadrant</td>
</tr> <tr>
<td valign="top" align="left">PVT tolerance</td>
<td valign="top" align="center" colspan="2">&#x025CB;</td>
<td valign="top" align="center">&#x025CB;</td>
<td valign="top" align="center">&#x000D7;</td>
<td valign="top" align="center">&#x025CB;</td>
<td valign="top" align="center">NA</td>
<td valign="top" align="center">NA</td>
<td valign="top" align="center">NA</td>
</tr></tbody>
</table>
<table-wrap-foot>
<p>*Estimated value. **Colored as favorable features of this work.</p>
</table-wrap-foot>
</table-wrap>
<p>When deploying DNNs to resource-constrained edge devices, trade-offs between accuracy, model size, latency, and energy efficiency need to be optimized, which is typically achieved by means of algorithm&#x02014;hardware codesigns (<xref ref-type="bibr" rid="B62">Shuvo et al., 2022</xref>; <xref ref-type="bibr" rid="B53">Ngo et al., 2025</xref>).</p>
<p>Our future work includes the design and fabrication of a fully parallel MVM AIMC core or macro and the measurement of DNN inference accuracy, latency, energy efficiency on more realistic datasets such as CIFAR-10 and CIFAR-100. With respect to NN model optimization for the hardware, the improvement of the accuracy of the NN model without bias, discussed in Section 4.1, will also be an important effort.</p></sec>
<sec sec-type="conclusions" id="s7">
<title>7 Conclusions</title>
<p>We introduced a time-domain four-quadrant MAC calculation model where signed inputs are encoded using a differential pair of spikes, and signed weights are implemented through a dummy weights scheme. The output is represented by a pair of spikes, with their timing difference proportional to the MAC results, enabled by the added dummy weights. Since both inputs and outputs are encoded in a timing format, the AIMC core with this model can be seamlessly integrated with efficient DTCs and TDCs. We proposed architectures for our TACT-based MLP with the weights configured in a complementary scheme. We demonstrated a proof-of-concept (PoC) CMOS circuit equivalent to the previously proposed RC circuit, with preliminary simulation suggesting that the energy efficiency could reach hundreds of Tera Operations Per Second Per Watt (TOPS/W) and the precision could be four bit or higher.</p>
<p>Our proposed time-domain weighted-sum calculation model promises to be a suitable approach for intensive in-memory computing (IMC) of deep neural networks (DNNs) with moderate multi-bit inputs/outputs and weights, and avoiding or reducing the cost of ADC overhead so as to ultimately run the DNNs energy efficiently on edge devices for inference tasks.</p></sec>
</body>
<back>
<sec sec-type="data-availability" id="s8">
<title>Data availability statement</title>
<p>The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.</p>
</sec>
<sec sec-type="author-contributions" id="s9">
<title>Author contributions</title>
<p>QW: Investigation, Software, Conceptualization, Writing &#x02013; review &#x00026; editing, Visualization, Validation, Writing &#x02013; original draft. HT: Project administration, Resources, Writing &#x02013; review &#x00026; editing, Supervision. TM: Writing &#x02013; review &#x00026; editing, Project administration, Conceptualization, Funding acquisition, Supervision.</p>
</sec>
<sec sec-type="funding-information" id="s10">
<title>Funding</title>
<p>The author(s) declare that financial support was received for the research and/or publication of this article. This work was supported by JSPS KAKENHI Grant Nos. 22240022 and 15H01706. Part of the work was carried out under project JPNP16007 commissioned by the New Energy and Industrial Technology Development Organization (NEDO).</p>
</sec>
<sec sec-type="COI-statement" id="conf1">
<title>Conflict of interest</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec sec-type="ai-statement" id="s11">
<title>Generative AI statement</title>
<p>The author(s) declare that no Gen AI was used in the creation of this manuscript.</p>
<p>Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.</p></sec>
<sec sec-type="disclaimer" id="s12">
<title>Publisher&#x00027;s note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
<ref-list>
<title>References</title>
<ref id="B1">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Abas</surname> <given-names>A.</given-names></name> <name><surname>Bystrov</surname> <given-names>A.</given-names></name> <name><surname>Kinniment</surname> <given-names>D.</given-names></name> <name><surname>Maevsky</surname> <given-names>O.</given-names></name> <name><surname>Russell</surname> <given-names>G.</given-names></name> <name><surname>Yakovlev</surname> <given-names>A.</given-names></name> <etal/></person-group>. (<year>2002</year>). <article-title>Time difference amplifier</article-title>. <source>Electron. Lett</source>. <volume>38</volume>, <fpage>1437</fpage>&#x02013;<lpage>1438</lpage>. <pub-id pub-id-type="doi">10.1049/el:20020961</pub-id></citation>
</ref>
<ref id="B2">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Aguirre</surname> <given-names>F.</given-names></name> <name><surname>Sebastian</surname> <given-names>A.</given-names></name> <name><surname>Le Gallo</surname> <given-names>M.</given-names></name> <name><surname>Song</surname> <given-names>W.</given-names></name> <name><surname>Wang</surname> <given-names>T.</given-names></name> <name><surname>Yang</surname> <given-names>J. J.</given-names></name> <etal/></person-group>. (<year>2024</year>). <article-title>Hardware implementation of memristor-based artificial neural networks</article-title>. <source>Nat. Commun</source>. <volume>15</volume>:<fpage>1974</fpage>. <pub-id pub-id-type="doi">10.1038/s41467-024-45670-9</pub-id><pub-id pub-id-type="pmid">38438350</pub-id></citation></ref>
<ref id="B3">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Al Maharmeh</surname> <given-names>H.</given-names></name> <name><surname>Ismail</surname> <given-names>M.</given-names></name> <name><surname>Alhawari</surname> <given-names>M.</given-names></name> <etal/></person-group>. (<year>2024</year>). <article-title>Energy-efficient time-domain computation for edge devices: challenges and prospects</article-title>. <source>Found. Trends Integr. Circuits Syst</source>. <volume>3</volume>, <fpage>1</fpage>&#x02013;<lpage>50</lpage>. <pub-id pub-id-type="doi">10.1561/3500000013</pub-id></citation>
</ref>
<ref id="B4">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Al Maharmeh</surname> <given-names>H.</given-names></name> <name><surname>Sarhan</surname> <given-names>N. J.</given-names></name> <name><surname>Hung</surname> <given-names>C.-C.</given-names></name> <name><surname>Ismail</surname> <given-names>M.</given-names></name> <name><surname>Alhawari</surname> <given-names>M.</given-names></name></person-group> (<year>2020</year>). <article-title>&#x0201C;Compute-in-time for deep neural network accelerators: challenges and prospects,&#x0201D;</article-title> in <source>2020 IEEE 63rd International Midwest Symposium on Circuits and Systems (MWSCAS)</source> (<publisher-loc>Springfield, MA</publisher-loc>: <publisher-name>IEEE</publisher-name>), <fpage>990</fpage>&#x02013;<lpage>993</lpage>. <pub-id pub-id-type="doi">10.1109/MWSCAS48704.2020.9184470</pub-id></citation>
</ref>
<ref id="B5">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Al Maharmeh</surname> <given-names>H.</given-names></name> <name><surname>Sarhan</surname> <given-names>N. J.</given-names></name> <name><surname>Ismail</surname> <given-names>M.</given-names></name> <name><surname>Alhawari</surname> <given-names>M.</given-names></name></person-group> (<year>2023</year>). <article-title>A 116 tops/w spatially unrolled time-domain accelerator utilizing laddered-inverter dtc for energy-efficient edge computing in 65 nm</article-title>. <source>IEEE Open J. Circuits Syst</source>. <volume>4</volume>, <fpage>308</fpage>&#x02013;<lpage>323</lpage>. <pub-id pub-id-type="doi">10.1109/OJCAS.2023.3332853</pub-id></citation>
</ref>
<ref id="B6">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ambrogio</surname> <given-names>S.</given-names></name> <name><surname>Narayanan</surname> <given-names>P.</given-names></name> <name><surname>Okazaki</surname> <given-names>A.</given-names></name> <name><surname>Fasoli</surname> <given-names>A.</given-names></name> <name><surname>Mackin</surname> <given-names>C.</given-names></name> <name><surname>Hosokawa</surname> <given-names>K.</given-names></name> <etal/></person-group>. (<year>2023</year>). <article-title>An analog-AI chip for energy-efficient speech recognition and transcription</article-title>. <source>Nature</source> <volume>620</volume>, <fpage>768</fpage>&#x02013;<lpage>775</lpage>. <pub-id pub-id-type="doi">10.1038/s41586-023-06337-5</pub-id><pub-id pub-id-type="pmid">37612392</pub-id></citation></ref>
<ref id="B7">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Asada</surname> <given-names>K.</given-names></name> <name><surname>Nakura</surname> <given-names>T.</given-names></name> <name><surname>Iizuka</surname> <given-names>T.</given-names></name> <name><surname>Ikeda</surname> <given-names>M.</given-names></name></person-group> (<year>2018</year>). <article-title>Time-domain approach for analog circuits in deep sub-micron LSI</article-title>. <source>IEICE Electron. Express</source> <volume>15</volume>:<fpage>20182001</fpage>. <pub-id pub-id-type="doi">10.1587/elex.15.20182001</pub-id></citation>
</ref>
<ref id="B8">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bavandpour</surname> <given-names>M.</given-names></name> <name><surname>Mahmoodi</surname> <given-names>M. R.</given-names></name> <name><surname>Strukov</surname> <given-names>D. B.</given-names></name></person-group> (<year>2019a</year>). <article-title>Energy-efficient time-domain vector-by-matrix multiplier for neurocomputing and beyond</article-title>. <source>IEEE Trans. Circuits Syst. II: Express Briefs</source> <volume>66</volume>, <fpage>1512</fpage>&#x02013;<lpage>1516</lpage>. <pub-id pub-id-type="doi">10.1109/TCSII.2019.2891688</pub-id></citation>
</ref>
<ref id="B9">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bavandpour</surname> <given-names>M.</given-names></name> <name><surname>Mahmoodi</surname> <given-names>M. R.</given-names></name> <name><surname>Strukov</surname> <given-names>D. B.</given-names></name></person-group> (<year>2020</year>). <article-title>Acortex: an energy-efficient multipurpose mixed-signal inference accelerator. <italic>IEEE J. Explor</italic></article-title>. <source>Solid-State Comput. Devices Circuits</source> <volume>6</volume>, <fpage>98</fpage>&#x02013;<lpage>106</lpage>. <pub-id pub-id-type="doi">10.1109/JXCDC.2020.2999581</pub-id></citation>
</ref>
<ref id="B10">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bavandpour</surname> <given-names>M.</given-names></name> <name><surname>Sahay</surname> <given-names>S.</given-names></name> <name><surname>Mahmoodi</surname> <given-names>M. R.</given-names></name> <name><surname>Strukov</surname> <given-names>D.</given-names></name></person-group> (<year>2019b</year>). <article-title>Efficient mixed-signal neurocomputing via successive integration and rescaling</article-title>. <source>IEEE Trans Very Large Scale Integr. Syst</source>. <volume>28</volume>, <fpage>823</fpage>&#x02013;<lpage>827</lpage>. <pub-id pub-id-type="doi">10.1109/TVLSI.2019.2946516</pub-id></citation>
</ref>
<ref id="B11">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bavandpour</surname> <given-names>M.</given-names></name> <name><surname>Sahay</surname> <given-names>S.</given-names></name> <name><surname>Mahmoodi</surname> <given-names>M. R.</given-names></name> <name><surname>Strukov</surname> <given-names>D. B.</given-names></name></person-group> (<year>2021</year>). <article-title>3D-acortex: an ultra-compact energy-efficient neurocomputing platform based on commercial 3D-nand flash memories</article-title>. <source>Neuromorphic Comput. Eng</source>. <volume>1</volume>:<fpage>014001</fpage>. <pub-id pub-id-type="doi">10.1088/2634-4386/ac0775</pub-id></citation>
</ref>
<ref id="B12">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Chen</surname> <given-names>Y.</given-names></name> <name><surname>Xie</surname> <given-names>Y.</given-names></name> <name><surname>Song</surname> <given-names>L.</given-names></name> <name><surname>Chen</surname> <given-names>F.</given-names></name> <name><surname>Tang</surname> <given-names>T.</given-names></name></person-group> (<year>2020</year>). <article-title>A survey of accelerator architectures for deep neural networks</article-title>. <source>Engineering</source> <volume>6</volume>, <fpage>264</fpage>&#x02013;<lpage>274</lpage>. <pub-id pub-id-type="doi">10.1016/j.eng.2020.01.007</pub-id></citation>
</ref>
<ref id="B13">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Chen</surname> <given-names>Z.</given-names></name> <name><surname>Jin</surname> <given-names>Q.</given-names></name> <name><surname>Yu</surname> <given-names>Z.</given-names></name> <name><surname>Wang</surname> <given-names>Y.</given-names></name> <name><surname>Yang</surname> <given-names>K.</given-names></name></person-group> (<year>2022</year>). <article-title>&#x0201C;DCT-RAM: a driver-free process-in-memory 8t sram macro with multi-bit charge-domain computation and time-domain quantization,&#x0201D;</article-title> in <source>2022 IEEE Custom Integrated Circuits Conference (CICC)</source> (<publisher-loc>Newport Beach, CA</publisher-loc>: <publisher-name>IEEE</publisher-name>), <fpage>1</fpage>&#x02013;<lpage>2</lpage>. <pub-id pub-id-type="doi">10.1109/CICC53496.2022.9772826</pub-id></citation>
</ref>
<ref id="B14">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Choi</surname> <given-names>E.</given-names></name> <name><surname>Choi</surname> <given-names>I.</given-names></name> <name><surname>Lukito</surname> <given-names>V.</given-names></name> <name><surname>Choi</surname> <given-names>D.-H.</given-names></name> <name><surname>Yi</surname> <given-names>D.</given-names></name> <name><surname>Chang</surname> <given-names>I.-J.</given-names></name> <etal/></person-group>. (<year>2023</year>). <article-title>&#x0201C;A 333tops/w logic-compatible multi-level embedded flash compute-in-memory macro with dual-slope computation,&#x0201D;</article-title> in <source>2023 IEEE Custom Integrated Circuits Conference (CICC)</source> (<publisher-loc>San Antonio, TX</publisher-loc>: <publisher-name>IEEE</publisher-name>), <fpage>1</fpage>&#x02013;<lpage>2</lpage>. <pub-id pub-id-type="doi">10.1109/CICC57935.2023.10121209</pub-id></citation>
</ref>
<ref id="B15">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Choi</surname> <given-names>J.</given-names></name> <name><surname>Wang</surname> <given-names>Z.</given-names></name> <name><surname>Venkataramani</surname> <given-names>S.</given-names></name> <name><surname>Chuang</surname> <given-names>P. I.-J.</given-names></name> <name><surname>Srinivasan</surname> <given-names>V.</given-names></name> <name><surname>Gopalakrishnan</surname> <given-names>K.</given-names></name> <etal/></person-group>. (<year>2018</year>). <article-title>Pact: parameterized clipping activation for quantized neural networks</article-title>. <source>arXiv [Preprint</source>]. arXiv:1805.06085. <pub-id pub-id-type="doi">10.48550/arXiv.1805.06085</pub-id></citation>
</ref>
<ref id="B16">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Cire&#x0015F;an</surname> <given-names>D. C.</given-names></name> <name><surname>Meier</surname> <given-names>U.</given-names></name> <name><surname>Gambardella</surname> <given-names>L. M.</given-names></name> <name><surname>Schmidhuber</surname> <given-names>J.</given-names></name></person-group> (<year>2010</year>). <article-title>Deep, big, simple neural nets for handwritten digit recognition</article-title>. <source>Neural Comput</source>. <volume>22</volume>, <fpage>3207</fpage>&#x02013;<lpage>3220</lpage>. <pub-id pub-id-type="doi">10.1162/NECO_a_00052</pub-id><pub-id pub-id-type="pmid">20858131</pub-id></citation></ref>
<ref id="B17">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Courbariaux</surname> <given-names>M.</given-names></name> <name><surname>Bengio</surname> <given-names>Y.</given-names></name> <name><surname>David</surname> <given-names>J.-P.</given-names></name></person-group> (<year>2015</year>). <article-title>&#x0201C;Binaryconnect: training deep neural networks with binary weights during propagations,&#x0201D;</article-title> in <source>Advances in Neural Information Processing System, 28</source> (<publisher-loc>Red Hook, NY</publisher-loc>: <publisher-name>Curran Associates</publisher-name>).</citation>
</ref>
<ref id="B18">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Deng</surname> <given-names>L.</given-names></name> <name><surname>Li</surname> <given-names>G.</given-names></name> <name><surname>Han</surname> <given-names>S.</given-names></name> <name><surname>Shi</surname> <given-names>L.</given-names></name> <name><surname>Xie</surname> <given-names>Y.</given-names></name></person-group> (<year>2020</year>). <article-title>Model compression and hardware acceleration for neural networks: a comprehensive survey</article-title>. <source>Proc. IEEE</source> <volume>108</volume>, <fpage>485</fpage>&#x02013;<lpage>532</lpage>. <pub-id pub-id-type="doi">10.1109/JPROC.2020.2976475</pub-id></citation>
</ref>
<ref id="B19">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Fick</surname> <given-names>L.</given-names></name> <name><surname>Blaauw</surname> <given-names>D.</given-names></name> <name><surname>Sylvester</surname> <given-names>D.</given-names></name> <name><surname>Skrzyniarz</surname> <given-names>S.</given-names></name> <name><surname>Parikh</surname> <given-names>M.</given-names></name> <name><surname>Fick</surname> <given-names>D.</given-names></name> <etal/></person-group>. (<year>2017</year>). <article-title>&#x0201C;Analog in-memory subthreshold deep neural network accelerator,&#x0201D;</article-title> in <source>2017 IEEE Custom Integrated Circuits Conference (CICC)</source> (<publisher-loc>Austin, TX</publisher-loc>: <publisher-name>IEEE</publisher-name>), <fpage>1</fpage>&#x02013;<lpage>4</lpage>. <pub-id pub-id-type="doi">10.1109/CICC.2017.7993629</pub-id></citation>
</ref>
<ref id="B20">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fick</surname> <given-names>L.</given-names></name> <name><surname>Skrzyniarz</surname> <given-names>S.</given-names></name> <name><surname>Parikh</surname> <given-names>M.</given-names></name> <name><surname>Henry</surname> <given-names>M. B.</given-names></name> <name><surname>Fick</surname> <given-names>D.</given-names></name></person-group> (<year>2022</year>). <article-title>&#x0201C;Analog matrix processor for edge AI real-time video analytics.&#x0201D;</article-title> in 2<italic>022 IEEE International Solid-State Circuits Conference (ISSCC), Vol. 65</italic> (San Francisco, CA: IEEE), <fpage>260</fpage>&#x02013;<lpage>262</lpage>. <pub-id pub-id-type="doi">10.1109/ISSCC42614.2022.9731773</pub-id></citation>
</ref>
<ref id="B21">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Freye</surname> <given-names>F.</given-names></name> <name><surname>Lou</surname> <given-names>J.</given-names></name> <name><surname>Bengel</surname> <given-names>C.</given-names></name> <name><surname>Menzel</surname> <given-names>S.</given-names></name> <name><surname>Wiefels</surname> <given-names>S.</given-names></name> <name><surname>Gemmeke</surname> <given-names>T.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>Memristive devices for time domain compute-in-memory. <italic>IEEE J. Explor</italic></article-title>. <source>Solid-State Comput. Devices Circuits</source> <volume>8</volume>, <fpage>119</fpage>&#x02013;<lpage>127</lpage>. <pub-id pub-id-type="doi">10.1109/JXCDC.2022.3217098</pub-id></citation>
</ref>
<ref id="B22">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Freye</surname> <given-names>F.</given-names></name> <name><surname>Lou</surname> <given-names>J.</given-names></name> <name><surname>Lanius</surname> <given-names>C.</given-names></name> <name><surname>Gemmeke</surname> <given-names>T.</given-names></name></person-group> (<year>2024</year>). <article-title>&#x0201C;Merits of time-domain computing for vmm-a quantitative comparison,&#x0201D;</article-title> in <source>2024 25th International Symposium on Quality Electronic Design (ISQED)</source> (<publisher-loc>San Francisco, CA</publisher-loc>: <publisher-name>IEEE</publisher-name>), <fpage>1</fpage>&#x02013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.1109/ISQED60706.2024.10528682</pub-id></citation>
</ref>
<ref id="B23">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gonugondla</surname> <given-names>S. K.</given-names></name> <name><surname>Sakr</surname> <given-names>C.</given-names></name> <name><surname>Dbouk</surname> <given-names>H.</given-names></name> <name><surname>Shanbhag</surname> <given-names>N. R.</given-names></name></person-group> (<year>2021</year>). <article-title>Fundamental limits on energy-delay-accuracy of in-memory architectures in inference applications. <italic>IEEE Trans. Comput.-Aided Des. Integr</italic></article-title>. <source>Circuits Syst</source>. <volume>41</volume>, <fpage>3188</fpage>&#x02013;<lpage>3201</lpage>. <pub-id pub-id-type="doi">10.1109/TCAD.2021.3124757</pub-id></citation>
</ref>
<ref id="B24">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Guo</surname> <given-names>X.</given-names></name> <name><surname>Bayat</surname> <given-names>F. M.</given-names></name> <name><surname>Bavandpour</surname> <given-names>M.</given-names></name> <name><surname>Klachko</surname> <given-names>M.</given-names></name> <name><surname>Mahmoodi</surname> <given-names>M.</given-names></name> <name><surname>Prezioso</surname> <given-names>M.</given-names></name> <etal/></person-group>. (<year>2017</year>). <article-title>&#x0201C;Fast, energy-efficient, robust, and reproducible mixed-signal neuromorphic classifier based on embedded nor flash memory technology,&#x0201D;</article-title> in <source>2017 IEEE International Electron Devices Meeting (IEDM)</source> (<publisher-loc>San Francisco, CA</publisher-loc>: <publisher-name>IEEE</publisher-name>), <fpage>6</fpage>&#x02013;<lpage>5</lpage>. <pub-id pub-id-type="doi">10.1109/IEDM.2017.8268341</pub-id></citation>
</ref>
<ref id="B25">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Gupta</surname> <given-names>S.</given-names></name> <name><surname>Agrawal</surname> <given-names>A.</given-names></name> <name><surname>Gopalakrishnan</surname> <given-names>K.</given-names></name> <name><surname>Narayanan</surname> <given-names>P.</given-names></name></person-group> (<year>2015</year>). <article-title>&#x0201C;Deep learning with limited numerical precision,&#x0201D;</article-title> in <source>International Conference on Machine Learning</source> (<publisher-loc>Lille</publisher-loc>: <publisher-name>JMLR.org</publisher-name>), <fpage>1737</fpage>&#x02013;<lpage>1746</lpage>.</citation>
</ref>
<ref id="B26">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hasler</surname> <given-names>J.</given-names></name> <name><surname>Marr</surname> <given-names>B.</given-names></name></person-group> (<year>2013</year>). <article-title>Finding a roadmap to achieve large neuromorphic hardware systems</article-title>. <source>Front. Neurosci</source>. <volume>7</volume>:<fpage>118</fpage>. <pub-id pub-id-type="doi">10.3389/fnins.2013.00118</pub-id><pub-id pub-id-type="pmid">24058330</pub-id></citation></ref>
<ref id="B27">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Horowitz</surname> <given-names>M.</given-names></name></person-group> (<year>2014</year>). <article-title>&#x0201C;1.1 computing&#x00027;s energy problem (and what we can do about it),&#x0201D;</article-title> in <source>2014 IEEE international solid-state circuits conference digest of technical papers (ISSCC)</source> (<publisher-loc>San Francisco, CA</publisher-loc>: <publisher-name>IEEE</publisher-name>), <fpage>10</fpage>&#x02013;<lpage>14</lpage>. <pub-id pub-id-type="doi">10.1109/ISSCC.2014.6757323</pub-id></citation>
</ref>
<ref id="B28">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Jia</surname> <given-names>H.</given-names></name> <name><surname>Ozatay</surname> <given-names>M.</given-names></name> <name><surname>Tang</surname> <given-names>Y.</given-names></name> <name><surname>Valavi</surname> <given-names>H.</given-names></name> <name><surname>Pathak</surname> <given-names>R.</given-names></name> <name><surname>Lee</surname> <given-names>J.</given-names></name> <etal/></person-group>. (<year>2021a</year>). <article-title>&#x0201C;15.1 a programmable neural-network inference accelerator based on scalable in-memory computing,&#x0201D;</article-title> in <source>2021 IEEE International Solid-State Circuits Conference (ISSCC), Vol. 64</source> (<publisher-loc>San Francisco, CA</publisher-loc>: <publisher-name>IEEE</publisher-name>), <fpage>236</fpage>&#x02013;<lpage>238</lpage>. <pub-id pub-id-type="doi">10.1109/ISSCC42613.2021.9365788</pub-id></citation>
</ref>
<ref id="B29">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Jia</surname> <given-names>H.</given-names></name> <name><surname>Ozatay</surname> <given-names>M.</given-names></name> <name><surname>Tang</surname> <given-names>Y.</given-names></name> <name><surname>Valavi</surname> <given-names>H.</given-names></name> <name><surname>Pathak</surname> <given-names>R.</given-names></name> <name><surname>Lee</surname> <given-names>J.</given-names></name> <etal/></person-group>. (<year>2021b</year>). <article-title>Scalable and programmable neural network inference accelerator based on in-memory computing</article-title>. <source>IEEE J. Solid-State Circuits</source> <volume>57</volume>, <fpage>198</fpage>&#x02013;<lpage>211</lpage>. <pub-id pub-id-type="doi">10.1109/JSSC.2021.3119018</pub-id></citation>
</ref>
<ref id="B30">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Jiang</surname> <given-names>H.</given-names></name> <name><surname>Huang</surname> <given-names>S.</given-names></name> <name><surname>Li</surname> <given-names>W.</given-names></name> <name><surname>Yu</surname> <given-names>S.</given-names></name></person-group> (<year>2022</year>). <article-title>Enna: An efficient neural network accelerator design based on adc-free compute-in-memory subarrays</article-title>. <source>IEEE Trans. Circuits Syst. I: Regul. Papers</source> <volume>70</volume>, <fpage>353</fpage>&#x02013;<lpage>363</lpage>. <pub-id pub-id-type="doi">10.1109/TCSI.2022.3208755</pub-id></citation>
</ref>
<ref id="B31">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Jiang</surname> <given-names>Z.</given-names></name> <name><surname>Yin</surname> <given-names>S.</given-names></name> <name><surname>Seo</surname> <given-names>J.-S.</given-names></name> <name><surname>Seok</surname> <given-names>M.</given-names></name></person-group> (<year>2020</year>). <article-title>C3sram: an in-memory-computing sram macro based on robust capacitive coupling computing mechanism</article-title>. <source>IEEE J. Solid-State Circuits</source> <volume>55</volume>, <fpage>1888</fpage>&#x02013;<lpage>1897</lpage>. <pub-id pub-id-type="doi">10.1109/JSSC.2020.2992886</pub-id></citation>
</ref>
<ref id="B32">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Joshi</surname> <given-names>V.</given-names></name> <name><surname>Le Gallo</surname> <given-names>M.</given-names></name> <name><surname>Haefeli</surname> <given-names>S.</given-names></name> <name><surname>Boybat</surname> <given-names>I.</given-names></name> <name><surname>Nandakumar</surname> <given-names>S. R.</given-names></name> <name><surname>Piveteau</surname> <given-names>C.</given-names></name> <etal/></person-group>. (<year>2020</year>). <article-title>Accurate deep neural network inference using computational phase-change memory</article-title>. <source>Nat. Commun</source>. <volume>11</volume>:<fpage>2473</fpage>. <pub-id pub-id-type="doi">10.1038/s41467-020-16108-9</pub-id><pub-id pub-id-type="pmid">32424184</pub-id></citation></ref>
<ref id="B33">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Khaddam-Aljameh</surname> <given-names>R.</given-names></name> <name><surname>Stanisavljevic</surname> <given-names>M.</given-names></name> <name><surname>Mas</surname> <given-names>J. F.</given-names></name> <name><surname>Karunaratne</surname> <given-names>G.</given-names></name> <name><surname>Br&#x000E4;ndli</surname> <given-names>M.</given-names></name> <name><surname>Liu</surname> <given-names>F.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>Hermes-core&#x02014;a 1.59-tops/mm 2 pcm on 14-nm cmos in-memory compute core using 300-ps/lsb linearized cco-based adcs</article-title>. <source>IEEE J. Solid-State Circuits</source> <volume>57</volume>, <fpage>1027</fpage>&#x02013;<lpage>1038</lpage>. <pub-id pub-id-type="doi">10.1109/JSSC.2022.3140414</pub-id></citation>
</ref>
<ref id="B34">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kingra</surname> <given-names>S. K.</given-names></name> <name><surname>Parmar</surname> <given-names>V.</given-names></name> <name><surname>Sharma</surname> <given-names>M.</given-names></name> <name><surname>Suri</surname> <given-names>M.</given-names></name></person-group> (<year>2022</year>). <article-title>Time-multiplexed in-memory computation scheme for mapping quantized neural networks on hybrid cmos-oxram building blocks</article-title>. <source>IEEE Trans. Nanotechnol</source>. <volume>21</volume>, <fpage>406</fpage>&#x02013;<lpage>412</lpage>. <pub-id pub-id-type="doi">10.1109/TNANO.2022.3193921</pub-id></citation>
</ref>
<ref id="B35">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kneip</surname> <given-names>A.</given-names></name> <name><surname>Bol</surname> <given-names>D.</given-names></name></person-group> (<year>2021</year>). <article-title>Impact of analog non-idealities on the design space of 6t-sram current-domain dot-product operators for in-memory computing</article-title>. <source>IEEE Trans. Circuits Sys. I: Regul. Papers</source> <volume>68</volume>, <fpage>1931</fpage>&#x02013;<lpage>1944</lpage>. <pub-id pub-id-type="doi">10.1109/TCSI.2021.3058510</pub-id></citation>
</ref>
<ref id="B36">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Krizhevsky</surname> <given-names>A.</given-names></name> <name><surname>Sutskever</surname> <given-names>I.</given-names></name> <name><surname>Hinton</surname> <given-names>G. E.</given-names></name></person-group> (<year>2012</year>). <article-title>&#x0201C;Imagenet classification with deep convolutional neural networks,&#x0201D;</article-title> in <source>Advances in Neural Information Processing System</source> (<publisher-loc>Red Hook, NY</publisher-loc>: <publisher-name>Curran Associates</publisher-name>), <fpage>25</fpage>.</citation>
</ref>
<ref id="B37">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Le Gallo</surname> <given-names>M.</given-names></name> <name><surname>Hrynkevych</surname> <given-names>O.</given-names></name> <name><surname>Kersting</surname> <given-names>B.</given-names></name> <name><surname>Karunaratne</surname> <given-names>G.</given-names></name> <name><surname>Vasilopoulos</surname> <given-names>A.</given-names></name> <name><surname>Khaddam-Aljameh</surname> <given-names>R.</given-names></name> <etal/></person-group>. (<year>2024</year>). <article-title>Demonstration of 4-quadrant analog in-memory matrix multiplication in a single modulation</article-title>. <source>Npj Unconv. Comput</source>. <volume>1</volume>:<fpage>11</fpage>. <pub-id pub-id-type="doi">10.1038/s44335-024-00010-4</pub-id><pub-id pub-id-type="pmid">39372606</pub-id></citation></ref>
<ref id="B38">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Le Gallo</surname> <given-names>M.</given-names></name> <name><surname>Khaddam-Aljameh</surname> <given-names>R.</given-names></name> <name><surname>Stanisavljevic</surname> <given-names>M.</given-names></name> <name><surname>Vasilopoulos</surname> <given-names>A.</given-names></name> <name><surname>Kersting</surname> <given-names>B.</given-names></name> <name><surname>Dazzi</surname> <given-names>M.</given-names></name> <etal/></person-group>. (<year>2023</year>). <article-title>A 64-core mixed-signal in-memory compute chip based on phase-change memory for deep neural network inference</article-title>. <source>Nat. Electron</source>. <volume>6</volume>, <fpage>680</fpage>&#x02013;<lpage>693</lpage>. <pub-id pub-id-type="doi">10.1038/s41928-023-01010-1</pub-id></citation>
</ref>
<ref id="B39">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>LeCun</surname> <given-names>Y.</given-names></name> <name><surname>Bengio</surname> <given-names>Y.</given-names></name> <name><surname>Hinton</surname> <given-names>G.</given-names></name></person-group> (<year>2015</year>). <article-title>Deep learning</article-title>. <source>Nature</source> <volume>521</volume>, <fpage>436</fpage>&#x02013;<lpage>444</lpage>. <pub-id pub-id-type="doi">10.1038/nature14539</pub-id><pub-id pub-id-type="pmid">26017442</pub-id></citation></ref>
<ref id="B40">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>LeCun</surname> <given-names>Y.</given-names></name> <name><surname>Bottou</surname> <given-names>L.</given-names></name> <name><surname>Bengio</surname> <given-names>Y.</given-names></name> <name><surname>Haffner</surname> <given-names>P.</given-names></name></person-group> (<year>2002</year>). <article-title>Gradient-based learning applied to document recognition</article-title>. <source>Proc. IEEE</source> <volume>86</volume>, <fpage>2278</fpage>&#x02013;<lpage>2324</lpage>. <pub-id pub-id-type="doi">10.1109/5.726791</pub-id></citation>
</ref>
<ref id="B41">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Lim</surname> <given-names>J.</given-names></name> <name><surname>Choi</surname> <given-names>M.</given-names></name> <name><surname>Liu</surname> <given-names>B.</given-names></name> <name><surname>Kang</surname> <given-names>T.</given-names></name> <name><surname>Li</surname> <given-names>Z.</given-names></name> <name><surname>Wang</surname> <given-names>Z.</given-names></name> <etal/></person-group>. (<year>2020</year>). <article-title>&#x0201C;AA-ResNet: energy efficient all-analog resnet accelerator,&#x0201D;</article-title> in <source>2020 IEEE 63rd International Midwest Symposium on Circuits and Systems (MWSCAS)</source> (<publisher-loc>Springfield, MA</publisher-loc>: <publisher-name>IEEE</publisher-name>), <fpage>603</fpage>&#x02013;<lpage>606</lpage>. <pub-id pub-id-type="doi">10.1109/MWSCAS48704.2020.9184587</pub-id></citation>
</ref>
<ref id="B42">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Maass</surname> <given-names>W.</given-names></name></person-group> (<year>1997a</year>). <article-title>Fast sigmoidal networks via spiking neurons</article-title>. <source>Neural Comput</source>. <volume>9</volume>, <fpage>279</fpage>&#x02013;<lpage>304</lpage>. <pub-id pub-id-type="doi">10.1162/neco.1997.9.2.279</pub-id><pub-id pub-id-type="pmid">9117904</pub-id></citation></ref>
<ref id="B43">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Maass</surname> <given-names>W.</given-names></name></person-group> (<year>1997b</year>). <article-title>Networks of spiking neurons: the third generation of neural network models</article-title>. <source>Neural Netw</source>. <volume>10</volume>, <fpage>1659</fpage>&#x02013;<lpage>1671</lpage>. <pub-id pub-id-type="doi">10.1016/S0893-6080(97)00011-7</pub-id></citation>
</ref>
<ref id="B44">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Maass</surname> <given-names>W.</given-names></name></person-group> (<year>1999</year>). <article-title>Computing with spiking neurons</article-title>. <source>Pulsed Neural Netw</source>. <volume>2</volume>, <fpage>55</fpage>&#x02013;<lpage>85</lpage>. <pub-id pub-id-type="doi">10.7551/mitpress/5704.003.0006</pub-id></citation>
</ref>
<ref id="B45">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Mahmoodi</surname> <given-names>M. R.</given-names></name> <name><surname>Strukov</surname> <given-names>D.</given-names></name></person-group> (<year>2018</year>). <article-title>&#x0201C;An ultra-low energy internally analog, externally digital vector-matrix multiplier based on nor flash memory technology,&#x0201D;</article-title> in <source>Proceedings of the 55th Annual Design Automation Conference</source> (<publisher-loc>New York, NY</publisher-loc>: <publisher-name>ACM</publisher-name>), <fpage>1</fpage>&#x02013;<lpage>6</lpage>. <pub-id pub-id-type="doi">10.1145/3195970.3195989</pub-id></citation>
</ref>
<ref id="B46">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Marinella</surname> <given-names>M. J.</given-names></name> <name><surname>Agarwal</surname> <given-names>S.</given-names></name> <name><surname>Hsia</surname> <given-names>A.</given-names></name> <name><surname>Richter</surname> <given-names>I.</given-names></name> <name><surname>Jacobs-Gedrim</surname> <given-names>R.</given-names></name> <name><surname>Niroula</surname> <given-names>J.</given-names></name> <etal/></person-group>. (<year>2018</year>). <article-title>Multiscale co-design analysis of energy, latency, area, and accuracy of a reram analog neural training accelerator</article-title>. <source>IEEE J. Emerg. Sel. Top. Circuits Syst</source>. <volume>8</volume>, <fpage>86</fpage>&#x02013;<lpage>101</lpage>. <pub-id pub-id-type="doi">10.1109/JETCAS.2018.2796379</pub-id></citation>
</ref>
<ref id="B47">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>McKinstry</surname> <given-names>J. L.</given-names></name> <name><surname>Esser</surname> <given-names>S. K.</given-names></name> <name><surname>Appuswamy</surname> <given-names>R.</given-names></name> <name><surname>Bablani</surname> <given-names>D.</given-names></name> <name><surname>Arthur</surname> <given-names>J. V.</given-names></name> <name><surname>Yildiz</surname> <given-names>I. B.</given-names></name> <etal/></person-group>. (<year>2018</year>). <article-title>Discovering low-precision networks close to full-precision networks for efficient embedded inference</article-title>. <source>arXiv [preprint</source>] arXiv:1809.04191. <pub-id pub-id-type="doi">10.48550/arXiv.1809.04191</pub-id></citation>
</ref>
<ref id="B48">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Morie</surname> <given-names>T.</given-names></name> <name><surname>Liang</surname> <given-names>H.</given-names></name> <name><surname>Tohara</surname> <given-names>T.</given-names></name> <name><surname>Tanaka</surname> <given-names>H.</given-names></name> <name><surname>Igarashi</surname> <given-names>M.</given-names></name> <name><surname>Samukawa</surname> <given-names>S.</given-names></name> <etal/></person-group>. (<year>2016</year>). <article-title>&#x0201C;Spike-based time-domain weighted-sum calculation using nanodevices for low power operation,&#x0201D;</article-title> in <source>2016 IEEE 16th International Conference on Nanotechnology (IEEE-NANO)</source> (<publisher-loc>Sendai</publisher-loc>: <publisher-name>IEEE</publisher-name>), <fpage>390</fpage>&#x02013;<lpage>392</lpage>. <pub-id pub-id-type="doi">10.1109/NANO.2016.7751490</pub-id></citation>
</ref>
<ref id="B49">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Morie</surname> <given-names>T.</given-names></name> <name><surname>Sun</surname> <given-names>Y.</given-names></name> <name><surname>Liang</surname> <given-names>H.</given-names></name> <name><surname>Igarashi</surname> <given-names>M.</given-names></name> <name><surname>Huang</surname> <given-names>C.-H.</given-names></name> <name><surname>Samukawa</surname> <given-names>S.</given-names></name> <etal/></person-group>. (<year>2010</year>). <article-title>&#x0201C;A 2-dimensional si nanodisk array structure for spiking neuron models,&#x0201D;</article-title> in <source>Proceedings of 2010 IEEE International Symposium on Circuits and Systems</source> (<publisher-loc>Paris</publisher-loc>: <publisher-name>IEEE</publisher-name>), <fpage>781</fpage>&#x02013;<lpage>784</lpage>. <pub-id pub-id-type="doi">10.1109/ISCAS.2010.5537456</pub-id></citation>
</ref>
<ref id="B50">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>N&#x000E4;gele</surname> <given-names>R.</given-names></name> <name><surname>Finkbeiner</surname> <given-names>J.</given-names></name> <name><surname>Stadtlander</surname> <given-names>V.</given-names></name> <name><surname>Gr&#x000F6;zing</surname> <given-names>M.</given-names></name> <name><surname>Berroth</surname> <given-names>M.</given-names></name></person-group> (<year>2023</year>). <article-title>Analog multiply-accumulate cell with multi-bit resolution for all-analog AI inference accelerators</article-title>. <source>IEEE Trans. Circuits Syst. I: Regul. Papers</source> <volume>70</volume>, <fpage>3509</fpage>&#x02013;<lpage>3521</lpage>. <pub-id pub-id-type="doi">10.1109/TCSI.2023.3268728</pub-id></citation>
</ref>
<ref id="B51">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Nair</surname> <given-names>V.</given-names></name> <name><surname>Hinton</surname> <given-names>G. E.</given-names></name></person-group> (<year>2010</year>). <article-title>&#x0201C;Rectified linear units improve restricted Boltzmann machines,&#x0201D;</article-title> in <source>Proceedings of the 27th International Conference on Machine Learning (ICML-10)</source> (<publisher-loc>Madison, WI</publisher-loc>: <publisher-name>Omnipress</publisher-name>), <fpage>807</fpage>&#x02013;<lpage>814</lpage>.</citation>
</ref>
<ref id="B52">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Narayanan</surname> <given-names>P.</given-names></name> <name><surname>Ambrogio</surname> <given-names>S.</given-names></name> <name><surname>Okazaki</surname> <given-names>A.</given-names></name> <name><surname>Hosokawa</surname> <given-names>K.</given-names></name> <name><surname>Tsai</surname> <given-names>H.</given-names></name> <name><surname>Nomura</surname> <given-names>A.</given-names></name> <etal/></person-group>. (<year>2021</year>). <article-title>Fully on-chip mac at 14 nm enabled by accurate row-wise programming of pcm-based weights and parallel vector-transport in duration-format</article-title>. <source>IEEE Trans. Electron Devices</source> <volume>68</volume>, <fpage>6629</fpage>&#x02013;<lpage>6636</lpage>. <pub-id pub-id-type="doi">10.1109/TED.2021.3115993</pub-id></citation>
</ref>
<ref id="B53">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ngo</surname> <given-names>D.</given-names></name> <name><surname>Park</surname> <given-names>H.-C.</given-names></name> <name><surname>Kang</surname> <given-names>B.</given-names></name></person-group> (<year>2025</year>). <article-title>Edge intelligence: a review of deep neural network inference in resource-limited environments</article-title>. <source>Electronics</source> <volume>14</volume>:<fpage>2495</fpage>. <pub-id pub-id-type="doi">10.3390/electronics14122495</pub-id></citation>
</ref>
<ref id="B54">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Prezioso</surname> <given-names>M.</given-names></name> <name><surname>Merrikh-Bayat</surname> <given-names>F.</given-names></name> <name><surname>Hoskins</surname> <given-names>B. D.</given-names></name> <name><surname>Adam</surname> <given-names>G. C.</given-names></name> <name><surname>Likharev</surname> <given-names>K. K.</given-names></name> <name><surname>Strukov</surname> <given-names>D. B.</given-names></name> <etal/></person-group>. (<year>2015</year>). <article-title>Training and operation of an integrated neuromorphic network based on metal-oxide memristors</article-title>. <source>Nature</source> <volume>521</volume>, <fpage>61</fpage>&#x02013;<lpage>64</lpage>. <pub-id pub-id-type="doi">10.1038/nature14441</pub-id><pub-id pub-id-type="pmid">25951284</pub-id></citation></ref>
<ref id="B55">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Roy</surname> <given-names>K.</given-names></name> <name><surname>Jaiswal</surname> <given-names>A.</given-names></name> <name><surname>Panda</surname> <given-names>P.</given-names></name></person-group> (<year>2019</year>). <article-title>Towards spike-based machine intelligence with neuromorphic computing</article-title>. <source>Nature</source> <volume>575</volume>, <fpage>607</fpage>&#x02013;<lpage>617</lpage>. <pub-id pub-id-type="doi">10.1038/s41586-019-1677-2</pub-id><pub-id pub-id-type="pmid">31776490</pub-id></citation></ref>
<ref id="B56">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sahay</surname> <given-names>S.</given-names></name> <name><surname>Bavandpour</surname> <given-names>M.</given-names></name> <name><surname>Mahmoodi</surname> <given-names>M. R.</given-names></name> <name><surname>Strukov</surname> <given-names>D.</given-names></name></person-group> (<year>2020</year>). <article-title>Energy-efficient moderate precision time-domain mixed-signal vector-by-matrix multiplier exploiting 1t-1r arrays</article-title>. <source>IEEE J. Explor. Solid-State Comput. Devices Circuits</source> <volume>6</volume>, <fpage>18</fpage>&#x02013;<lpage>26</lpage>. <pub-id pub-id-type="doi">10.1109/JXCDC.2020.2981048</pub-id></citation>
</ref>
<ref id="B57">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Schlottmann</surname> <given-names>C. R.</given-names></name> <name><surname>Hasler</surname> <given-names>P. E.</given-names></name></person-group> (<year>2011</year>). <article-title>A highly dense, low power, programmable analog vector-matrix multiplier: The FPAA implementation</article-title>. <source>IEEE J. Emer. Sel. Top. Circuits Syst</source>. <volume>1</volume>, <fpage>403</fpage>&#x02013;<lpage>411</lpage>. <pub-id pub-id-type="doi">10.1109/JETCAS.2011.2165755</pub-id></citation>
</ref>
<ref id="B58">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sebastian</surname> <given-names>A.</given-names></name> <name><surname>Le Gallo</surname> <given-names>M.</given-names></name> <name><surname>Khaddam-Aljameh</surname> <given-names>R.</given-names></name> <name><surname>Eleftheriou</surname> <given-names>E.</given-names></name></person-group> (<year>2020</year>). <article-title>Memory devices and applications for in-memory computing</article-title>. <source>Nat. Nanotechnol</source>. <volume>15</volume>, <fpage>529</fpage>&#x02013;<lpage>544</lpage>. <pub-id pub-id-type="doi">10.1038/s41565-020-0655-z</pub-id><pub-id pub-id-type="pmid">32231270</pub-id></citation></ref>
<ref id="B59">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Seo</surname> <given-names>J.-O.</given-names></name> <name><surname>Seok</surname> <given-names>M.</given-names></name> <name><surname>Cho</surname> <given-names>S.</given-names></name></person-group> (<year>2022</year>). <article-title>&#x0201C;Archon: a 332.7 tops/w 5b variation-tolerant analog cnn processor featuring analog neuronal computation unit and analog memory,&#x0201D;</article-title> in <source>2022 IEEE International Solid-State Circuits Conference (ISSCC), Volume 65</source> (<publisher-loc>San Francisco, CA</publisher-loc>: <publisher-name>IEEE</publisher-name>), <fpage>258</fpage>&#x02013;<lpage>260</lpage>. <pub-id pub-id-type="doi">10.1109/ISSCC42614.2022.9731654</pub-id></citation>
</ref>
<ref id="B60">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Shafiee</surname> <given-names>A.</given-names></name> <name><surname>Nag</surname> <given-names>A.</given-names></name> <name><surname>Muralimanohar</surname> <given-names>N.</given-names></name> <name><surname>Balasubramonian</surname> <given-names>R.</given-names></name> <name><surname>Strachan</surname> <given-names>J. P.</given-names></name> <name><surname>Hu</surname> <given-names>M.</given-names></name> <etal/></person-group>. (<year>2016</year>). <article-title>ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars</article-title>. <source>ACM SIGARCH Comput. Archit. News</source> <volume>44</volume>, <fpage>14</fpage>&#x02013;<lpage>26</lpage>. <pub-id pub-id-type="doi">10.1145/3007787.3001139</pub-id></citation>
</ref>
<ref id="B61">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Shukla</surname> <given-names>S.</given-names></name> <name><surname>Fleischer</surname> <given-names>B.</given-names></name> <name><surname>Ziegler</surname> <given-names>M.</given-names></name> <name><surname>Silberman</surname> <given-names>J.</given-names></name> <name><surname>Oh</surname> <given-names>J.</given-names></name> <name><surname>Srinivasan</surname> <given-names>V.</given-names></name> <etal/></person-group>. (<year>2019</year>). <article-title>A scalable multi-teraops core for AI training and inference</article-title>. <source>IEEE Solid-State Circuits Lett</source>. <volume>1</volume>, <fpage>217</fpage>&#x02013;<lpage>220</lpage>. <pub-id pub-id-type="doi">10.1109/LSSC.2019.2902738</pub-id></citation>
</ref>
<ref id="B62">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Shuvo</surname> <given-names>M. M. H.</given-names></name> <name><surname>Islam</surname> <given-names>S. K.</given-names></name> <name><surname>Cheng</surname> <given-names>J.</given-names></name> <name><surname>Morshed</surname> <given-names>B. I.</given-names></name></person-group> (<year>2022</year>). <article-title>Efficient acceleration of deep learning inference on resource-constrained edge devices: a review</article-title>. <source>Proc. IEEE</source> <volume>111</volume>, <fpage>42</fpage>&#x02013;<lpage>91</lpage>. <pub-id pub-id-type="doi">10.1109/JPROC.2022.3226481</pub-id></citation>
</ref>
<ref id="B63">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sze</surname> <given-names>V.</given-names></name> <name><surname>Chen</surname> <given-names>Y.-H.</given-names></name> <name><surname>Yang</surname> <given-names>T.-J.</given-names></name> <name><surname>Emer</surname> <given-names>J. S.</given-names></name></person-group> (<year>2017</year>). <article-title>Efficient processing of deep neural networks: a tutorial and survey</article-title>. <source>Proc. IEEE</source> <volume>105</volume>, <fpage>2295</fpage>&#x02013;<lpage>2329</lpage>. <pub-id pub-id-type="doi">10.1109/JPROC.2017.2761740</pub-id></citation>
</ref>
<ref id="B64">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tohara</surname> <given-names>T.</given-names></name> <name><surname>Liang</surname> <given-names>H.</given-names></name> <name><surname>Tanaka</surname> <given-names>H.</given-names></name> <name><surname>Igarashi</surname> <given-names>M.</given-names></name> <name><surname>Samukawa</surname> <given-names>S.</given-names></name> <name><surname>Endo</surname> <given-names>K.</given-names></name> <etal/></person-group>. (<year>2016</year>). <article-title>Silicon nanodisk array with a fin field-effect transistor for time-domain weighted sum calculation toward massively parallel spiking neural networks</article-title>. <source>Appl. Phys. Express</source> <volume>9</volume>:<fpage>034201</fpage>. <pub-id pub-id-type="doi">10.7567/APEX.9.034201</pub-id></citation>
</ref>
<ref id="B65">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tsai</surname> <given-names>H.</given-names></name> <name><surname>Ambrogio</surname> <given-names>S.</given-names></name> <name><surname>Narayanan</surname> <given-names>P.</given-names></name> <name><surname>Shelby</surname> <given-names>R. M.</given-names></name> <name><surname>Burr</surname> <given-names>G. W.</given-names></name></person-group> (<year>2018</year>). <article-title>Recent progress in analog memory-based accelerators for deep learning</article-title>. <source>J. Phys. D Appl. Phys</source>. <volume>51</volume>:<fpage>283001</fpage>. <pub-id pub-id-type="doi">10.1088/1361-6463/aac8a5</pub-id></citation>
</ref>
<ref id="B66">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Valavi</surname> <given-names>H.</given-names></name> <name><surname>Ramadge</surname> <given-names>P. J.</given-names></name> <name><surname>Nestler</surname> <given-names>E.</given-names></name> <name><surname>Verma</surname> <given-names>N.</given-names></name></person-group> (<year>2019</year>). <article-title>A 64-tile 2.4-mb in-memory-computing cnn accelerator employing charge-domain compute</article-title>. <source>IEEE J. Solid-State Circuits</source> <volume>54</volume>, <fpage>1789</fpage>&#x02013;<lpage>1799</lpage>. <pub-id pub-id-type="doi">10.1109/JSSC.2019.2899730</pub-id></citation>
</ref>
<ref id="B67">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Verma</surname> <given-names>N.</given-names></name> <name><surname>Jia</surname> <given-names>H.</given-names></name> <name><surname>Valavi</surname> <given-names>H.</given-names></name> <name><surname>Tang</surname> <given-names>Y.</given-names></name> <name><surname>Ozatay</surname> <given-names>M.</given-names></name> <name><surname>Chen</surname> <given-names>L.-Y.</given-names></name> <etal/></person-group>. (<year>2019</year>). <article-title>In-memory computing: advances and prospects</article-title>. <source>IEEE Solid-State Circuits Mag</source>. <volume>11</volume>, <fpage>43</fpage>&#x02013;<lpage>55</lpage>. <pub-id pub-id-type="doi">10.1109/MSSC.2019.2922889</pub-id></citation>
</ref>
<ref id="B68">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wan</surname> <given-names>W.</given-names></name> <name><surname>Kubendran</surname> <given-names>R.</given-names></name> <name><surname>Schaefer</surname> <given-names>C.</given-names></name> <name><surname>Eryilmaz</surname> <given-names>S. B.</given-names></name> <name><surname>Zhang</surname> <given-names>W.</given-names></name> <name><surname>Wu</surname> <given-names>D.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>A compute-in-memory chip based on resistive random-access memory</article-title>. <source>Nature</source> <volume>608</volume>, <fpage>504</fpage>&#x02013;<lpage>512</lpage>. <pub-id pub-id-type="doi">10.1038/s41586-022-04992-8</pub-id><pub-id pub-id-type="pmid">35978128</pub-id></citation></ref>
<ref id="B69">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wang</surname> <given-names>L.</given-names></name> <name><surname>Ye</surname> <given-names>W.</given-names></name> <name><surname>Dou</surname> <given-names>C.</given-names></name> <name><surname>Si</surname> <given-names>X.</given-names></name> <name><surname>Xu</surname> <given-names>X.</given-names></name> <name><surname>Liu</surname> <given-names>J.</given-names></name> <etal/></person-group>. (<year>2021</year>). <article-title>Efficient and robust nonvolatile computing-in-memory based on voltage division in 2t2r rram with input-dependent sensing control</article-title>. <source>IEEE Trans. Circuits Syst. II: Express Briefs</source> <volume>68</volume>, <fpage>1640</fpage>&#x02013;<lpage>1644</lpage>. <pub-id pub-id-type="doi">10.1109/TCSII.2021.3067385</pub-id></citation>
</ref>
<ref id="B70">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Wang</surname> <given-names>Q.</given-names></name> <name><surname>Tamukoh</surname> <given-names>H.</given-names></name> <name><surname>Morie</surname> <given-names>T.</given-names></name></person-group> (<year>2016</year>). <article-title>&#x0201C;Time-domain weighted-sum calculation for ultimately low power vlsi neural networks,&#x0201D;</article-title> in <source>International Conference on Neural Information Processing</source> (<publisher-loc>Cham</publisher-loc>: <publisher-name>Springer</publisher-name>), <fpage>240</fpage>&#x02013;<lpage>247</lpage>. <pub-id pub-id-type="doi">10.1007/978-3-319-46687-3_26</pub-id></citation>
</ref>
<ref id="B71">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wang</surname> <given-names>Q.</given-names></name> <name><surname>Tamukoh</surname> <given-names>H.</given-names></name> <name><surname>Morie</surname> <given-names>T.</given-names></name></person-group> (<year>2018</year>). <article-title>A time-domain analog weighted-sum calculation model for extremely low power vlsi implementation of multi-layer neural networks</article-title>. <source>arXiv [preprint</source>]. arXiv:1810.06819.<pub-id pub-id-type="doi">10.48550/arXiv:1810.06819</pub-id></citation>
</ref>
<ref id="B72">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Wang</surname> <given-names>S.</given-names></name> <name><surname>Zhou</surname> <given-names>T.</given-names></name> <name><surname>Bilmes</surname> <given-names>J.</given-names></name></person-group> (<year>2019</year>). <article-title>&#x0201C;Bias also matters: bias attribution for deep neural network explanation,&#x0201D;</article-title> in <source>International Conference on Machine Learning</source> (<publisher-loc>Long Beach, CA</publisher-loc>), <fpage>6659</fpage>&#x02013;<lpage>6667</lpage>.</citation>
</ref>
<ref id="B73">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Wu</surname> <given-names>P.-C.</given-names></name> <name><surname>Su</surname> <given-names>J.-W.</given-names></name> <name><surname>Chung</surname> <given-names>Y.-L.</given-names></name> <name><surname>Hong</surname> <given-names>L.-Y.</given-names></name> <name><surname>Ren</surname> <given-names>J.-S.</given-names></name> <name><surname>Chang</surname> <given-names>F.-C.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>&#x0201C;A 28nm 1mb time-domain computing-in-memory 6t-sram macro with a 6.6 ns latency, 1241gops and 37.01 tops/w for 8b-mac operations for edge-AI devices,&#x0201D;</article-title> in <source>2022 IEEE International Solid-State Circuits Conference (ISSCC), Volume 65</source> (<publisher-loc>San Francisco, CA</publisher-loc>: <publisher-name>IEEE</publisher-name>), <fpage>1</fpage>&#x02013;<lpage>3</lpage>. <pub-id pub-id-type="doi">10.1109/ISSCC42614.2022.9731681</pub-id></citation>
</ref>
<ref id="B74">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Xiao</surname> <given-names>T. P.</given-names></name> <name><surname>Feinberg</surname> <given-names>B.</given-names></name> <name><surname>Bennett</surname> <given-names>C. H.</given-names></name> <name><surname>Prabhakar</surname> <given-names>V.</given-names></name> <name><surname>Saxena</surname> <given-names>P.</given-names></name> <name><surname>Agrawal</surname> <given-names>V.</given-names></name> <etal/></person-group>. (<year>2023</year>). <article-title>On the accuracy of analog neural network inference accelerators</article-title>. <source>IEEE Circuits Syst. Mag</source>. <volume>22</volume>, <fpage>26</fpage>&#x02013;<lpage>48</lpage>. <pub-id pub-id-type="doi">10.1109/MCAS.2022.3214409</pub-id></citation>
</ref>
<ref id="B75">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Yamaguchi</surname> <given-names>M.</given-names></name> <name><surname>Iwamoto</surname> <given-names>G.</given-names></name> <name><surname>Nishimura</surname> <given-names>Y.</given-names></name> <name><surname>Tamukoh</surname> <given-names>H.</given-names></name> <name><surname>Morie</surname> <given-names>T.</given-names></name></person-group> (<year>2020</year>). <article-title>An energy-efficient time-domain analog cmos binaryconnect neural network processor based on a pulse-width modulation approach</article-title>. <source>IEEE Access</source> <volume>9</volume>, <fpage>2644</fpage>&#x02013;<lpage>2654</lpage>. <pub-id pub-id-type="doi">10.1109/ACCESS.2020.3047619</pub-id></citation>
</ref>
<ref id="B76">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Yang</surname> <given-names>J.</given-names></name> <name><surname>Kong</surname> <given-names>Y.</given-names></name> <name><surname>Wang</surname> <given-names>Z.</given-names></name> <name><surname>Liu</surname> <given-names>Y.</given-names></name> <name><surname>Wang</surname> <given-names>B.</given-names></name> <name><surname>Yin</surname> <given-names>S.</given-names></name> <etal/></person-group>. (<year>2019</year>). <article-title>&#x0201C;24.4 sandwich-ram: an energy-efficient in-memory bwn architecture with pulse-width modulation,&#x0201D;</article-title> in <source>2019 IEEE International Solid-State Circuits Conference-(ISSCC)</source> (<publisher-loc>San Francisco, CA</publisher-loc>: <publisher-name>IEEE</publisher-name>), <fpage>394</fpage>&#x02013;<lpage>396</lpage>. <pub-id pub-id-type="doi">10.1109/ISSCC.2019.8662435</pub-id></citation>
</ref>
<ref id="B77">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zhang</surname> <given-names>M.</given-names></name> <name><surname>Wang</surname> <given-names>J.</given-names></name> <name><surname>Wu</surname> <given-names>J.</given-names></name> <name><surname>Belatreche</surname> <given-names>A.</given-names></name> <name><surname>Amornpaisannon</surname> <given-names>B.</given-names></name> <name><surname>Zhang</surname> <given-names>Z.</given-names></name> <etal/></person-group>. (<year>2021</year>). <article-title>Rectified linear postsynaptic potential function for backpropagation in deep spiking neural networks</article-title>. <source>IEEE Trans. Neural Netw. Learn. Syst</source>. <volume>33</volume>, <fpage>1947</fpage>&#x02013;<lpage>1958</lpage>. <pub-id pub-id-type="doi">10.1109/TNNLS.2021.3110991</pub-id><pub-id pub-id-type="pmid">34534091</pub-id></citation></ref>
</ref-list>
</back>
</article>