Reservoir computing is a widely used machine learning framework known for its non-linear and dynamic characteristics (Yan et al., 2024). This approach facilitates the handling of complex temporal patterns without the need for extensive retraining or parameter adjustments inherent in traditional models (Cucchi et al., 2022). By leveraging a fixed number of randomly generated neurons in the reservoir to extract information in high dimensions, it allows for efficient computation and robust performance across a variety of temporal-spatial tasks (Tanaka et al., 2019). For hardware implementation, the memristor is a popular candidate to work as the physical reservoir due to its non-linear and dynamic programming properties (Cao et al., 2022). The memristor has shown its unique short-term or long-term memory with both optical and electrical programming abilities. It can work as the reservoir nodes to perform non-linear information transformation (encoding) of the temporal input data into the stored reservoir states (Zhang et al., 2023). In such systems, the reservoir node encodes the spatiotemporal information naturally by device dynamics, which eliminates the need for external memory or arithmetic and logic units (ALUs). By implementing physical reservoir with memristors, it can achieve outstanding energy efficiency and power consumption as the number of nodes is fixed.

The STM characteristic of the memristor brings its non-linear and time-dependent properties, which can serve as a suitable hardware platform for the dynamics required for reservoir computing. Many studies have shown the potential ability to utilize memristor-based circuit as the reservoir node and take advantage of the storage ability (Liang et al., 2024; Yan et al., 2024; Yang et al., 2024). Wu et al. (2023) integrated organic light-sensing materials into memristors to simulate an optical reservoir system with a spike neuromorphic network for pattern recognition. Yang et al. (2024) proposed a mask reservoir computing circuit using memristor arrays to process analog biomedical signals. Zhou et al. (2023) proposed light-responsive vision sensors that convert dynamic motion into event signals and extract features for efficient motion recognition. These studies have integrated memristors into reservoir computing platforms, offering potential opportunities for hardware-based reservoir computing applications.

However, most of the existing studies primarily concentrate on device characteristics and software simulations. In this study, we provide a general event representation method via memristor modeling and circuit design for reservoir computing, which enhances the event-based feature extraction with low energy consumption.

The memristor circuit unit plays as the basic element of reservoir node for event data interpretation. It is used to extract high dimension information and behave as non-linear state of reservoir. The DVS camera measures output with event behavior through the brightness change (Lichtsteiner et al., 2008). The pixel circuit is shown in Figure 2A, where the brightness change over threshold value of comparator is generated as ON and OFF event. The asynchronous event behaviors can be directly fed and processed by the proposed reservoir circuit as shown in Figure 2B. The ON and OFF signal represents the digital control signal of event data, while the V_in is the constant voltage supply for memristor programming. By definition of event data, the ON and OFF signal cannot be activated simultaneously as brightness change can be either positive or negative. The schematic diagram is shown in Figures 2C, D when ON and OFF signal is activated at a high level. The event data can drive the switch control of voltage, where the memristor state changes accordingly with the event type and stores the previous integrated information. The bipolar programming of the memristor device provides the flexibility of reservoir state modification with non-linear dynamics. According to the memristor readout, the multiplexer with clock control is utilized to read the conductance of each memristor simultaneously. During the read phase, a small voltage (< 0.15 V) is applied across the memristor to prevent any alteration of its conductance state. A transimpedance amplifier (TIA) can be utilized to convert the small input current into a proportional voltage signal for further processing.

The VTEAM model is a popular voltage-controlled model considering the threshold voltage phenomenon, which is compatible with many window functions (Kvatinsky et al., 2015). In addition, the VTEAM model has great flexibility to simulate the non-linear dopant drift phenomenon. Considering the STM behavior in many solid-state memory devices, we improve the memristor model by adding a decay term below the threshold value. The optimized VTEAM model is shown below:

where w(t) is an internal state variable in [0, W], W is a constant that represents the maximum value of w, x(t) is an internal state variable in [0, 1], τ is the constant decay value, f(x) is the window function, v(t) is the voltage applied to the memristor, i(t) is the current passing through the memristor, R(t) is the resistance of the memristor, and t is the time. The parameters v_on and v_off are threshold voltages, and R_on and R_off are resistance values of the memristor in its ON and OFF states. The parameters k_on, k_off, α_on, and α_off are constants. The window function f(x) is applied to model the non-linear drift behavior with scalable parameters (Soni and Sahoo, 2022).

Considering the dynamic behavior in the time surface, the state change with the memristor can be modeled as follows:

In the VTEAM model, the memristor's resistance change is primarily governed by the direction and magnitude of the voltage. When the memristor device is applied under serial constant voltage pulses δ_spk(t) over than threshold v_on and v_off, the internal state variable w behaves as below:

where c₁ and c₂ denote constants defining how much the resistance changes within the time according to pulse state. The state change △R is dependent on internal state w and can be expressed as below:

where 1 is the indicator function that equals 1 if δ_spk(t) = 0 and takes the value 0 in other cases, and f(x) is the appeid window function. In this case, we consider the boundary effect and apply Biolek's window function (Biolek et al., 2009), which is given by f(x) = 1−(x−stp(−i))^2p, where stp is the step function. We substitute the parameter x with Equation 3, and the window function can be expressed as follows:

Our model in NEIR method is derived from the VTEAM model yet it is significantly simplified. Compared with the other existing memristor model, our model focuses exclusively on pulse behavior instead of I-V behavior. It has the much simplified model with less fitting parameters (4 in total), which is easy to use in pulse simulation. This approach relies on the memristor's response under the pulse stimulus, which is a more intuitive framework that prioritizes efficiency in electrical pulse manipulation. This direct and effective method provides a particular advantage for simulating the dynamic memristor property for reservoir computing scenarios, where time-dependent spike data can be treated as pulse stimulus. By mirroring the natural processing mechanisms in biological neurons, this method allows for physical implementation in memristors responding to electrical or optical impulses over time.

To demonstrate the effectiveness and precision of modeling the memristor device property, we conducted a comparative simulation using serials of random pulses with SPICE-based model and our simplified model as shown in Figure 3A. Based on the proposed reservoir node circuit, the SPICE simulation is conducted with the modified VTEAM model. The simplified model aims to capture the pulse behavior of the memristor state with simulation frequency of 200 Hz. For demonstration purposes, we increase the event frequency and decay penalty, whereas in real scenarios, events usually fire at low rates with sparsity. The well-fitting result indicates that our simplified model can accurately extract spike-based features and retain critical dynamics, which makes it a valuable tool for fast and efficient modeling. The parameters of the simplified memristor behavior simulation in this study are listed in Figure 3B. From Equation 7, it shows the strong non-linear relationship between programming pulses and the resistance value of the memristor, which is essential for the reservoir computing simulation due to the non-linearity and dynamics. The simulated memristor model for a pair of positive and negative pulses is shown in Figures 3C, D. The resistance change curve shows a non-linear manner, where state variable x gradually increases with positive pulses and decreases with negative pulses. The resistance is limited between R_on and R_off with boundary effect constraint. The non-linearity value of the curve can be easily modified by two parameters c1 and c2 or through curve fitting with experimental memristor data. The model is simplified and purely based on the pulse mode with few parameters, which is easy to use in non-linear fitting cases.

The event behavior can be described and transmitted with three essential parameters, including time step, pixel coordinates (x, y position), and event polarity (Leñero-Bardallo et al., 2018). The event behavior can be expressed as

where x_i and y_i are the positions of an active pixel, p_i represents the polarity of an event, t_i is the time step, and e_i is the ith event in the stream. For the reservoir encoding method, the memristor array updates its conductance state according to the event streams. We partition the event recordings into smaller chunks and perform feature extraction by reading the current periodically at each segment. These patterns in an analog state can be processed by subsequent DNN/SNN blocks. The proposed encoding method is shown in Algorithm 1.

Event data are processed and transformed into alternative representations to extract significant information and facilitate specified algorithm processing. These representation methods focus on converting raw event data into a format that is more amenable to algorithmic processing, enabling efficient information extraction and decision-making (Lakshmi et al., 2019). Our reservoir layer serves as the encoder with the same purpose but applying a memristor-based hardware framework with non-linear dynamics. This is distinct from traditional methods and aligns with our objective to optimize both performance and hardware compatibility. In this study, we make a comparison of existing popular representation methods for event data in Table 1. This comparison presents the characteristics and storage feature of each method, which highlight their suitability for different computational scenarios. Our proposed memristor-based method can extract both spatial and temporal features with the advantage of being hardware-friendly and energy-efficient.

Figure 4A displays the visualization result of different event representation methods for digit “9” in NMNIST dataset. The visualization result contains four different methods, namely, raw event, frame, VoxelGrid, and our method. The raw event data consist of individual events plotted in space and time, resulting in a sparse and scattered visualization. This method retains all event information but lacks structure, making it difficult to discern the underlying pattern of the digit “9.” The frame-based representation method accumulates events over a fixed time interval to create a frame, which could blur temporal information and introduce motion artifacts. This VoxelGrid approach balances both spatial and temporal information but may suffer from quantization effects, which may lead to a loss of fine details. Our method extracts both spatial and temporal information through non-linear behavior of memristor device, maintaining clarity in the contours. The event representation methods aim to capture and retain essential features through sparse event behaviors, enabling efficient processing and analysis of dynamic visual scenes. Figure 4B shows the comparison result for raw event data and the data after memritsor-based reservoir encoder. The visualization result utilizes t-distributed stochastic neighbor embedding (t-SNE) method for dimensionality reduction. It contains 2,000 samples randomly chosen from test set of NMNIST dataset with the same experimental setup. Samples from the same category are more distinctly clustered after the proposed encoder, which indicates the information extraction ability of proposed memristor reservoir event representation method.

We evaluate the memristor encoding method in DNN/SNN bottleneck structure through event-based datasets, including DVS Gesture (Amir et al., 2017), DVS CIFAR10 (Li et al., 2017), NMNIST (Orchard et al., 2015), and DVSLip (Tan et al., 2022). To integrate the information effectively, the encoder captures data at a frequency of 30 Hz and sent to the subsequent layers. In the simulation, the initial states of neurons in the reservoir are drawn from a normal distribution with a mean of (Rmin+Rmax)/2. The NEIR method allows efficient feature description of the temporal information in all prior spike inputs with simple DNN backpropagation rules across time steps. We compare the NEIR with existing encoding approaches including frame, time surface, and voxel grid. The comparison results are shown in Table 2. With the same experiment setting, we evaluate the event representation methods via spike and non-spike bottleneck network. For image recognition on DVS Gesture dataset, our NEIR method reaches the accuracy of 88.42% in VGG 11 model, which is much higher than the other encoding method (87.11% for time surface).

We also apply spike-based bottleneck network to evaluate the performance of encoding methods. To solve the non-differentiability of spike behavior, we implement the spiking activation function with an approximation of gradient (Wu et al., 2019). We set the default time steps as 5 for each event sample and conduct experiment with different encoding methods. We observed that our method achieved an accuracy of 66.45% on the DVS CIFAR10 dataset using a spike-based VGG 16 model, compared with 64.95% when employing time-surface techniques. The NEIR method exhibits superior performance over other encoding method in both spike and non-spike architecture. The ablation study demonstrates that the NEIR method achieves high accuracy on the test dataset, indicating its effectiveness in capturing essential features from event data. The Pytorch platform and Tonic package were used for all the experiments with methods described above.

Figures 5A, B provide a summary of the impact of non-idealities on our system for image recognition on DVS Gesture dataset. Each data point in the figure shows the mean and standard deviation across five arbitrary seed values. Due to the stochastic ion behavior and vacancy forming process, the conductance of memristors displays variations and fluctuations from expected conductance value (Ielmini and Wong, 2018). Figure 5A illustrates the impact of device conductance variation. As device conductance variation increases, network performance gradually declines, dropping from 88.45% at a 5% conductance range to 80.47% at a 25% conductance range, with an increasing of accuracy variation. Figure 5B shows the effect of ADC resolution. With the higher resolution ADC, the precision of information improves and the network achieves better performance.

Table 3 shows the hardware evaluation and comparison for the proposed reservoir circuit node. We conduct the circuit simulation via PSPICE in Cadence and explore critical performance metrics including power consumption, energy usage, and latency. The result shows an average power consumption of 137 μW and normalized energy of 0.69 μJ per event with supply voltage of 3.3V, which highlights efficiency in information encoding at low cost. The expected propagation delay of 280.19 ns represents the total time delay across the critical path, where the memristor crossbar analysis is evaluated in Lu et al. (2021). Compared to other studies, our design demonstrates competitive performance. Zhong et al. (2021) reports a lower power consumption of 50 μW and a normalized energy of 0.006 μJ but emphasized on device measurement. Our evaluation is based on the individual memristor reservoir circuit and achieve competitive accuracies on multiple event-based datasets, which is more complex task across others. This circuit performance evaluation shows the requirements for the memristor-based reservoir node, which reveals the potential application for energy-efficient event sensing system.