Affective computing thrives to teach machines to understand and express emotions (Picard, 1997). Emotions are multifaceted phenomena with physiological manifestations and bodily expressions (Scherer, 2005). Although the majority of existing methods for automatic emotion recognition are based on audiovisual analysis (D’Mello and Kory, 2015), there is a growing body of research on emotion recognition from peripheral and central nervous system physiological responses (Picard et al., 2001; Lisetti and Nasoz, 2004; Chanel et al., 2009, 2011; Kolodyazhniy et al., 2011; Koelstra et al., 2012; Soleymani et al., 2012b; Mühl et al., 2014). There are certain advantages for using physiological signals for emotion recognition compared to the audiovisual signals; for example, they cannot be easily faked, they do not require a front facing camera, and they can be used in any illumination and in noisy environments. Moreover, they can be combined with audiovisual modalities to construct a more robust and accurate multimodal emotion recognizer (D’Mello and Kory, 2015).

In order to train machines to automatically recognize emotions, we use machine learning techniques. Typically, the procedure includes translating raw signals to a lower dimensional representation space (features) that will be fed into a statistical model. Current physiological feature extraction methods involve calculating statistical moments and time-frequency analysis over segments of physiological signals with the goal of generating emotionally discriminative features. For example, heart rate increase is associated with excitement and activation (arousal) and, as a result, we extract heart rate as a feature for detecting arousal or the level of activation.

To extract features from physiological signals, one can implement methods that translate raw signals into features. Emotional features extracted from physiological signals are often based on the previous work in psychophysiology (Kreibig, 2010). To facilitate research on physiological signal analysis, a number of tools have been developed to the benefit of the community, e.g., EEGLAB (Delorme and Makeig, 2004). To the best of our knowledge, the Augsburg Biosignal Toolobox (AuBT) (Wagner et al., 2005) is the only toolbox available with the goal of extracting emotionally significant physiological features. However, the number of features and the type of signals in AUBT are limited. In this paper, we introduce a Toolbox for Emotional feAture extraction using Physiological signals or TEAP. We aim to create an open source platform that can be further extended by the community with the goal of advancing the field of affective physiological signal analysis. We developed TEAP in MathWorks MATLAB but it also works with Octave, its free alternative. TEAP is able to preprocess and extract features from multiple central and peripheral physiological signals including electroencephalogram (EEG), galvanic skin response (GSR), electrocardiogram (ECG), blood volume pulse (BVP), skin temperature, respiration pattern, and electromyogram (EMG). New physiological channels can be easily added to this toolbox and the implemented statistical and time-frequency analysis functions can be applied on any signal.

TEAP is designed to be useful for both novice and advanced users. A user can simply choose the desired features, define the channels in the input files, and extract physiological features. More advanced users are able to add new functionalities and modules, including support for new signals.

In this section, we review the existing freely available and open source tools for physiological signal analysis. There are a number of toolboxes for processing physiological signals and most of them are tuned for only one type of signal. For instance, EEGLAB (Delorme and Makeig, 2004) and FieldTrip (Oostenveld et al., 2011) are dedicated to brain signal analysis, focusing on signals such as EEG and magnetoencephalogram (MEG). There are also tools for analyzing peripheral physiological signals such as Kubios,¹ which analyzes heart rate variability, Ledalab (Benedek and Kaernbach, 2010), and PsPM (Bach and Friston, 2013), which focus on the analysis of galvanic skin responses. There are also tools for analyzing a wide range of physiological signals including Biosig (Schlogl and Brunner, 2008) and the ANSLAB.² Tools dedicated to one specific physiological signal offer advanced functionalities (e.g., source reconstruction). More general purpose tools handle a diverse range of signals with a more limited set of choices for their analysis. The spirit of TEAP is to allow for the processing of several types of signals, e.g., BVP, ECG, EEG, while maintaining the possibility to extract specific features from each signal. In addition, its architecture and license permits further development in the future.

Most of existing toolboxes requires to have some knowledge about physiological signal processing. In contrast, TEAP can serve researchers who aim at developing data-driven techniques with limited knowledge on the nature of physiological signals. Taking advantage of the BioSig code in its data import interface (Schlogl and Brunner, 2008), TEAP accepts several data formats which are traditionally used to store physiological signals (e.g., the EDF—(European Data Format), while its output is a design matrix representing a set of features for several samples. The Augsburg Biosignal Toolobox (AuBT) (Wagner et al., 2005) has been designed with the same objective in mind. It can perform feature extraction, feature selection, and classification. However, the AuBT only offers the possibility to extract general features based on statistical moments of signal derivatives (e.g., mean of first derivative). In addition, the AuBT input data should be properly formatted and have limited filtering capabilities. This leads to the need to develop file parsers and to gain knowledge in the properties of physiological signals to build proper preprocessing filters. The reviewed characteristics of the existing and proposed toolboxes are summarized in Table 1.

In order to verify TEAP’s performance in emotion recognition, we made use of two publicly available databases to serve as our benchmark, namely MAHNOB-HCI (Soleymani et al., 2012a) and DEAP (Koelstra et al., 2012). We tried to replicate the work presented in four original articles on these databases, i.e., Soleymani et al. (2011, 2012a,b) and Koelstra et al. (2012). In addition to TEAP, reproduction of these results required LIBSVM (Chang and Lin, 2001). We tried to re-implement the same procedure in terms of cross-validation strategy, feature selection, machine learning models, and evaluation metrics. For MAHNOB-HCI database, we only analyzed the emotion experiment whose original results are published in Soleymani et al. (2012a,b). The results of participant-independent cross validation on MAHNOB-HCI database are reported alongside the original results in Table 3. The regression and classification results on DEAP database are also given in Table 3. In most cases, TEAP features are performing similarly with small differences compared to the original work, the reasons behind this difference is twofold. First, we shortened and simplified the set of features in TEAP compared to the original work to make them more generally applicable. For example, EEG asymmetry features are not implemented in TEAP, since it depends on the electrode placement and do not make a large difference. Second, even though we tried to replicate the same machine learning models, our hyper-parameter optimization and feature selection slightly differ from the original work. Nevertheless, the results from TEAP features are, in most cases, not significantly different from the original results and validates its effectiveness in capturing emotionally significant representation of physiological signals.

We propose a new open-source toolbox dedicated to extracting and calculating emotionally relevant representations from physiological signals. This toolbox will facilitate and accelerate research in emotional physiological signals analysis. This toolbox is able to import, preprocess, and visualize its signals. It can export features vectors that can be directly fed to the machine learning models. In this paper, we outlined its abilities and demonstrated its effectiveness according to the publicly available benchmarks. Ideal candidates for future feature implementation include synchronization, intermodality features such as pulse ejection time and event-related potentials (ERP). We hope this toolbox can serve the researchers interested in affective physiological signal analysis.

MS and GC developed and adopted the original features (Chanel et al., 2009, 2011; Soleymani et al., 2009, 2012b; Koelstra et al., 2012). MS performed the experiments and reported the results. FV-D designed the architecture and modules and created the toolbox. TP participated in drafting the article and the discussions.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.