A physiological-based emotion recognition system (ERS) with a unimodal approach such as an electrocardiogram (ECG) is not as popular compared to a multimodal approach. However, a single modality has the advantage of lower development and computational cost. Therefore, this study focuses on a unimodal ECG-based ERS. The ECG-based ERS has the potential to become the next mass-adopted consumer application due to the wide availability of wearable and mobile ECG devices in the market. Currently, ECG-inclusive affective datasets are limited, and many of the existing datasets have small sample sizes. Hence, ECG-based ERS studies are stunted by the lack of quality data. A novel multi-filtering augmentation technique is proposed here to increase the sample size of the ECG data. This technique augments the ECG signals by cleaning the data in different ways. Three small ECG datasets labelled according to emotion state are used in this study. The benefit of the proposed augmentation techniques is measured using the classification accuracy of five machine learning algorithms; k-nearest neighbours (KNN), support vector machine, decision tree, random forest and multilayer perceptron. The results show that with the proposed technique, there is a significant improvement in performance for all the datasets and classifiers. KNN classifier improved the most with the augmented data and the reported classification accuracies of over 90%.
Affective computing is a field of study that integrates human affects and emotions with artificial intelligence into systems or devices. A system or device with affective computing is beneficial for the mental health and wellbeing of individuals that are stressed, anguished, or depressed. Emotion recognition systems are an important technology that enables affective computing. Currently, there are a lot of ways to build an emotion recognition system using various techniques and algorithms. This review paper focuses on emotion recognition research that adopted electrocardiograms (ECGs) as a unimodal approach as well as part of a multimodal approach for emotion recognition systems. Critical observations of data collection, pre-processing, feature extraction, feature selection and dimensionality reduction, classification, and validation are conducted. This paper also highlights the architectures with accuracy of above 90%. The available ECG-inclusive affective databases are also reviewed, and a popularity analysis is presented. Additionally, the benefit of emotion recognition systems towards healthcare systems is also reviewed here. Based on the literature reviewed, a thorough discussion on the subject matter and future works is suggested and concluded. The findings presented here are beneficial for prospective researchers to look into the summary of previous works conducted in the field of ECG-based emotion recognition systems, and for identifying gaps in the area, as well as in developing and designing future applications of emotion recognition systems, especially in improving healthcare.