Application of multiscale fuzzy entropy features for multilevel subject-dependent emotion recognition

Emotion recognition can be used in clinical and nonclinical situations. Despite previous works which mostlyused time and frequency features of electroencephalogram (EEG) signals in subject-dependent emotion recognitionissues, we used multiscale fuzzy entropy as a nonlinear dynamic feature. The EEG signals of the well-known Databasefor Emotion Analysis Using Physiological signals dataset was used for classification of two and three levels of emotionsin arousal and valence space. The compound feature selection with a cost of average accuracy of support vector machineclassifier was used to reduce feature dimensions. For subject-dependent systems, the proposed method is superior incomparison to previous works with 90.81% and 90.53% accuracies in two-level classification and 79.83% and 77.80%accuracies in three-level classification in arousal and valence dimensions, respectively.

PDF

___

[1] Dalgleish T, Dunn BD, Mobbs D. Affective neuroscience: past, present, and future. Emotion Review. 2009; 1(4): 355-368.
[2] Bajaj V, Pachori RB. Detection of human emotions using features based on the multiwavelet transform of EEG signals. Journal of Brain-Computer Interfaces: Springer 2015; 74: 215-240.
[3] Verma GK, Tiwary US. Affect representation and recognition in 3d continuous valence–arousal–dominance space. Journal of Multimedia Tools and Applications 2016:1-25.
[4] Tao J, Tan T, Picard R. Affective computing and intelligent interaction. First International Conference, ACII 2005; Beijing, China; 2005, Proceedings; 2005: Springer.
[5] Salmeron JL. Fuzzy cognitive maps for artificial emotions forecasting. Journal of Applied Soft Computing 2012; 12(12): 3704-3710.
[6] Lang PJ, Bradley MM, Cuthbert BN. International Affective Picture System (IAPS): Instruction Manual and Affective Ratings. The Center for Research in Psychophysiology, University of Florida, 1999.
[7] Bradley MM, Lang PJ. The International Affective Digitized Sounds (IADS-2): Affective Ratings of Sounds and Instruction Manual. University of Florida, Gainesville, FL, Tech Rep B-3. 2007.
[8] Zhou F, Qu X, Jiao JR, Helander MG. Emotion prediction from physiological signals: A comparison study between visual and auditory elicitors. Journal of Interacting with Computers 2014; 26(3): 285-302.
[9] Gao J, Zheng C, Wang P. Online removal of muscle artifact from electroencephalogram signals based on canonical correlation analysis. Journal of Clinical EEG and neuroscience 2010; 41(1): 53-59.
[10] Soleymani M, Pantic M, Pun T. Multimodal emotion recognition in response to videos. IEEE Journal of Transactions on Affective Computing 2012; 3(2): 211-223.
[11] Takahashi K. Remarks on SVM-based emotion recognition from multimodal bio-potential signals. 13th IEEE International Workshop on Robot and Human Interactive Communication; Kurashiki, Okayama Japan; 2004. pp. 1-25.
[12] Chanel G, Kierkels JJ, Soleymani M, Pun T. Short-term emotion assessment in a recall paradigm. International Journal of Human-Computer Studies 2009; 67(8): 607-627.
[13] Yoon HJ, Chung SY. EEG-based emotion estimation using Bayesian weighted-log-posterior function and perceptron convergence algorithm. Journal of Computers in Biology and Medicine 2013; 43(12): 2230-2237.
[14] Zhang Y, Zhang S, Ji X. EEG-based classification of emotions using empirical mode decomposition and autoregressive model. Journal of Springer Science & Business Media, LLC, part of Springer Nature 2018; 20(77): 26697-26710.
[15] Piho L, Tjahjadi T. A mutual information based adaptive windowing of informative EEG for emotion recognition. IEEE Journal of Transactions on Affective Computing 2018; 1-1. doi: 10.1109/TAFFC.2018.2840973
[16] Costa M, Goldberger AL, Peng CK. Multiscale entropy analysis of complex physiologic time series. Journal of Physical Review Letters 2002; 89(6): 068102.
[17] Grassberger P. Toward a quantitative theory of self-generated complexity. International Journal of Theoretical Physics 1986; 25(9): 907-938.
[18] Goldenfeld N, Kadanoff LP. Simple lessons from complexity. Journal of Science 1999; 284(5411): 87-89.
[19] Manson SM. Simplifying complexity: a review of complexity theory. Journal of Geoforum 2001; 32(3): 405-414.
[20] Escudero J, Abásolo D, Hornero R, Espino P, López M. Analysis of electroencephalograms in Alzheimer’s disease patients with multiscale entropy. Journal of Physiological Measurement 2006; 27(11): 1091.
[21] Hornero R, Abásolo D, Escudero J, Gómez C. Nonlinear analysis of electroencephalogram and magnetoencephalogram recordings in patients with Alzheimer’s disease. Journal of Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences. 2009; 367(1887): 317-336.
[22] Costa M, Peng CK, Goldberger AL, Hausdorff JM. Multiscale entropy analysis of human gait dynamics. Physical A: Statistical Mechanics and Its Applications 2003; 330(1-2): 53-60.
[23] Takahashi T, Cho RY, Murata T, Mizuno T, Kikuchi M et al. Age-related variation in EEG complexity to photic stimulation: a multiscale entropy analysis. Clinical Neurophysiology 2009;120(3):476-483.
[24] Costa M, Ghiran I, Peng CK, Nicholson-Weller A, Goldberger AL. Complex dynamics of human red blood cell flickering: alterations with in vivo aging. Physical Review E. 2008; 78(2): 020901.
[25] Li P, Ji L, Yan C, Li K, Liu C et al. Coupling between short-term heart rate and diastolic period is reduced in heart failure patients as indicated by multivariate entropy analysis. Age (years). 2014; 56(7.6): 59.8-10.6.
[26] Ahmed MU, Chanwimalueang T, Thayyil S, Mandic DP. A multivariate multiscale fuzzy entropy algorithm with application to uterine EMG complexity analysis. Entropy 2016; 19(1): 2.
[27] Russel J. A circumplex model of affect. Journal of Personality and Social Psychology 1980; 39: 1161-1178.
[28] Koelstra S, Muhl C, Soleymani M, Lee J-S, Yazdani A et al. Deap: A database for emotion analysis; using physiological signals. IEEE Journal of Transactions on Affective Computing 2012; 3(1): 18-31.
[29] Xiang J, Li C, Li H, Cao R, Wang B et al. The detection of epileptic seizure signals based on fuzzy entropy. Journal of neuroscience methods 2015; 243: 18-25.
[30] James G, Witten D, Hastie T, Tibshirani R. An Introduction to Statistical Learning, vol. 112. NY, USA: Springer, 2013.
[31] Molina LC, Belanche L, Nebot À. Feature selection algorithms: A survey and experimental evaluation. In: Proceedings 2002 IEEE International Conference on Data Mining; Maebashi City, Japan; 2002. pp. 1-19.
[32] Cortes C, Vapnik V. Support-vector networks. Journal of Machine learning 1995; 20(3): 273-297.
[33] Vapnik V. The Nature of Statistical Learning Theory. NY, USA: Springer Science & Business Media, 2000.
[34] Knerr S, Personnaz L, Dreyfus G. Single-layer learning revisited: a stepwise procedure for building and training a neural network. Neurocomputing Journal of Springer 1990; 3(68): 41-50.
[35] Hsu CW, Lin CJ. A comparison of methods for multiclass support vector machines. IEEE Journal of transactions on Neural Networks 2002; 13(2): 415-425.
[36] Geisser S. Predictive Inference. New York, NY, USA: CRC Press, 1993.
[37] Kaya GT, Kaya H. Optimization of SVM parameters using High Dimensional Model Representation and its application to hyperspectral images. Signal Processing and Communications Applications Conference (SIU); Trabzon, Turkey; 2014. pp. 1-20.
[38] Fox NA. If it’s not left, it’s right: electroencephalograph asymmetry and the development of emotion. American Psychologist 1991; 46(8): 863.
[39] Fox NA. Dynamic cerebral processes underlying emotion regulation. Journal of Monographs of the Society for Research in Child Development 1994; 59(2‐3): 152-66.
[40] Schmidt LA, Trainor LJ. Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions. Journal of Cognition & Emotion 2001;15(4): 487-500.
[41] Naser DS, Saha G, editors. Recognition of emotions induced by music videos using DT-CWPT. Medical Informatics and Telemedicine (ICMIT), 2013 Indian Conference; Kharagpur, India; 2013. pp. 53-57.
[42] Matiko JW, Beeby SP, Tudor J. Fuzzy logic based emotion classification. Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference; Hong Kong, China; 2014. pp. 6974-6980.
[43] Chen J, Hu B, Moore P, Zhang X, Ma X. Electroencephalogram-based emotion assessment system using ontology and data mining techniques. Journal of Applied Soft Computing 2015; 30: 663-674.
[44] Zheng WL, Lu BL. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Journal of Transactions on Autonomous Mental Development 2015; 7(3):162-75.
[45] Atkinson J, Campos D. Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers. Journal of Expert Systems with Applications 2016; 47: 35-41.