Humar Kahramanlı Örnek, Samad Barri Khojasteh

The Robust EEG Based Emotion Recognition using Deep Neural Network

This paper focuses on a novel Electroencephalography (EEG) based one dimensional convolution neural network (CNN) to classify emotional states. Differential entropy (DE) is considered as a feature extraction method after pre-processing phase. Besides, feature smoothing-linear dynamic system (LDS) and min-max normalization are used on the DE features before feeding into deep model. We design a one dimensional CNN model with six convolutions and fully connected blocks which gives outstanding performance in six combinations of SEED dataset. The model presented average accuracy of 98.55% and 95.91% in binary and single sessions respectively by using 10 fold cross validation. The proposed results fully demonstrate that our method achieves out of the best performance compare with other EEG based emotion recognition systems. Therefore, this model can be applied to other emotional datasets as a classifier and health care decision support system (DSS) as well.

PDF

___

[1] S. M. Alarcão and M. J. Fonseca, “Emotions recognition using EEG signals: A survey,” IEEE Transactions on Affective Computing, vol. 10, no. 3, pp. 374–393, 2019, doi: 10.1109/TAFFC.2017.2714671.
[2] F. Demir, N. Sobahi, S. Siuly, and A. Sengur, “Exploring Deep Learning Features For Automatic Classification Of Human Emotion Using EEG Rhythms,” IEEE Sensors Journal, vol. 21, no. 13, pp. 14923–14930, 2021, doi:
[3] A. M. Al-Kaysi et al., “Predicting tDCS treatment outcomes of patients with major depressive disorder using automated EEG classification,” Journal of Affective Disorders, vol. 208, no. July 2016, pp. 597–603, 2017, doi: 10.1016/j.jad.2016.10.021.
[4] H. Berger, “Über das Elektrenkephalogramm des Menschen - Dritte Mitteilung,” Archiv für Psychiatrie und Nervenkrankheiten, vol. 94, no. 1, pp. 16–60, 1931, doi: 10.1007/BF01835097 . [5] M. J. Black and Y. Yacoob, “Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion,” International Journal of Computer Vision, vol. 25, no. 1, pp. 23–48, 1997, doi: 10.1023/A:1007977618277.
[6] L. F. Nicolas-alonso and J. Gomez-gil, “Brain Computer Interfaces, a Review,” Sensors, pp. 1211–1279, 2012, doi: 10.3390/s120201211.
[7] S. Hwang, K. Hong, G. Son, and H. Byun, “Learning CNN features from DE features for EEG-based emotion recognition,” Pattern Analysis and Applications, vol. 23, no. 3, pp. 1323–1335, 2020, doi: 10.1007/s10044-019-00860-w.
[8] R. N. Duan, J. Y. Zhu, and B. L. Lu, “Differential entropy feature for EEG-based emotion classification,” International IEEE/EMBS Conference on Neural Engineering, NER, pp. 81–84, 2013, doi: 10.1109/NER.2013.6695876.
[9] W. L. Zheng and B. L. Lu, “Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks,” IEEE Transactions on Autonomous Mental Development, vol. 7, no. 3, pp. 162– 175, 2015, doi: 10.1109/TAMD.2015.2431497.
[10] Y. Lu, M. Wang, W. Wu, Y. Han, Q. Zhang, and S. Chen, “Dynamic entropy-based pattern learning to identify emotions from EEG signals across individuals,” Measurement: Journal of the International Measurement Confederation, vol. 150, p. 107003, 2020, doi: 10.1016/j.measurement.2019.107003.
[11] M. Asadur Rahman, M. Foisal Hossain, M. Hossain, and R. Ahmmed, “Employing PCA and t-statistical approach for feature extraction and classification of emotion from multichannel EEG signal,” Egyptian Informatics Journal, vol. 21, no. 1, pp. 23–35, 2020, doi: 10.1016/j.eij.2019.10.002.
[12] J. Fdez, N. Guttenberg, O. Witkowski, and A. Pasquali, “Cross-Subject EEG-Based Emotion Recognition Through Neural Networks With Stratified Normalization,” Frontiers in Neuroscience, vol. 15, no. February, 2021, doi: 10.3389/fnins.2021.626277.
[13] D. W. Chen et al., “A feature extraction method based on differential entropy and linear discriminant analysis for emotion recognition,” Sensors (Switzerland), vol. 19, no. 7, 2019, doi: 10.3390/s19071631.
[14] T. M. Li, H. C. Chao, and J. Zhang, “Emotion classification based on brain wave: a survey,” Human-centric Computing and Information Sciences, vol. 9, no. 1, 2019, doi: 10.1186/s13673-019-0201-x.
[15] V. Gupta, M. D. Chopda, and R. B. Pachori, “Cross-Subject Emotion Recognition Using Flexible Analytic Wavelet Transform from EEG Signals,” IEEE Sensors Journal, vol. 19, no. 6, pp. 2266–2274, 2019, doi: 10.1109/JSEN.2018.2883497.
[16] X. H. Wang, T. Zhang, X. M. Xu, L. Chen, X. F. Xing, and C. L. P. Chen, “EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks and Broad Learning System,” Proceedings - 2018 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2018, pp. 1240–1244, 2019, doi: 10.1109/BIBM.2018.8621147.
[17] C. Wei, L. lan Chen, Z. zhen Song, X. guang Lou, and D. dong Li, “EEG-based emotion recognition using simple recurrent units network and ensemble learning,” Biomedical Signal Processing and Control, vol. 58, p. 101756, 2020, doi: 10.1016/j.bspc.2019.101756.
[18] W. L. Zheng, J. Y. Zhu, and B. L. Lu, “Identifying stable patterns over time for emotion recognition from eeg,” IEEE Transactions on Affective Computing, vol. 10, no. 3, pp. 417–429, 2019, doi: 10.1109/TAFFC.2017.2712143.
[19] L. Shi and B. L. S. Member, “Off-Line and On-Line Vigilance Estimation Based on Linear Dynamical System and Manifold Learning,” in 32nd Annual International Conference of the IEEE EMBS, 2010, pp. 6587–6590.
[20] W. Zhang, F. Sun, C. Tan, and S. Liu, “Linear Dynamical Systems Modeling for EEG-Based Motor Imagery Brain- Computer Interface,” Communications in Computer and Information Science, vol. 710, pp. 521–528, Nov. 2016, doi: 10.1007/978-981-10-5230-9_50.
[21] A. Craik, Y. He, and J. L. Contreras-Vidal, “Deep learning for electroencephalogram (EEG) classification tasks: A review,” Journal of Neural Engineering, vol. 16, no. 3, 2019, doi: 10.1088/1741-2552/ab0ab5.
[22] K. He, X. Zhang, S. Ren, and J. un, “Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification,” in 2015 IEEE International Conference on Computer Vision (ICCV), 2015, vol. 2015 Inter, pp. 1026–1034, doi: 10.1109/ICCV.2015.123.
[23] U. R. Acharya, S. L. Oh, Y. Hagiwara, J. H. Tan, and H. Adeli, “Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals,” Computers in Biology and Medicine, vol. 100, no. August 2017, pp. 270–278, 2018, doi: 10.1016/j.compbiomed.2017.09.017.
[24] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in 32nd International Conference on Machine Learning, ICML 2015, 2015, vol. 1, pp. 448–456.
[25] R. Kohavi, “A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection,” 1995.
[26] L. Bai, J. Guo, T. Xu, and M. Yang, “Emotional Monitoring of Learners Based on EEG Signal Recognition,” Procedia Computer Science, vol. 174, pp. 364–368, Jan. 2020, doi: 10.1016/j.procs.2020.06.100.