Anju Mishra, Archana Singh, Amit Ujlayan

Pragmatic Approach for EEG-based Affect Classification

Mapping human cognition into automated analysis is the key area of research due to its fascinating applications in almost every area of developing artificially intelligent machines. The best way to understand the functioning of brain is to study electroencephalogram (EEG) patterns, therefore, a lot of research has been directed towards studying EEG signals. Since, EEG recordings are subject dependent and exhibit variations due to external influences or type of recording instruments, it is hard to develop a generalized affect categorization system that can provide robust affect labelling to the EEG patterns. To overcome this, proposed work presents a novel general framework for affect-based cognitive analysis. The proposed system involves following steps: pre-processing, feature selection, Generalized Procrustes Analysis (GPA) step to reduce the inter-class and intra-class variance and finally, the processed pattern is passed on to a trained classifier for classifying the pattern into appropriate affect categories. The presented approach has been tested on single as well as multiple subjects’ EEG data taken from two different datasets, Database for Emotion Analysis using Physiological signals (DEAP) and SJTU Emotion EEG Dataset (SEED) and performance of popular classifiers are assessed. The experimental results suggest that SVM classifier is the best among the selected ones for classifying single as well as mixed subjects’ data.

PDF

___

[1] S. Koelstra et al., "DEAP: A Database for Emotion Analysis; Using Physio-logical Signals," in IEEE Transactions on Affective Computing, vol. 3, no. 1, pp. 18-31, Jan.-March 2012. doi: 10.1109/T-AFFC.2011.15
[2] W.-L. Zheng, B.-L. Lu, "Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks", IEEE Trans. Auton. Mental Develop., vol. 7, no. 3, pp. 162-175, Sep. 2015.
[3] S. M. Alarcão and M. J. Fonseca, "Emotions Recognition Using EEG Signals: A Survey," in IEEE Transactions on Affective Computing, vol. 10, no. 3, pp. 374-393, 1 July-Sept. 2019. doi: 10.1109/TAFFC.2017.2714671
[4] W. Zheng, J. Zhu and B. Lu, "Identifying Stable Patterns over Time for Emotion Recognition from EEG," in IEEE Transactions on Affective Computing, vol. 10, no. 3, pp. 417-429, 1 July-Sept. 2019. doi: 10.1109/TAFFC.2017.2712143
[5] Preston, Simon P., and Andrew TA Wood. "Two‐Sample Bootstrap Hypothesis Tests for Three‐Dimensional Labelled Landmark Data." Scandinavian journal of statistics 37, no. 4 (2010): 568-587.
[6] Preston, S. P., and Andrew TA Wood. "Bootstrap inference fr mean reflection shape and size-and-shape with three-dimensional landmark data." Biometrika 98, no. 1 (2011): 49-63.
[7] H. Xiong, D. Zhang, C. J. Martyniuk, V. L. Trudeau, and X. Xia, “Using generalized procrustes analysis (GPA) for normali-zation of cDNA microarray data.,” BMC Bioinformatics, vol. 9, p. 25, 2008.
[8] X.-W. Wang, D. Nie, B.-L. Lu, "Emotional state classification from EEG data using machine learning approach", Neuro-computing, vol. 129, pp. 94-106, 2014.
[9] S. K. D'mello, J. Kory, "A review and meta-analysis of multimodal affect detection systems", ACM Computing. Surveys, vol. 47, no. 3, 2015.
[10] Y.-P. Lin et al., "EEG-based emotion recognition in mu-sic listening", IEEE Trans. Biomed. Eng., vol. 57, no. 7, pp. 1798-1806, Jul. 2010.
[11] M. Soleymani, J. Lichtenauer, T. Pun, M. Pantic, "A multimodal database for affect recognition and implicit tagging", IEEE Trans. Affect. Comput., vol. 3, no. 1, pp. 42-55, Jan. 2012.
[12] R. Duan, J. Zhu and B. Lu, "Differential entropy feature for EEGbased emotion classification," 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), San Diego, CA, 2013, pp. 81-84. doi: 10.1109/NER.2013.6695876
[13] R. Jenke, A. Peer and M. Buss, "Feature Extraction and Selection for Emotion Recognition from EEG," in IEEE Transactions on Affective Computing, vol. 5, no. 3, pp. 327-339, 1 July-Sept. 2014. doi: 10.1109/TAFFC.2014.2339834
[14] R. Majid Mehmood, R. Du and H. J. Lee, "Optimal Feature Selection and Deep Learning Ensembles Method for Emotion Recognition From Human Brain EEG Sensors," in IEEE Access, vol. 5, pp. 14797-14806, 2017. doi: 10.1109/ACCESS.2017.2724555
[15] D. Li et al., "The Fusion of Electroencephalography and Facial Expression for Continuous Emotion Recognition," in IEEE Access, vol. 7, pp. 155724-155736, 2019. doi: 10.1109/ACCESS.2019.2949707
[16] M.-K. Kim, M. Kim, E. Oh, S.-P. Kim, "A review on the computational methods for emotional state estimation from the human EEG", Comput. Math. Methods Med., vol. 2013, 2013.
[17] I. B. Mauss, M. D. Robinson, "Measures of emotion: A review", Cognition Emotion, vol. 23, no. 2, pp. 209-237, 2009.
[18] Z. Lan, O. Sourina, L. Wang, Y. Liu, "Stability of features in realtime EEG-based emotion recognition algorithm", Proc. Int. Conf. Cyberworlds, pp. 137-144, Oct. 2014.
[19] A. K. Jain, R. P. W. Duin, J. Mao, "Statistical pattern recognition: A review", IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 1, pp. 4-37, Jan. 2000.
[20] W.-L. Zheng, B.-L. Lu, "Personalizing EEG-based affective models with transfer learning", Proc. 25th Int. Joint Conf. Artif. Intell., pp. 2732-2738, 2016.
[21] S. M. Alarcão and M. J. Fonseca, "Emotions Recognition Using EEG Signals: A Survey," in IEEE Transactions on Affective Computing, vol. 10, no. 3, pp. 374-393, 1 July-Sept. 2019. doi: 10.1109/TAFFC.2017.2714671
[22] Z. Lan, O. Sourina, L. Wang, R. Scherer and G. R. Müller-Putz, "Domain Adaptation Techniques for EEG-Based Emotion Recognition: A Comparative Study on Two Public Datasets," in IEEE Transactions on Cognitive and Developmental Systems, vol. 11, no. 1, pp. 85-94, March 2019. doi: 10.1109/TCDS.2018.2826840
[23] Gupta, Rishabh, and Tiago H. Falk. "Relevance vector classifier decision fusion and EEG graph-theoretic features for automatic affective state characterization." Neurocomputing 174 (2016): 875- 884.
[24] Minjie Ren, Weizhi Nie, Anan Liu, Yuting Su, “Multi-modal Correlated Network for emotion recognition in speech”, Visual Informatics, Volume 3, Issue 3, 2019, Pages 150-155, ISSN 2468- 502X, https://doi.org/10.1016/j.visinf.2019.10.003.
[25] Fayek, Haytham M., Margaret Lech, and Lawrence Cavedon. "Evaluating deep learning architectures for Speech Emotion Recognition." Neural Networks 92 (2017): 60-68.
[26] Han, Wenjing, Huabin Ruan, Xiaomin Chen, Zhixiang Wang, Haifeng Li, and Björn W. Schuller. "Towards Temporal Modelling of Categorical Speech Emotion Recognition." In Inter-speech, pp. 932-936. 2018.
[27] Liu, Zhen-Tao, Min Wu, Wei-Hua Cao, Jun-Wei Mao, Jian-Ping Xu, and Guan-Zheng Tan. "Speech emotion recognition based on feature selection and extreme learning machine decision tree." Neurocomputing 273 (2018): 271-280.
[28] Sun, Linhui, Bo Zou, Sheng Fu, Jia Chen, and Fu Wang. "Speech emotion recognition based on DNN-decision tree SVM model." Speech Communication 115 (2019): 29-37.
[29] Hassana, Alia K., and Suhaila N. Mohammed. "A novel facial emotion recognition scheme based on graph mining." De-fence Technology (2019).
[30] He, Xuanyu, and Wei Zhang. "Emotion recognition by assisted learning with convolutional neural networks." Neuro-computing 291 (2018): 187-194.
[31] Yan, Jingwei, Wenming Zheng, Zhen Cui, Chuangao Tang, Tong Zhang, and Yuan Zong. "Multi-cue fusion for emotion recognition in the wild." Neurocomputing 309 (2018): 27-35.
[32] Y. Xu and G. Liu, "A Method of Emotion Recognition Based on ECG Signal," 2009 International Conference on Computational Intelligence and Natural Computing, Wuhan, 2009, pp. 202-205. doi: 10.1109/CINC.2009.102
[33] Guo Xianhai, “Study of Emotion Recognition Based on Electrocardiogram and RBF neural network”, Procedia Engineering, Volume 15, 2011, Pages 2408-2412, ISSN 1877-7058, https://doi.org/10.1016/j.proeng.2011.08.452.
[34] Atefeh Goshvarpour, Ataollah Abbasi, Ateke Goshvarpour, “An accurate emotion recognition system using ECG and GSR signals and matching pursuit method”, Biomedical Journal, Volume 40, Issue 6, 2017, Pages 355-368, ISSN 2319-4170, https://doi.org/10.1016/j.bj.2017.11.001.
[35] Atkinson, J. and Campos, D., 2016. Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers. Expert Systems with Applications, 47, pp.35-41.
[36] Mert, A. and Akan, A., 2018. Emotion recognition from EEG signals by using multivariate empirical mode decomposition. Pattern Analysis and Applications, 21(1), pp.81-89.
[37] Liu, W., Zheng, W.L. and Lu, B.L., 2016, October. Emotion recognition using multimodal deep learning. In International conference on neural information processing (pp. 521-529). Springer, Cham.
[38] Wichakam, I. and Vateekul, P., 2014, May. An evaluation of feature extraction in EEG-based emotion prediction with support vector machines. In 2014 11th international joint conference on computer science and software engineering (JCSSE) (pp. 106-110). IEEE.
[39] Tripathi, S., Acharya, S., Sharma, R.D., Mittal, S. and Bhattacharya, S., 2017, February. Using Deep and Convolutional Neural Networks for Accurate Emotion Classification on DEAP Dataset. In Twenty- Ninth IAAI Conference.
[40] P. Arnau-González, M. revalillo-Herráez, N. Ramzan (2017) Fusing highly dimensional energy and connectivity features to identify affective states from EEG signals. Neurocomputing, Volume 244, 2017, Pages 81-89, ISSN 0925-2312,
[41] Lin W., Li C., Sun S. (2017) Deep Convolutional Neural Network for Emotion Recognition Using EEG and Peripheral Physiological Signal. In: Zhao Y., Kong X., Taubman D. (eds) Image and Graphics. ICIG 2017. Lecture Notes in Computer Science, vol 10667. Springer, Cham. https://doi.org/10.1007/978-3-319-71589- 6_33