Derin Öğrenme Modeli ile Yüz İfadelerinden Duygu Tanıma

Yüz ifadesinden duygu tanıma; insan-bilgisayar etkileşimi, duygusal hesaplama vb. gibi birçok bilgisayarla görme alanında uygulanabilen güncel bir araştırma konusudur. Bu çalışmada, KDEF ve PICS veri setleri kullanılarak derin öğrenme ile duygu tanımaya yönelik bir uygulama yapılmıştır. Öznitelik çıkarımı için derin öğrenme tekniklerinden olan ve yapay sinir ağları içeren bir yapay zekâ yaklaşımı olan Evrişimsel Sinir Ağı (ESA) kullanılarak yeni bir model geliştirilmiştir. Derin öğrenmenin yüksek başarımı için büyük veri setine ihtiyaç duyulmaktadır. KDEF veri setinde 4900, PICS veri setinde 322 görüntü bulunmaktadır. Bu amaçla öncelikle PICS veri setindeki görüntü sayısının az olmasından dolayı veri artırma yöntemi ile görüntü çoğaltma işlemi uygulanmıştır ve PICS veri seti 4830 görüntüye çıkarılmıştır. Daha sonra bu iki farklı veri seti üzerinde ayrı ayrı eğitim gerçekleştirilerek geliştirilen yeni model test edilmiştir. ESA modellerinden olan VGGNet temel alınarak geliştirilen yeni model ile gerçekleştirilen çalışmada, her bir veri setinde yedi farklı duygu sınıfı (korku, öfke, iğrenme, mutluluk, nötr, üzüntü, şaşırma) ele alınmıştır. Geliştirilen model ile KDEF veri setinin geçerleme kümesinde %97.44, PICS veri setinin geçerleme kümesinde %98.24 doğruluk değerleri elde edilerek yüksek bir başarı oranına ulaşılmıştır.

Anahtar Kelimeler:

Derin öğrenme, evrişimsel sinir ağı, duygu analizi

Emotion Recognition from Facial Expressions by Deep Learning Model

Emotion recognition from facial expression is a current research topic that can be applied in the many fields of computer vision, such as human-computer interaction, emotional computing, etc. In this study, an application for emotion recognition through deep learning was made using KDEF and PICS datasets. A new model was established using the convolutional neural network (CNN), an artificial intelligence approach that involves artificial neural networks, which is one of the deep learning techniques for attribute inference. Large datasets are needed for the high performance of deep learning. There are 4900 images in the KDEF dataset and 322 images in the PICS dataset. For this purpose, primarily due to the small number of images in the PICS dataset, image iteration was applied with the data augmentation method, and the PICS dataset was increased to 4830 images. Then, the new model developed by conducting separate training on these two different datasets was tested. Seven different classes of emotion (afraid, angry, disgusted, happy, neutral, sad, surprised) were covered in each dataset in the study conducted with a new model developed based on VGGNet which is one of the CNN models. With the developed model, a high success rate was achieved by obtaining 97.44% accuracy values in the validation set of the KDEF and 98.24% accuracy values in the validation set of the PICS dataset.

Keywords:

Deep learning, convolutional neural network, emotion recognition,

PDF

___

Ahmad F, Najam A, Ahmed Z, 2013. Image-based Face Detection and Recognition: “State of the Art.”. ArXiv Preprint ArXiv: 1302.6379.
Altan G, 2019. DeepGraphNet: Grafiklerin Sınıflandırılmasında Derin Öğrenme Modelleri. European Journal of Science and Technology, October: 319–329. https://doi.org/10.31590/ejosat.638256
Calvo MG, Lundqvist D, 2008. Facial expressions of emotion (KDEF): Identification under different display-duration conditions. Behavior Research Methods, 40(1): 109–115. https://doi.org/10.3758/BRM.40.1.109
Chen M, Lu Z, Jan PA, 2015. Learning deep features for image emotion classification. 2015 IEEE International Conference on Image Processing (ICIP), 4491–4495.
Dandıl E, Özdemir R, 2019. Real-time Facial Emotion Classification Using Deep Learning. DATA SCIENCE AND APPLICATIONS, 2(1): 13–17.
Eng SK, Ali H, Cheah AY, Chong YF, 2019. Facial expression recognition in JAFFE and KDEF Datasets using histogram of oriented gradients and support vector machine. IOP Conference Series: Materials Science and Engineering, 705(1). https://doi.org/10.1088/1757-899X/705/1/012031
Fayek HM, Lech M, Cavedon L, 2017. Evaluating deep learning architectures for Speech Emotion Recognition. Neural Networks, 92: 60–68. https://doi.org/10.1016/j.neunet.2017.02.013
Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T, Wang X, Wang G, Cai J, Chen T, 2018. Recent advances in convolutional neural networks. Pattern Recognition, 77: 354–377. https://doi.org/10.1016/j.patcog.2017.10.013
Hancock P, 2008. Psychological image collection at stirling (pics). Web Address: Http://Pics. Psych. Stir. Ac. Uk.
He K, Zhang X, Ren S, Sun J, 2016. Deep Residual Learning for Image Recognition Kaiming. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1002/chin.200650130
Hossain MS, Muhammad G, 2019. Emotion recognition using deep learning approach from audio–visual emotional big data. Information Fusion, 49: 69–78. https://doi.org/10.1016/j.inffus.2018.09.008
Huang Y, Chen F, Lv S, Wang X, 2019. Facial expression recognition: A survey. Symmetry, 11(10). https://doi.org/10.3390/sym11101189
Irtija N, Sami M, Ahad MAR, 2018. Fatigue detection using facial landmarks. Joint Conference ISASE-MAICS 2018 - 4th International Symposium on Affective Science and Engineering 2018, and the 29th Modern Artificial Intelligence and Cognitive Science Conference, 1–6. https://doi.org/10.5057/isase.2018-c000041
Koç M, Özdemir R, 2019. Yeni Bir Veri Kümesi (RidNet) Kullanarak Kontrolsüz Ortamda Yüz İfadesi Tanımanın Derin Öğrenme Yöntemleri ile İyileştirilmesi. Bilecik Şeyh Edebali Üniversitesi Fen Bilimleri Dergisi, 6(2): 384–396. https://doi.org/10.35193/bseufbd.645138
Krizhevsky A, Sutskever I, Hinton GE, 2012. ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems, 1097–1105. https://doi.org/10.1201/9781420010749
Lecun Y, Bengio Y, Hinton G, 2015. Deep learning. Nature, 521(7553): 436–444. https://doi.org/10.1038/nature14539
LeCun Y, Bottou L, Bengio Y, Haffner P, 1998. Gradient-based Learning Applied to Document Recognition. Proceedings of the IEEE, 86(11): 2278–2323. https://doi.org/10.1109/5.726791
Li J, Jin K, Zhou D, Kubota N, Ju Z, 2020. Attention mechanism-based CNN for facial expression recognition. Neurocomputing, 411: 340–350. https://doi.org/10.1016/j.neucom.2020.06.014
Li S, Deng W, 2020. Deep Facial Expression Recognition: A Survey. IEEE Transactions on Affective Computing, 3045(c): 1–20. https://doi.org/10.1109/TAFFC.2020.2981446
Lu Y, Wang S, Zhao W, 2019. Facial expression recognition based on discrete separable shearlet transform and feature selection. Algorithms, 12(1). https://doi.org/10.3390/a12010011
Lundqvist D, Flykt A, Öhman A, 1998. The Karolinska directed emotional faces (KDEF). CD ROM from Department of Clinical Neuroscience, Psychology Section, Karolinska Institutet 91(630): 2–2.
Mehta D, Siddiqui MFH, Javaid AY, 2019. Recognition of emotion intensities using machine learning algorithms: A comparative study. Sensors (Switzerland), 19(8): 1–24. https://doi.org/10.3390/s19081897
Özdemir, D, Karaman, S, 2017. Investigating interactions between students with mild mental retardation and humanoid robot in terms of feedback types. Egitim ve Bilim, 42(191).
Ruiz-Garcia A, Elshaw M, Altahhan A, Palade V, 2016. Deep learning for emotion recognition in faces. International Conference on Artificial Neural Networks Springer, 38–46. https://doi.org/10.1007/978-3-319-44781-0_5
Shorten C, Khoshgoftaar TM, 2019. A survey on Image Data Augmentation for Deep Learning. Journal of Big Data, 6(1). https://doi.org/10.1186/s40537-019-0197-0
Simonyan K, Zisserman A, 2014. Very deep convolutional networks for large-scale image recognition. ArXiv Preprint ArXiv: 1409.1556.
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A, 2015. Going Deeper with Convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1002/jctb.4820
Xu M, Cheng W, Zhao Q, Ma L, Xu F, 2015. Facial expression recognition based on transfer learning from deep convolutional networks. 2015 11th International Conference on Natural Computation (ICNC). IEEE, 702–708. https://doi.org/10.1109/ICNC.2015.7378076