A computational study on aging effect for facial expression recognition

This work uses newly introduced variations of the sparse representation-based classifier (SRC) to challengethe issue of automatic facial expression recognition (FER) with faces belonging to a wide span of ages. Since facialexpression is one of the most powerful and immediate ways to disclose individuals’ emotions and intentions, the studyof emotional traits is an active research topic both in psychology and in engineering fields. To date, automatic FERsystems work well with frontal and clean faces, but disturbance factors can dramatically decrease their performance.Aging is a critical disruption element, which is present in any real-world situation and which can finally be consideredthanks to the recent introduction of new databases storing expressions over a lifespan. This study addresses the FERwith aging challenge using sparse coding (SC) that represents the input signal as the linear combination of the columnsof a dictionary. Dictionary learning (DL) is a subfield of SC that aims to learn from the training samples the bestspace capable of representing the query image. Focusing on one of the main challenges of SC, this work compares theperformance of recently introduced DL algorithms. We run both a mixed-age experiment, where all faces are mixed,and a within-age experiment, where faces of young, middle-aged, and old actors are processed independently. We firstwork with the entire face and then we improve our initial performance using only discriminative patches of the face.Experimental results provide a fair comparison between the two recently developed DL techniques. Finally, the samealgorithms are also tested on a database of expressive faces without the aging disturbance element, so as to evaluate DLalgorithms’ performance strictly on FER.

PDF

___

[1] Puce A, Allison T, Gore JC, McCarthy G. Face-sensitive regions in human extrastriate cortex studied by functional MRI. Journal of Neurophysiology 1995; 74 (3): 1192-1199.
[2] Rybarczyk BD, Hart SP, Harkins SW. Age and forgetting rate with pictorial stimuli. Psychology and Aging 1987; 2 (4): 404-406.
[3] Grady CL, McIntosh AR, Horwitz B, Maisog JM, Ungerleider LG et al. Age-related reductions in human recognition memory due to impaired encoding. Science 1995; 269: 218-221.
[4] Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z et al. Comprehensive database for facial expression analysis. In: Fourth IEEE International Conference on Automatic Face and Gesture Recognition; Grenoble, France; 2000. pp. 46-53.
[5] Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z et al. The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops; San Francisco, CA, USA; 2010. pp. 94-101.
[6] Lyons M, Akamatsu S, Kamachi M, Gyoba J. Coding facial expressions with Gabor wavelets. In: Third IEEE International Conference on Automatic Face and Gesture Recognition; Nara, Japan; 2018. pp. 200-205.
[7] Pantic M, Valstar M, Rademaker R, Maat L. Web-based database for facial expression analysis. In: IEEE International Conference on Multimedia and Expo; Amsterdam, the Netherlands; 2005. p. 5.
[8] Dhall A, Goecke R, Joshi J, Hoey J, Gedeon T. EmotiW 2016: Video and group-level emotion recognition challenges. In: 18th ACM International Conference on Multimodal Interaction; Tokyo, Japan; 2016. pp. 427-432.
[9] Minear M, Park DC. A lifespan database of adult facial stimuli. Behavior Research Methods, Instruments, & Computers 2004; 36 (4): 630-633.
[10] Ebner NC. Age of face matters: age-group differences in ratings of young and old faces. Behavior Research Methods 2008; 4: 130-136.
[11] Ebner NC, Riediger M, Lindenberger U. FACES—A database of facial expressions in young, middle-aged, and older women and men: development and validation. Behavior Research Methods 2010; 42: 351-362.
[12] Ebner NC, Johnson MK. Age-group differences in interference from young and older emotional faces. Cognition and Emotion 2010; 24 (7): 1095-1116.
[13] Guo G, Guo R, Li X. Facial expression recognition influenced by human aging. IEEE Transactions on Affective Computing 2013; 4 (3): 291-298.
[14] Algaraawi N, Morris T. Study on aging effect on facial expression recognition. In: World Congress on Engineering; London, UK; 2016.
[15] Johnston B, Chazal P. A review of image-based automatic facial landmark identification techniques. EURASIP Journal on Image and Video Processing 2018; 1: 86.
[16] Xu Y, Li Z, Yang J, Zhang D. A survey of dictionary learning algorithms for face recognition. IEEE Access 2017; 5: 8502-8514.
[17] Chellappa R. The changing fortunes of pattern recognition and computer vision. Image and Vision Computing 2016; 55 (1): 3-5.
[18] Elad M, Aharon M. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing 2006; 15 (12): 3736-3745.
[19] Fu Y, Lam A, Sato I, Sato Y. Adaptive spatial-spectral dictionary learning for hyperspectral image restoration. International Journal of Computer Vision 2017; 122 (2): 228-245.
[20] Chen YC, Sastry CS, Patel VM, Phillips PJ, Chellappa R. In-plane rotation and scale invariant clustering using dictionaries. IEEE Transactions on Image Processing 2013; 22 (6): 2166-2180.
[21] Xiang S, Meng G, Wang Y, Panm C, Zhang C. Image deblurring with coupled dictionary learning. International Journal of Computer Vision 2015; 114 (2): 248-271.
[22] Happy SL, Routray A. Robust facial expression classification using shape and appearance features. In: 8th International Conference on Advances in Pattern Recognition; Kolkata, India; 2015. pp. 1-5.
[23] Burkert P, Trier F, Afzal MZ, Dengel A, Liwicki M. DeXpression: Deep convolutional neural network for expression recognition. CoRR 2015; abs/1509.05371.
[24] Ouyang Y, Sang N, Huang R. Accurate and robust facial expressions recognition by fusing multiple sparse representation based classifiers. Neurocomputing 2015; 149: 71-78.
[25] Battini Sönmez E, Albayrak S. A facial component-based system for emotion classification. Turkish Journal of Electrical Engineering and Computer Sciences 2016; 28 (3): 1663-1673.
[26] Feng Q, Yuan C, Pan JS, Yang JF, Chou YT et al. Superimposed sparse parameter classifiers for face recognition. IEEE Transactions on Cybernetics 2017; 47 (2): 378-390.
[27] Moeini A, Faez K, Moeini H, Safai AM. Facial expression recognition using dual dictionary learning. Journal of Visual Communication and Image Representation 2017; 45 (C): 20-33.
[28] Wen J, Xu Y, Li Z, Ma Z, Xu Y. Inter-class sparsity based discriminative least square regression. Neural Networks 2018; 102: 36-47.
[29] Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y. Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 2009; 31 (2): 210-227.
[30] Yang M, Zhang L, Feng X, Zhang D. Sparse representation based Fisher discrimination dictionary learning for image classification. International Journal of Computer Vision 2014; 109 (3): 209-232.
[31] Baraniuk RG. Compressive sensing [lecture sotes]. IEEE Signal Processing Magazine 2007; 24 (4): 118-121.
[32] Candes EJ, Wakin MB. An introduction to compressive sampling. IEEE Signal Processing Magazine 2008; 25 (2): 21-30.
[33] Donoho DL. Compressed sensing. IEEE Transactions on Information Theory 2006; 52 (4): 1289-1306.
[34] Battini Sönmez E. Robust Classification Based on Sparsity. 1st ed. Saarbrucken, Germany: LAP Lambert Academic Publishing, 2013.
[35] Battini Sönmez E, Albayrak S. Critical parameters of the sparse representation-based classifier. IET Computer Vision Journal 2013; 7 (6): 500-507.
[36] De La Torre F, Chu W, Xiong X, Vicente F, Cohn JF. Intraface. In: IEEE International Conference on Face and Gesture Recognition; Slovenia; 2015. pp. 1-8.
[37] Shan C, Gong S, McOwan PW. Facial expression recognition based on Local Binary Patterns: a comprehensive study. Image and Vision Computing 2009; 27 (6): 803-816.
[38] Battini Sönmez E, Cangelosi A. Convolutional neural networks with balanced batches for facial expressions recognition. In: SPIE 10341, Ninth International Conference on Machine Vision (ICMV); Nice, France; 2016. doi: 10.1117/12.2268412
[39] Lopes AT, Aguiar Ed, Souza AF, Oliveira-Santos T. Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recognition 2017; 61: 610-628.
[40] Yang P, Yang H, Wei Y, Tang X. Geometry-based facial expression recognition via large deformation: diffeomorphic metric curve mapping. In: 25th IEEE International Conference on Image Processing; Athens, Greece; 2018. pp. 1937-1941.