Farkhod MAKHMUDKHUJAEV, Md. Tauhid Bin IQBAL, Byungyong RYU, Oksam CHAE

Local directional-structural pattern for person-independent facial expression recognition

Existing popular descriptors for facial expression recognition often suffer from inconsistent feature description, experiencing poor accuracies. We present a new local descriptor, local directional-structural pattern (LDSP), in this work to address this issue. Unlike the existing local descriptors using only the texture or edge information to represent the local structure of a pixel, the proposed LDSP utilizes the positional relationship of the top edge responses of the target pixel to extract more detailed structural information of the local texture. We further exploit such information to characterize expression-affiliated crucial textures while discarding the random noisy patterns. Moreover, we introduce a globally adaptive thresholding strategy to exclude futile flat patterns. Hence, LDSP offers a stable description of facial expressions with the explicit representation of the expression-affiliated features along with the exclusion of random futile textures. We visualize the efficacy of the proposed method in three folds. First, the LDSP descriptor possesses a moderate code-length owing to the exclusion of the futile patterns, yielding less computation time than other edge descriptors. Second, for person-independent expression recognition in benchmark datasets, LDSP demonstrates higher accuracy than existing descriptors and other state-of-the-art methods. Third, LDSP shows better performance than other descriptors against noise and low resolution, exhibiting its robustness under such uneven conditions.

Keywords:

Local directional-structural pattern, facial expression recognition, feature extraction, person-independent expression recognition,

PDF

___

Tian YL, Kanade T, Cohn JF. Facial expression analysis. In: Li SZ, Jain AK, editors. Handbook of Face Recognition. New York, NY, USA: Springer, 2005. pp. 247-275.
Shan CF, Gong SG, McOwan PW. Facial expression recognition based on local binary patterns: a comprehensive study. Image Vision Comput 2009; 27: 803-816.
Bourbakis N, Esposito A, Kavraki D. Extracting and associating meta-features for understanding people’s emotional behaviour: face and speech. Cogn Comput 2011; 3: 436–448.
Kotsia I, Pitas I. Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE T Image Process 2007; 16: 172-187.
Pantic M, Bartlett MS. Machine analysis of facial expressions. In: Delac K, Grgic M, editors. Face Recognition. Rijeka, Croatia: Intech Open, 2007. pp. 377-416.
Turk M, Pentland A. Eigenfaces for recognition. J Cognitive Neurosci 1991; 3: 71-86.
Belhumeur PN, Hespanha JP, Kriegman DJ, Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE T Pattern Anal 1997; 19: 711-720.
Yang J, Zhang D, Frangi AF, Yang JY. Two-dimensional PCA: a new approach to appearance-based face repre- sentation and recognition. IEEE T Pattern Anal 2004; 26: 131-137.
Rivera AR, Castillo JR, Chae O. Local directional number pattern for face analysis: face and expression recognition. IEEE T Image Process 2013; 22: 1740-1752
Heisele B, Serre T, Poggio T. A component-based framework for face detection and identification. Int J Comput Vision 2007; 74: 167-181.
Jabid T, Kabir MH, Chae O. Robust facial expression recognition based on local directional pattern. ETRI J 2010; 32: 784-794.
Kirsch RA. Computer determination of the constituent structure of biological images. Comput Biomed Res 1971; 4: 315–328.
Rivera AR, Castillo JR, Chae O. Recognition of face expressions using local principal texture pattern. In: IEEE 2012 International Conference on Image Processing; 30 September–3 October 2012; Orlando, FL, USA. New York, NY, USA: IEEE. pp. 2613–2616.
Iqbal MT, Ryu B, Song G, Chae O. Positional ternary pattern (PTP): an edge based image descriptor for human age recognition. In: IEEE 2016 International Conference on Consumer Electronics; 7–11 January 2016; Las Vegas, NV, USA. New York, NY, USA: IEEE. pp. 289-292.
Iqbal MT, Ryu B, Song G, Kim J, Makhmudkhujaev F, Chae O. Exploring positional ternary pattern (PTP) for conventional facial expression recognition from static images. In: 2016 Conference Proceedings; 29 June–1 July 2016; Jeju, Republic of Korea. pp. 853-855.
Liu M, Li S, Shan S, Chen X. AU-aware deep networks for facial expression recognition. In: IEEE 2013 International Conference and Workshops on Automatic Face and Gesture Recognition; 22–26 April 2013; Shanghai, China. New York, NY, USA: IEEE. pp. 1-6.
Khorrami P, Paine T, Huang T. Do deep neural networks learn facial action units when doing expression recognition? In: IEEE 2015 International Conference on Computer Vision Workshops; 7–13 December 2015; Santiago, Chile. New York, NY, USA: IEEE. pp. 19-27.
Ding H, Zhou SK, Chellappa R. Facenet2expnet: regularizing a deep face recognition net for expression recognition. In: IEEE 2017 International Conference on Automatic Face and Gesture Recognition; 30 May–3 June 2017; Washington, DC, USA. New York, NY, USA: IEEE. pp. 118-126.
Lopes AT, de Aguiar E, De Souza AF, Oliveira-Santos T. Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recogn 2017; 61: 610-28.
Mollahosseini A, Chan D, Mahoor MH. Going deeper in facial expression recognition using deep neural networks. In: IEEE 2016 Winter Conference on Applications of Computer Vision; 7–10 March 2016; Lake Placid, NY, USA. New York, NY, USA: IEEE. pp. 1-10.
Iqbal MT, Shoyaib M, Ryu B, Abdullah-Al-Wadud M, Chae O. Directional age-primitive pattern (DAPP) for human age group recognition and age estimation. IEEE T Inf Foren Sec 2017; 12: 2505-2517.
Chardy P, Glemarec M, Laurec A. Application of inertia methods to benthic marine ecology: practical implications of the basic options. Estuar Coast Mar Sci 1976; 4: 179-205.
Hsu CW, Lin CJ. A comparison of methods for multiclass support vector machines. IEEE T Neural Networ 2002; 13: 415-25.
Happy SL, Patnaik P, Routray A, Guha R. The Indian spontaneous expression database for emotion recognition. IEEE T Affect Comput 2017; 8: 131-142.
Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z, Matthews I. The extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression. In: IEEE 2010 Computer Society Conference on Computer Vision and Pattern Recognition Workshops; 13–18 June 2010; San Francisco, CA, USA. New York, NY, USA: IEEE. pp. 94-101.
Valstar M, Pantic M. Induced disgust, happiness and surprise: an addition to the MMI facial expression database. In: Third International Workshop on EMOTION (Satellite of LREC): Corpora for Research on Emotion and Affect; 19–21 May 2010; Valletta, Malta. p. 65-70.
Yin L, Wei X, Sun Y, Wang J, Rosato MJ. A 3D facial expression database for facial behavior research. In: IEEE 2006 International Conference on Automatic Face and Gesture Recognition; 10–12 April 2006; Southampton, UK. New York, NY, USA: IEEE. pp. 211-216.
Ylioinas J, Hadid A, Guo Y, Pietikäinen M. Efficient image appearance description using dense sampling based local binary patterns. In: Asian Conference on Computer Vision; 5–9 November 2012; Daejeon, Korea. Berlin, Germany: Springer. pp. 375-388.
Lee SH, Baddar WJ, Ro YM. Collaborative expression representation using peak expression and intra class variation face images for practical subject-independent emotion recognition in videos. Pattern Recogn 2016; 54: 52-67.
Lee SH, Plataniotis KN, Ro YM. Intra-class variation reduction using training expression images for sparse representation based facial expression recognition. IEEE T Affect Comput 2014; 5: 340-351.
Ptucha R, Tsagkatakis G, Savakis A. Manifold based sparse representation for robust expression recognition without neutral subtraction. In: IEEE 2011 International Conference on Computer Vision Workshops (ICCV Workshops); 6–13 November 2011; Barcelona, Spain. New York, NY, USA: IEEE. pp. 2136-2143.
Zhang Z, Lu G, Yan J, Li H, Sun N, Li X. Compact local Gabor directional number pattern for facial expression recognition. Turk J Elec Eng & Comp Sci 2018; 26: 1236-1248.
Hu Y, Zeng Z, Yin L, Wei X, Tu J, Huang TS. A study of non-frontal-view facial expressions recognition. In: 19th International Conference on Pattern Recognition; 8–11 December 2008; Tampa, FL, USA. New York, NY, USA: IEEE. pp. 1-4.
Zheng W, Tang H, Lin Z, Huang TS. Emotion recognition from arbitrary view facial images. In: European Conference on Computer Vision; 5–11 September 2010; Heraklion, Greece. Berlin, Germany: Springer-Verlag. pp. 490-503.
Tariq U, Huang TS. Features and fusion for expression recognition — A comparative analysis. In: IEEE 2012 Com- puter Society Conference on Computer Vision and Pattern Recognition Workshops; 16–21 June 2012; Providence, RI, USA. New York, NY, USA: IEEE. pp. 146-152.

Turkish Journal of Electrical Engineering and Computer Science-Cover

ISSN: 1300-0632
Yayın Aralığı: Yılda 6 Sayı
Yayıncı: TÜBİTAK

Arşiv