Nasir AHMAD, Khalil KHAN, Khalil ULLAH, Irfanud DIN

Multiclass semantic segmentation of faces using CRFs

Multiclass semantic image segmentation is widely used in a variety of computer vision tasks, such as object segmentation and complex scene understanding. As it decomposes an image into semantically relevant regions, it can be applied in segmentation of face images. In this paper, an algorithm based on multiclass semantic segmentation of faces is proposed using conditional random fields. In the proposed model, each node corresponds to a superpixel, while the neighboring superpixels are connected to nodes through edges. Unlike previous approaches, which rely on three or four classes, the label set is extended here to six classes, i.e. hair, eyes, nose, mouth, skin, and background. The proposed framework is evaluated on standard face databases FASSEG, FIGARO, and LFW. Experimental results reveal that the performance of the proposed model is comparable with state-of-the-art techniques on these standard databases.

PDF

___

[1] Ferrara M, Franco A, Maio D. A multi-classifier approach to face image segmentation for travel documents. Exper Syst Appl 2012; 9: 8452-8466.
[2] Liu C, Yuen J, Torralba A. Nonparametric scene parsing via label transfer. IEEE T Pattern Anal 2011; 33: 2368- 2382.
[3] Liu S, Yang J, Huang C, Yang MH. Multi-objective convolutional learning for face labeling. In: Proc CVPR IEEE; 810 June 2015; Boston, MA, USA: IEEE. pp. 3451-3459.
[4] Everingham M, Van GL, Williams CK, Winn J, Zisserman A. The Pascal visual object classes challenge. Int J Comput Vision 2010; 2: 303-338.
[5] Huang GB, Narayana M, Learned-Miller E. Towards unconstrained face recognition. In: Proc CVPR IEEE; 2426 June 2008; Alaska, USA: IEEE. pp. 1-8.
[6] Zebrowitz LA, Montepare JM. Social psychological face perception: why appearance matters. Soc Personal Psychol Compass 2008; 3: 1497-1517.
[7] Shepherd J, Ellis H, Davies G. Perceiving and Remembering Faces. 1st ed. Chicago, IL, USA: Academic Press, 1981.
[8] Sinha P, Balas B, Ostrovsky Y, Russell R. Face recognition by humans: nineteen results all computer vision researchers should know about. Proc IEEE 2006; 11: 1948-1962.
[9] Moon JT, Marschner SR. Simulating multiple scattering in hair using a photon mapping approach. ACM T Graphic 2006; 25: 1067-1074.
[10] Ward K, Bertails F, Kim TY, Marschner SR, Cani MP, Lin MC. A survey on hair modelling: styling, simulation, and rendering. IEEE T Vis Comput Gr 2007; 2: 213-234.
[11] Lee C, Schramm MT, Boutin M, Allebach JP. An algorithm for automatic skin smoothing in digital portraits. In: IEEE Image Proc; 710 September 2009; Cairo, Egypt: IEEE. pp. 3113-3116.
[12] Chen CW, Huang DY, Fuh CS. Automatic skin color beautification. In: Proceedings of the International Conference on Arts and Technology; 1012 December 2010; Berlin, Germany: IEEE. pp. 157-164.
[13] Xu L, Du Y, Zhang Y. An automatic framework for example-based virtual makeup. In: IEEE Image Proc; 1518 September 2013; Melbourne, Australia: IEEE. pp. 3206-3210.
[14] Khan K, Mauro M, Leonardi R. Multi-class semantic segmentation of faces. In: IEEE Image Proc; 2730 September 2015; Qu´ebec, Canada: IEEE. pp. 827-831.
[15] Lafferty J, McCallum A, Pereira F. Conditional random fields: probabilistic models for segmenting and labelling sequence data. In: Proceedings of the International Conference on Machine Learning; June 28July 1 2001; Williamstown, MA, USA: pp. 282-289.
[16] Kae A, Sohn K, Lee H, Learned ME. Augmenting CRFs with Boltzmann machine shape priors for image labelling. In: Proc CVPR IEEE; 2527 June 2013; Portland, OR, USA: IEEE. pp. 2019-2026.
[17] Eslami SA, Heess N, Williams CK, Winn J. The shape Boltzmann machine: a strong model of object shape. Int J Comput Vision 2014; 2: 155-176.
[18] Li Y, Wang S, Ding X. Person-independent head pose estimation based on random forest regression. In: IEEE Image Proc; 2629 September 2010; Hong Kong, China: IEEE. pp. 1521-1524.
[19] Scheffler C, Odobez JM. Joint adaptive colour modelling and skin, hair and clothing segmentation using coherent probabilistic index maps. In: Proceedings of the British Machine Vision Conference; 29 August2 September 2011; Dundee, UK: BMVA Press.
[20] Svanera M, Muhammad UR, Leonardi R, Benini S. Figaro, hair detection and segmentation in the wild. In: IEEE Image Proc; 2528 September 2016; Phoenix, USA: IEEE. pp. 546-550.
[21] Gourier N, Hall D, Crowley JL. Estimating face orientation from robust detection of salient facial features. In: Proceedings of the International Workshop on Visual Observation of Deictic Gestures; 22 August 2004; Cambridge, UK.
[22] Laurentini A, De SM, Bottino AG, Vieira TF. A new problem in face image analysis: finding kinship clues for sibling pairs. In: Proceedings of the International Conference on Pattern Recognition Application and Methods; 68 February 2012; Vilamoura, Algarve, Portugal: Springer. pp. 153-162.
[23] Van DB, Michael, Xavier B, Gemma R, Benjamin DC, Luc VG. Seeds: Super-pixels extracted via energy-driven sampling. In: Proceedings of the European Conference on Computer Vision; 713 October 2012; Firenze, Italy: Springer. pp. 13-26.
[24] Dalal N, Triggs B. Histograms of oriented gradients for human detection. In: Proc CVPR IEEE; 2026 June 2005; San Diego, CA, USA: IEEE. pp. 886-893.
[25] Zhao W, Chellappa R, Phillips PJ, Rosenfeld A. Face recognition: a literature survey. ACM Computing Surveys 2003; 4: 399-458.
[26] Wang N, Ai H, Lao S. A compositional exemplar-based model for hair segmentation. In: Proceedings of the Asian Conference on Computer Vision; 812 November 2010; New Zealand: Springer. pp. 171-184.
[27] Lee KC, Anguelov D, Sumengen B, Gokturk SB. Markov random field models for hair and face segmentation. In: Proceedings of the International Conference on Automatic Face & Gesture Recognition; 1719 September 2008; Amsterdam, the Netherlands: IEEE. pp. 1-6.