Mehmet Fatih GÜNDOĞAR, Çiğdem EROĞLU ERDEM

Presentation attack detection for face recognition using remote photoplethysmography and cascaded fusion

Spoofing (presentation) attacks are important threats for face recognition and authentication systems, which try to deceive them by presenting an image or video of a different subject, or by using a 3D mask. Remote (non-contact) photoplethysmography (rPPG) is useful for liveness detection using a facial video by estimating the heart-rate of the subject. In this paper, we first compare the presentation attack detection performance of three different rPPG-based heart rate estimation methods on four datasets (3DMAD, Replay-Attack, Replay-Mobile, and MSU-MFSD). We also present a cascaded fusion system, which utilizes a multistage ensemble of classifiers using rPPG, motion-based (including head-pose, eye-gaze and eye-blink), and texture-based features. Experimental results show that the proposed method outperforms several other presentation attack detection methods in the literature, which utilize rPPG.

PDF

___

[1] Chen X, Cheng J, Song R, Liu Y, Ward R et al. Video-based heart rate measurement: recent advances and future prospects. IEEE Transactions on Instrumentation and Measurement 2019; 68 (10): 3600-3615. doi: 10.1109/TIM.2018.2879706
[2] Li X, Komulainen J, Zhao G, Yuen PC, Pietikäinen M. Generalized face anti-spoofing by detecting pulse from face videos. In: 2016 23rd International Conference on Pattern Recognition (ICPR); Cancun; 2016. pp. 4244-4249. doi: 10.1109/ICPR.2016.7900300
[3] Heusch G, Marcel S. Pulse-based features for face presentation attack detection. In: 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS); Redondo Beach, CA, USA; 2018. pp. 1-8. doi: 10.1109/BTAS.2018.8698579
[4] Ramachandra R, Busch C. Presentation attack detection methods for face recognition systems: a comprehensive survey. ACM Computing Surveys 2017; 50 (1): 8. doi: 10.1145/3038924
[5] Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002; 24 (7): 971–987. doi: 10.1109/TPAMI.2002.1017623
[6] Asthana A, Zafeiriou S, Cheng S, Pantic M. Robust discriminative response map fitting with constrained local models. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition; Portland, OR; 2013. pp. 3444- 3451. doi: 10.1109/CVPR.2013.442
[7] Hernandez-Ortega J, Fierrez J, Morales A, Tome P. Time analysis of pulse-based face anti-spoofing in visible and NIR. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW); Salt Lake City, UT; 2018. pp. 657-6578. doi: 10.1109/CVPRW.2018.00096
[8] Nowara EM, Sabharwal A, Veeraraghavan A. PPGSecure: biometric presentation attack detection using photopletysmograms. In: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017); Washington, DC; 2017. pp. 56-62. doi: 10.1109/FG.2017.16
[9] Ciftci UA, Demir I, Yin L. FakeCatcher: detection of synthetic portrait videos using biological signals. IEEE Transactions on Pattern Analysis and Machine Intelligence 2020; pp. 1-1. doi: 10.1109/TPAMI.2020.3009287
[10] Shao R, Lan X, Yuen PC. Joint discriminative learning of deep dynamic textures for 3D mask face anti-spoofing. IEEE Transactions on Information Forensics and Security 2019; 14 (4): 923-938. doi: 10.1109/TIFS.2018.2868230
[11] Li H, He P, Wang S, Rocha A, Jiang X et al. Learning generalized deep feature representation for face anti-spoofing. IEEE Transactions on Information Forensics and Security 2018; 13 (10): 2639-2652. doi: 10.1109/TIFS.2018.2825949
[12] Song X, Zhao X, Fang L, Lin T. Discriminative representation combinations for accurate face spoofing detection. Pattern Recognition 2019; 85: 220-231. doi: 10.1016/j.patcog.2018.08.019
[13] Mohammadi A, Bhattacharjee S, Marcel S. Improving cross-dataset performance of face presentation attack detection systems using face recognition datasets. In: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); Barcelona, Spain; 2020. pp. 2947-2951. doi: 10.1109/ICASSP40776.2020.9053922
[14] Li Z, Li H, Lam K, Kot AC. Unseen face presentation attack detection with hypersphere loss. In: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); Barcelona, Spain; 2020. pp. 2852-2856. doi: 10.1109/ICASSP40776.2020.9054420
[15] Liu A, Wan J, Escalera S, Escalante HJ, Tan Z et al. Multi-modal face anti-spoofing attack detection challenge at CVPR2019. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW); Long Beach, CA, USA; 2019. pp. 1601-1610. doi: 10.1109/CVPRW.2019.00202
[16] Yang X, Luo W, Bao L, Gao Y, Gong D et al. Face anti-spoofing: model matters, so does data. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); Long Beach, CA, USA; 2019. pp. 3502-3511. doi: 10.1109/CVPR.2019.00362
[17] Patel K, Han H, Jain A, Ott G. Live face video vs. spoof face video: use of moiré patterns to detect replay video attacks. In: 2015 International Conference on Biometrics (ICB); Phuket; 2015. pp. 98-105. doi: 10.1109/ICB.2015.7139082
[18] Chingovska I, Anjos A. On the use of client identity information for face antispoofing. IEEE Transactions on Information Forensics and Security 2015; 10 (4): 787-796. doi: 10.1109/TIFS.2015.2400392
[19] Hao H, Pei M, Zhao M. Face liveness detection based on client identity using Siamese network. In: Chinese Conference on Pattern Recognition and Computer Vision (PRCV); 2019. pp. 172-180.
[20] Chingovska I, Anjos A, Marcel S. On the effectiveness of local binary patterns in face anti-spoofing. In: 2012 BIOSIG - Proceedings of the International Conference of Biometrics Special Interest Group (BIOSIG); Darmstadt; 2012. pp. 1-7.
[21] Costa-Pazo A, Bhattacharjee S, Vazquez-Fernandez E, Marcel S. The Replay-Mobile face presentation-attack database. In: 2016 International Conference of the Biometrics Special Interest Group (BIOSIG); Darmstadt; 2016. pp. 1-7. doi: 10.1109/BIOSIG.2016.7736936
[22] Wen D, Han H, Jain AK. Face spoof detection with image distortion analysis. IEEE Transactions on Information Forensics and Security 2015. 10 (4): 746-761. doi: 10.1109/TIFS.2015.2400395.
[23] Erdogmus N, Marcel S. Spoofing in 2D face recognition with 3D masks and anti-spoofing with Kinect. In: 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS); Arlington, VA; 2013. pp. 1-6. doi: 10.1109/BTAS.2013.6712688
[24] Zhang Z, Yan J, Liu S, Lei Z, Yi D et al. A face antispoofing database with diverse attacks. In: 2012 5th IAPR International Conference on Biometrics (ICB); New Delhi; 2012. pp. 26-31. doi: 10.1109/ICB.2012.6199754
[25] Boulkenafet Z, Komulainen J, Li L, Feng X, Hadid A. OULU-NPU: a mobile face presentation attack database with real-world variations. In: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017); Washington, DC; 2017. pp. 612-618. doi: 10.1109/FG.2017.77
[26] Dolhansky B, Howes R, Pflaum B, Baram N, Ferrer CC. The Deepfake Detection Challenge (DFDC) preview dataset. arXiv preprint 2019, arXiv:191008854.
[27] Viola P, Jones M. Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001); Kauai, HI, USA; 2001. pp. I-I. doi: 10.1109/CVPR.2001.990517
[28] Wang W, Stuijk S, de Haan G. A novel algorithm for remote photoplethysmography: spatial subspace rotation. IEEE Transactions on Biomedical Engineering 2016; 63 (9): 1974-1984. doi: 10.1109/TBME.2015.2508602
[29] Demirezen H, Erdem CE. Remote photoplethysmography using nonlinear mode decomposition. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); Calgary, AB; 2018. pp. 1060-1064. doi: 10.1109/ICASSP.2018.8462538
[30] Demirezen H, Erdem CE. Heart rate estimation from facial videos using nonlinear mode decomposition and improved consistency check. Signal, Image and Video Processing (SIVP) 2021; doi:10.1007/s11760-021-01873-x
[31] de Haan G, Jeanne V. Robust pulse rate from chrominance-based rPPG. IEEE Transactions on Biomedical Engineering 2013; 60 (10): 2878-2886. doi: 10.1109/TBME.2013.2266196
[32] Li X, Chen J, Zhao G, Pietikäinen M. Remote heart rate measurement from face videos under realistic situations. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition; Columbus, OH; 2014. pp. 4264-4271. doi: 10.1109/CVPR.2014.543
[33] Rouast PV, Adam MTP, Chiong R, Cornforth D, Lux E. Remote heart rate measurement using low-cost RGB face video: a technical literature review. Frontiers of Computer Science 2018; 12: 858-872. doi: 10.1007/s11704-016- 6243-6
[34] Anjos A, Shafey E, Wallace R, Günther M, Mccool C et al. Bob: a free signal processing and machine learning toolbox for researchers. In: Proceedings of the 20th ACM international conference on Multimedia; New York, NY, USA; 2012. pp. 1449–1452. doi: 10.1145/2393347.2396517
[35] Baltrusaitis T, Zadeh A, Lim YC, Morency L. OpenFace 2.0: facial behavior analysis toolkit. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018); Xi’an; 2018. pp. 59-66. doi: 10.1109/FG.2018.00019
[36] Zadeh A, Lim YC, Baltrušaitis T, Morency L. Convolutional experts constrained local model for 3D facial landmark detection. In: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW); Venice; 2017. pp. 2519-2528. doi: 10.1109/ICCVW.2017.296
[37] Baltrusaitis T, Robinson P, Morency L. Constrained local neural fields for robust facial landmark detection in the wild. In: 2013 IEEE International Conference on Computer Vision Workshops; Sydney, NSW; 2013. pp. 354-361. doi: 10.1109/ICCVW.2013.54
[38] Wood E, Baltruaitis T, Zhang X, Sugano Y, Robinson P et al. Rendering of eyes for eye-shape registration and gaze estimation. In: 2015 IEEE International Conference on Computer Vision (ICCV); Santiago; 2015. pp. 3756-3764. doi: 10.1109/ICCV.2015.428
[39] Baltrušaitis T, Mahmoud M, Robinson P. Cross-dataset learning and person-specific normalisation for automatic action unit detection. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG); Ljubljana; 2015. pp. 1-6. doi: 10.1109/FG.2015.7284869
[40] Banzhaf C. Extracting facial data using feature-based image processing and correlating it with alternative biosensors metrics. MSc, University of Stuttgart, Stuttgart, Germany, 2017.
[41] Alotaibi A, Mahmood A. Deep face liveness detection based on nonlinear diffusion using convolution neural network. Signal, Image and Video Processing (SIVP) 2017; 11: 713-720. doi: 10.1007/s11760-016-1014-2
[42] Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B et al. Scikit-learn: Machine learning in python. The Journal of Machine Learning Research 2011; 12: 2825-2830.
[43] Geurts P, Ernst D, Wehenkel L. Extremely randomized trees. Machine Learning 2006; 63 (1): 3–42. doi: 10.1007/s10994-006-6226-1
[44] Quinlan JR. Induction of decision trees. Machine Learning 1986; 1: 81–106. doi: 10.1007/BF00116251
[45] Erdogmus N, Marcel S. Spoofing face recognition with 3D masks. IEEE Transactions on Information Forensics and Security 2014; 9 (7): 1084-1097. doi: 10.1109/TIFS.2014.2322255
[46] Maaten L, Hinton G. Visualizing Data using t-SNE. The Journal of Machine Learning Research 2008; 9 (86): 2579-2605.