Shahidul Islam KHAN, Mysha Sarin KABİSHA, Kazi Anisa RAHİM, Md. KHALİLUZZAMAN

Face and Hand Gesture Recognition Based Person Identification System using Convolutional Neural Network

Person identification system is now become the most hyped system for security purpose. It’s also gaining a lot of attention in the field of computer vision. For verification of human, facial recognition and hand gesture recognition are the most common topics of research. In the current days, various researchers focused on facial and hand gesture recognition using various shallow techniques and Deep Convolutional Neural Network (DCNN). However, using one feature of human for person identification is the most researched topic till now. In this paper, we proposed a Convolutional Neural Network (CNN) based system which will identify a person using two traits i.e., face and hand gesture of number sign of that person. For feature extraction and recognition Neural Network have shown immense good result. This proposed system works on two models, one is a VGG16 architecture model for face recognition and another model is for hand gesture which is based on simple CNN with two convolutional layers. With two customized dataset our face model gained 98.00% accuracy and hand gesture (number sign) model gained an accuracy of 98.33%.

PDF

___

[1] B. Kranthikiran, and P. Pulicherla, “Face Detection and Recognition for use in Campus Surveillance,” International Journal of Innovative Technology and Exploring Engineering, vol. 9, no. 3, pp. 2908–2913, Jan. 2020, doi: 10.35940/ijitee.b7682.019320.
[2] M. Coşkun, A. Uçar, Ö. Yildirim, and Y. Demir, “A FACE RECOGNITION ALGORITHM BASED ON CONVOLUTIONAL NEURAL NETWORK,” REVISTA ARGENTINA DE CLINICA PSICOLOGICA, 2020, doi: 10.24205/03276716.2020.339.
[3] Y. Said, M. Barr, and H. E. Ahmed, “Design of a Face Recognition System based on Convolutional Neural Network (CNN),” Engineering, Technology & Applied Science Research, vol. 10, no. 3, pp. 5608–5612, Jun. 2020, doi: 10.48084/etasr.3490.
[4] K. B. Pranav, and J. Manikandan, “Design and Evaluation of a Real- Time Face Recognition System using Convolutional Neural Networks,” Procedia Computer Science, vol. 171, pp. 1651–1659, Jan. 2020, doi: 10.1016/j.procs.2020.04.177.
[5] G. Hu, Y. Yang, D. Yi, J. Kittler, W. Christmas, S. Z. Li, & T. Hospedales, “When face recognition meets with deep learning: an evaluation of convolutional neural networks for face recognition,” In Proceedings of the IEEE international conference on computer vision workshops, pp. 142-150, IEEE, 2015.
[6] Z. Pei, H. Xu, Y. Zhang, M. Guo, and Y.-H. Yang, “Face Recognition via Deep Learning Using Data Augmentation Based on Orthogonal Experiments,” Electronics, vol. 8, no. 10, p. 1088, Sep. 2019, doi: 10.3390/electronics8101088.
[7] P. Gupta, N. Saxena, M. Sharma, and J. Tripathi, “Deep learning model for group face recognition based on Convolution Neural Network,” Journal of Xidian University, vol. 14, no. 5, May 2020, doi: 10.37896/jxu14.5/415.
[8] Z. Chen, J.-T. Kim, J. Liang, J. Zhang, and Y.-B. Yuan, “Real- Time Hand Gesture Recognition Using Finger Segmentation,” The Scientific World Journal, vol. 2014, p. e267872, Jun. 2014, doi: 10.1155/2014/267872.
[9] R. F. Pinto, C. D. B. Borges, A. M. A. Almeida, and I. C. Paula, “Static Hand Gesture Recognition Based on Convolutional Neural Networks,” Journal of Electrical and Computer Engineering, vol. 2019, pp. 1–12, Oct. 2019, doi: 10.1155/2019/4167890.
[10] J. Chen, S. Bi, G. Zhang, and G. Cao, “High-Density Surface EMG-Based Gesture Recognition Using a 3D Convolutional Neural Network,” Sensors, vol. 20, no. 4, p. 1201, Feb. 2020, doi: 10.3390/s20041201.
[11] P. Molchanov, S. Gupta, K. Kim, & J. Kautz, “Hand gesture recognition with 3D convolutional neural network,” In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 1-7, IEEE, 2015.
[12] L. Pigou, S, Dieleman, P. J. Kindermans, and B. Schrauwen, “Sign language recognition using convolutional neural networks,” In European Conference on Computer Vision, pp. 572-578, Springer, Chamm, 2014, September.
[13] M.R. Islam, U.K. Mitu, R.A. Bhuiyan, and J. Shin, “Hand gesture feature extraction using deep convolutional neural network for recognizing American Sign Language,” In 2018 4th International Conference on Frontiers of Signal Processing (ICFSP), pp. 115- 119, IEEE, 2018, September.
[14] M. M. Islam, M. R. Islam, and M. S. Islam, “An Efficient Human Computer Interaction through Hand Gesture Using Deep Convolutional Neural Network,” SN Computer Science, vol. 1, no. 4, Jun. 2020, doi: 10.1007/s42979-020-00223-x.
[15] Simonyan, K. and Zisserman, A., 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
[16] D. P. Kingma, & J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014. [17] J. Duchi, E. Hazan, & Y. Singer, “Adaptive Subgradient Methods for Online Learning and Stochastic Optimization,” Journal of Machine Learning Research, Vol.12, 2121–2159, 2011.
[18] S. Ruder, “An overview of gradient descent optimization algorithms,” ArXiv, abs/1609.04747, 2016.
[19] https://www.kaggle.com/muhammadkhalid/sign-language-fornumbers (21 February, 2021).
[20] I. Cinar, and M. Koklu, “Classification of rice varieties using artificial intelligence methods,” International Journal of Intelligent Systems and Applications in Engineering, 7(3), pp.188-194, 2019.
[21] A. Krizhevsky, I. Sutskever, & G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in neural information processing systems, 25, 2012.
[22] Y. S. Taspinar, I. Cinar, & M. Koklu, “Prediction of Computer Type Using Benchmark Scores of Hardware Units,” Selcuk University Journal of Engineering Sciences, 20(1), 11-17, 2021.