Mustafa Altin, Murat Koklu, Yavuz Selim Taspinar

Fire Detection in Images Using Framework Based on Image Processing, Motion Detection and Convolutional Neural Network

Fire detection in images has been frequently used recently to detect fire at an early stage. These methods play an important role in reducing the loss of life and property. Fire is not only chemically complex, but also physically very complex. The shape and color of the flame varies according to the type of fuel in the fire. This has made fire detection a very challenging problem. Advanced image processing algorithms are also needed to accurately detect fire. To solve this problem, a three-stage fire framework was created in this study. In the first stage, the flame region was extracted from the images containing the fire region with the basic image processing algorithms. At this stage, reduce brightness, HSL, YCbCr, median and herbaceous filters are applied successively to the image. Since the flame image has a polygonal structure by nature, the number of edges of the flame region has been found. In the second stage, the mobility feature of the flame was utilized. For this purpose, the mobility of the flame was determined by comparing the video frames containing the fire image. The CNN method was used to detect the fire in the images. The CNN model was trained with the transfer learning method using the Inception V3, SequeezeNet, VGG16 and VGG19 trained models. As a result of the tests of the models, 98.8%, 97.0%, 97.3% and 96.8% classification success were obtained, respectively. With the proposed fire detection framework, it is thought that the damage caused by the fire can be reduced by early detection of the fire and timely intervention.

PDF

___

[1] Di Biase, V. and G. Laneve, Geostationary sensor based forest fire detection and monitoring: An improved version of the SFIDE algorithm. Remote Sensing, 2018. 10(5): p. 741.
[2] Li, P. and W. Zhao, Image fire detection algorithms based on convolutional neural networks. Case Studies in Thermal Engineering, 2020. 19: p. 100625.
[3] Derbel, F., Performance improvement of fire detectors by means of gas sensors and neural networks. Fire Safety Journal, 2004. 39(5): p. 383-398.
[4] Keller, A., et al., Open photoacoustic sensor as smoke detector. Sensors and Actuators B: Chemical, 2005. 104(1): p. 1-7.
[5] Fonollosa, J., A. Solórzano, and S. Marco, Chemical sensor systems and associated algorithms for fire detection: A review. Sensors, 2018. 18(2): p. 553.
[6] Stec, A.A., Fire toxicity–The elephant in the room? Fire Safety Journal, 2017. 91: p. 79-90.
[7] Qureshi, W.S., et al., QuickBlaze: early fire detection using a combined video processing approach. Fire technology, 2016. 52(5): p. 1293-1317.
[8] Chen, T.-H., P.-H. Wu, and Y.-C. Chiou. An early fire-detection method based on image processing. in 2004 International Conference on Image Processing, 2004. ICIP'04. 2004. IEEE.
[9] Marbach, G., M. Loepfe, and T. Brupbacher, An image processing technique for fire detection in video images. Fire safety journal, 2006. 41(4): p. 285-289.
[10] Töreyin, B.U., et al., Computer vision based method for real-time fire and flame detection. Pattern recognition letters, 2006. 27(1): p. 49-58.
[11] Han, D. and B. Lee. Development of early tunnel fire detection algorithm using the image processing. in International Symposium on Visual Computing. 2006. Springer.
[12] Borges, P.V.K. and E. Izquierdo, A probabilistic approach for vision-based fire detection in videos. IEEE transactions on circuits and systems for video technology, 2010. 20(5): p. 721-731.
[13] Khan, R.A., et al., Machine vision based indoor fire detection using static and dynamic features. International Journal of Control and Automation, 2018. 11(6): p. 87-98.
[14] Seebamrungsat, J., S. Praising, and P. Riyamongkol. Fire detection in the buildings using image processing. in 2014 Third ICT International Student Project Conference (ICT-ISPC). 2014. IEEE.
[15] Krüger, S., et al., Early fire detection: Are hydrogen sensors able to detect pyrolysis of house hold materials? Fire Safety Journal, 2017. 91: p. 1059-1067.
[16] Han, X.-F., et al., Video fire detection based on Gaussian Mixture Model and multi-color features. Signal, Image and Video Processing, 2017. 11(8): p. 1419-1425.
[17] Ko, B., K.-H. Cheong, and J.-Y. Nam, Early fire detection algorithm based on irregular patterns of flames and hierarchical Bayesian Networks. Fire safety journal, 2010. 45(4): p. 262-270.
[18] Celik, T., et al., Fire detection using statistical color model in video sequences. Journal of Visual Communication and Image Representation, 2007. 18(2): p. 176-185.
[19] Phillips Iii, W., M. Shah, and N. da Vitoria Lobo, Flame recognition in video. Pattern recognition letters, 2002. 23(1-3): p. 319-327.
[20] Ko, B.C., K.-H. Cheong, and J.-Y. Nam, Fire detection based on vision sensor and support vector machines. Fire Safety Journal, 2009. 44(3): p. 322-329.
[21] Liu, C.-B. and N. Ahuja. Vision based fire detection. in Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. 2004. IEEE.
[22] Van Hamme, D., et al. Fire detection in color images using Markov random fields. in International conference on advanced concepts for intelligent vision systems. 2010. Springer.
[23] Liu, Z.-G., Y. Yang, and X.-H. Ji, Flame detection algorithm based on a saliency detection technique and the uniform local binary pattern in the YCbCr color space. Signal, Image and Video Processing, 2016. 10(2): p. 277-284.
[24] Celik, T. and H. Demirel, Fire detection in video sequences using a generic color model. Fire safety journal, 2009. 44(2): p. 147-158.
[25] Horng, W.-B., J.-W. Peng, and C.-Y. Chen. A new image-based real-time flame detection method using color analysis. in Proceedings. 2005 IEEE Networking, Sensing and Control, 2005. 2005. IEEE.
[26] Xu, G., et al., Deep domain adaptation based video smoke detection using synthetic smoke images. Fire safety journal, 2017. 93: p. 53- 59.
[27] Nemalidinne, S.M. and D. Gupta, Nonsubsampled contourlet domain visible and infrared image fusion framework for fire detection using pulse coupled neural network and spatial fuzzy clustering. Fire Safety Journal, 2018. 101: p. 84-101.
[28] Dimitropoulos, K., P. Barmpoutis, and N. Grammalidis, Spatiotemporal flame modeling and dynamic texture analysis for automatic video-based fire detection. IEEE transactions on circuits and systems for video technology, 2014. 25(2): p. 339-351.
[29] Muhammad, K., J. Ahmad, and S.W. Baik, Early fire detection using convolutional neural networks during surveillance for effective disaster management. Neurocomputing, 2018. 288: p. 30- 42.
[30] Fan, Z., et al., Dim infrared image enhancement based on convolutional neural network. Neurocomputing, 2018. 272: p. 396- 404.
[31] Zhang, H.-j., N. Zhang, and N.-f. Xiao, Fire detection and identification method based on visual attention mechanism. Optik,[1] Di Biase, V. and G. Laneve, Geostationary sensor based forest fire detection and monitoring: An improved version of the 2015. 126(24): p. 5011-5018.
[32] Rahman, A. and M. Murshed, Detection of multiple dynamic textures using feature space mapping. IEEE transactions on circuits and systems for video technology, 2009. 19(5): p. 766-771.
[33] Gargiulo, M., et al. A CNN-Based Super-Resolution Technique for Active Fire Detection on Sentinel-2 Data. in 2019 PhotonIcs & Electromagnetics Research Symposium-Spring (PIERS-Spring). 2019. IEEE.
[34] Muhammad, K., et al., Efficient fire detection for uncertain surveillance environment. IEEE Transactions on Industrial Informatics, 2019. 15(5): p. 3113-3122.
[35] Sharma, J., et al. Deep convolutional neral networks for fire detection in images. in International conference on engineering applications of neural networks. 2017. Springer.
[36] Foggia, P., A. Saggese, and M. Vento, Real-time fire detection for video-surveillance applications using a combination of experts based on color, shape, and motion. IEEE TRANSACTIONS on circuits and systems for video technology, 2015. 25(9): p. 1545- 1556.
[37] Geetha, S., C. Abhishek, and C. Akshayanat, Machine Vision Based Fire Detection Techniques: A Survey. Fire Technology, 2021. 57(2): p. 591-623.
[38] Kolkur, S., et al., Human skin detection using RGB, HSV and YCbCr color models. arXiv preprint arXiv:1708.02694, 2017.
[39] Erkan, U., L. Gökrem, and S. Enginoğlu, Different applied median filter in salt and pepper noise. Computers & Electrical Engineering, 2018. 70: p. 789-798.
[40] Xiao, L., et al., Adaptive trapezoid region intercept histogram based Otsu method for brain MR image segmentation. Journal of Ambient Intelligence and Humanized Computing, 2021: p. 1-16.
[41] Bora, D.J., A novel approach for color image edge detection using multidirectional Sobel filter on HSV color space. Int. J. Comput. Sci. Eng, 2017. 5(2): p. 154-159.
[42] Zhang, Y.-D., et al., Image based fruit category classification by 13- layer deep convolutional neural network and data augmentation. Multimedia Tools and Applications, 2019. 78(3): p. 3613-3632.
[43] Koklu, M., I. Cinar, and Y.S. Taspinar, Classification of rice varieties with deep learning methods. Computers and Electronics in Agriculture, 2021. 187: p. 106285.
[44] Ahmed, T., et al., Rice grain disease identification using dual phase convolutional neural network based system aimed at small dataset. arXiv preprint arXiv:2004.09870, 2020.
[45] Yurttakal, A.H., et al., Classification of Diabetic Rat Histopathology Images Using Convolutional Neural Networks. International Journal of Computational Intelligence Systems, 2020. 14(1): p. 715-722.
[46] Albawi, S., T.A. Mohammed, and S. Al-Zawi. Understanding of a convolutional neural network. in 2017 International Conference on Engineering and Technology (ICET). 2017. Ieee.
[47] Bicakci, M., et al., Metabolic Imaging Based Sub-Classification of Lung Cancer. IEEE Access, 2020. 8: p. 218470-218476.
[48] Ozkan, I.A., A Novel Basketball Result Prediction Model Using a Concurrent Neuro-Fuzzy System. Applied Artificial Intelligence, 2020. 34(13): p. 1038-1054.
[49] Koklu, M. and I.A. Ozkan, Multiclass classification of dry beans using computer vision and machine learning techniques. Computers and Electronics in Agriculture, 2020. 174: p. 105507.