Erhan AKIN, Mehmet SEVİ, İlhan AYDIN, Mehmet Umut SALUR

efect classification of railway fasteners using image preprocessing and a lightweight convolutional neural network

Railway fasteners are used to securely fix rails to sleeper blocks. Partial wear or complete loss of these components can lead to serious accidents and cause train derailments. To ensure the safety of railway transportation, computer vision and pattern recognition-based methods are increasingly used to inspect railway infrastructure. In particular, it has become an important task to detect defects in railway tracks. This is challenging since rail track images are acquired using a measuring train in varying environmental conditions, at different times of day and in poor lighting conditions, and the resulting images often have low contrast. In this study, a new method is proposed for the classification of defects on rail track fasteners. The proposed approach uses image enhancement to first filter the rail images and obtain a high contrast image. Then, the rail track and sleeper positions are determined from the high contrast image. The location of the fastener is determined by applying the line local binary pattern method and the defects of the fastener are classified using an improved lightweight convolutional neural network (LCNN) model. Features are extracted from two fully connected layers of the developed LCNN model and the feature vector is constructed by concatenating these layers. The concatenated features are processed using a number of machine learning methods and the optimum classifier is chosen. Experimental results show that Cubic SVM gives the best results with a detection accuracy rate of 99.7%.

PDF

___

[1] Zuwen L. Overall comments on track technology of high-speed railway. Journal of Railway Engineering Society 2007; 1: 41-54.
[2] Feng H, Jiang Z, Xie F, Yang P, Shi J et al. Automatic fastener classification and defect detection in vision-based railway inspection systems. IEEE Transactions on Instrumentation and Measurement 2013; 63 (4): 877-888.
[3] Peng Z, Wang C, Ma Z, Liu H. A multi-feature hierarchical locating algorithm for hexagon nut of railway fasteners. IEEE Transactions on Instrumentation and Measurement 2019; 69 (3): 693-699.
[4] Utrata D, Clark R. Groundwork for rail flaw detection using ultrasonic phased array inspection. In AIP Conference Proceedings 2003; 657 (1): 799-805.
[5] Chen Q, Niu X, Zuo L, Zhang T, Xiao F et al. A railway track geometry measuring trolley system based on aided INS. Sensors 2018; 18 (2): 538.
[6] Gan J, Wang J, Yu H, Li Q, Shi Z. Online rail surface inspection utilizing spatial consistency and continuity. IEEE Transactions on Systems, Man, and Cybernetics: Systems 2018; 50 (7): 2741-2751.
[7] Zhang H, Jin X, Wu QJ, Wang Y, He Z et al. Automatic visual detection system of railway surface defects with curvature filter and improved gaussian mixture model. IEEE Transactions on Instrumentation and Measurement 2018; 67 (7): 1593-1608.
[8] Gibert X, Patel VM, Chellappa R. Deep multitask learning for railway track inspection. IEEE Transactions on Intelligent Transportation Systems 2016; 18 (1): 153-164.
[9] Liu J, Huang Y, Zou Q, Tian M, Wang S et al. Learning visual similarity for inspecting defective railway fasteners. IEEE Sensors Journal 2019; 19 (16): 6844-6857.
[10] Guo F, Qian Y, Shi Y. Real-time railroad track components inspection based on the improved YOLOv4 framework. Automation in Construction 2021; 125: 103596.
[11] Bai T, Yang J, Xu G, Yao D. An optimized railway fastener detection method based on modified Faster R-CNN. Measurement 2021; 182. doi: 10.1016/j.measurement.2021.109742.
[12] Dou Y, Huang Y, Li Q, Luo S. A fast template matching-based algorithm for railway bolts detection. International Journal of Machine Learning and Cybernetics 2014; 5 (6): 835-844.
[13] He B, Luo J, Ou Y, Xiong Y, Li B. Railway fastener defects detection under various illumination conditions using fuzzy c-means part model. Transportation Research Record 2020; 2675 (4): 271-280.
[14] Wang L, Zhang B, Wu J, Xu H, Chen X et al. Computer vision system for detecting the loss of rail fastening nuts based on kernel two-dimensional principal component–two-dimensional principal component analysis and a support vector machine. Proceedings of the Institution of Mechanical Engineers, Part F: Journal of Rail and Rapid Transit 2016; 230 (8): 1842-1850.
[15] Xia Y, Xie F, Jiang Z. Broken railway fastener detection based on adaboost algorithm. In 2010 International Conference on Optoelectronics and Image Processing 2010; 1: 313-316.
[16] Ni X, Liu H, Ma Z, Wang C, Liu J. Detection for rail surface defects via partitioned edge feature. IEEE Transactions on Intelligent Transportation Systems 2021; 1-17. doi: 10.1109/TITS.2021.3058635.
[17] Karaduman G, Karaköse M, Aydın İ, Akın E. Contactless rail profile measurement and rail fault diagnosis approach using featured pixel counting. Intelligent Automation and Soft Computing 2020; 26 (3): 455-463.
[18] Liu J, Teng Y, Ni X, Liu H. A fastener inspection method based on defective sample generation and deep convolutional neural network. IEEE Sensors Journal 2021; 21 (10): 12179-12188.
[19] Wei X, Yang Z, Liu Y, Wei D, Jia L et al. Railway track fastener defect detection based on image processing and deep learning techniques: a comparative study. Engineering Applications of Artificial Intelligence 2019; 80: 66-81.
[20] Mery D, Pedreschi F. Segmentation of colour food images using a robust algorithm. Journal of Food Engineering 2005; 66 (3): 353-360.
[21] Otsu N. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics 1979; 9 (1): 62-66.
[22] Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002; 24 (7): 971-987.
[23] Fan H, Cosman PC, Hou Y, Li B. High-speed railway fastener detection based on a line local binary pattern. IEEE Signal Processing Letters 2018; 25 (6): 788-792.
[24] Singh N, Sabrol H. Convolutional neural networks-an extensive arena of deep learning. A comprehensive study. Archives of Computational Methods in Engineering 2021; 1-26. doi: 10.1007/s11831-021-09551-4
[25] Mahalakshmi P, Fatima NS. Ensembling of text and images using deep convolutional neural networks for intelligent information retrieval. Wireless Personal Communications 2021; 1-19. doi: 10.1007/s11277-021-08211-x
[26] Van der Maaten L, Hinton G. Visualizing data using t-SNE. Journal of Machine Learning Research 2008; 9 (86): 2579-2605.
[27] Santur Y, Karaköse M, Akın E. An adaptive fault diagnosis approach using pipeline implementation for railway inspection. Turkish Journal of Electrical Engineering & Computer Sciences 2018; 26 (2): 987-998.