Dhiren Kumar Behera, Rabinarayan Sethi, Krishna Chandra Patra

Anomaly detection in rotating machinery using autoencoders based on bidirectional LSTM and GRU neural networks

A time series anomaly is a form of anomalous subsequence that indicates future faults will occur. The development of novel techniques for detecting this type of anomaly is significant for real-time system monitoring. Several algorithms have been used to classify anomalies successfully. However, the time series anomaly detection algorithm was not studied well. We use a new bidirectional LSTM and GRU neural networks-based hybrid autoencoder to detect if a machine is operating normally in this research. An autoencoder is trained on a set of 12 features taken from healthy operating data gathered promptly after a planned maintenance period using vibration sensors. The features taken from new data are then reconstructed using the trained model. If the model accurately reconstructs the features, the machine is in good working order. If the reconstruction exceeds a certain error threshold, the machine is functioning strangely and needs to be serviced.

PDF

___

[1] Chandola V, Banerjee A, Kumar V. Anomaly Detection: A Survey. Association for Computing Machinery 2009; 41 (3): PP. 1-58. doi: 10.1145/1541880.1541882
[2] Ariyaluran Habeeb RA, Nasaruddin F, Gani A, Targio Hashem IA, Ahmed E et al. Real-time big data processing for anomaly detection: A Survey. International Journal of Information Management 2019; 45: 289-307. doi: 10.1016/j.ijinfomgt.2018.08.006
[3] Chalapathy R, Chawla S. Deep Learning for Anomaly Detection: A Survey. 2019; doi: 10.48550/arXiv.1901.03407
[4] Fu T. A review on time series data mining. Engineering Applications of Artificial Intelligence 2011; 24 (1): 164-181. doi: 10.1016/j.engappai.2010.09.007
[5] Maharaj EA, D’Urso P. A coherence-based approach for the pattern recognition of time series. Physica A: Statistical Mechanics and its Applications 2010; 389 (17): 3516-3537. doi: 10.1016/J.PHYSA.2010.03.051
[6] Domańska D, Wojtylak M. Application of Fuzzy Time Series Models for Forecasting Pollution Concentrations. Expert Syst. Appl. 2012; 39 (9): 7673–7679. doi: 10.1016/j.eswa.2012.01.023
[7] Yang G, Yang H, Dai L. Time-series prediction modelling based on an efficient self-organization learning neural network. Application of fuzzy time series models for forecasting pollution concentrations 2015; 28 (8): 248-253. doi: 10.1016/J.IFACOL.2015.08.189
[8] Canizo M, Triguero I, Conde A, Onieva E. Multi-head CNN–RNN for multi-time series anomaly detection: An industrial case study. Neurocomputing 2019; 363: 246-260. doi: 10.1016/j.neucom.2019.07.034
[9] Hashemian HM, Bean WC. State-of-the-Art Predictive Maintenance Techniques. IEEE Transactions on Instrumentation and Measurement 2011; 60 (10): 3480-3492. doi: 10.1109/TIM.2009.2036347
[10] Theissler A. Detecting known and unknown faults in automotive systems using ensemble-based anomaly detection. Knowledge-Based Systems 2017; 123: 163-173. doi: 10.1016/j.knosys.2017.02.023
[11] Scime L, Beuth J. Anomaly detection and classification in a laser powder bed additive manufacturing process using a trained computer vision algorithm. Additive Manufacturing 2018; 19: 114-126. doi: 10.1016/j.addma.2017.11.009
[12] Liu C, Ghosal S, Jiang Z, Sarkar S. An Unsupervised Spatiotemporal Graphical Modeling Approach to Anomaly Detection in Distributed CPS. In: ACM/IEEE 7th International Conference on Cyber-Physical Systems (ICCPS) 2016; PP. 1-10. doi: 10.1109/ICCPS.2016.7479069
[13] Angelo C, Luvisotto M, Michieletto G. Distributed Clustering Strategies in Industrial Wireless Sensor Networks. IEEE Transactions on Industrial Informatics 2017; 13 (1): 228-237. doi: 10.1109/TII.2016.2628409
[14] Goceri E. Skin Disease Diagnosis from Photographs Using Deep Learning. In: Lecture Notes in Computational Vision and Biomechanics 2019; 34: PP. 239-246. doi: 10.1007/978-3-030-32040-9-25
[15] Yaqoob I, Ahmed E, Hashem IAT, Abdalla ahmed AI, Gani AB et al. Internet of Things Architecture: Recent Advances, Taxonomy, Requirements, and Open Challenges. IEEE Wireless Communications 2017; 24 (3): 10-16. doi: 10.1109/MWC.2017.1600421
[16] Du S, Li T, Yang Y, Horng S. Multivariate time series forecasting via attention-based encoder–decoder framework. Neurocomputing 2020; 388: 269-279. doi: 10.1016/j.neucom.2019.12.118
[17] Triguero I, García-Gil D, Maillo J, Luengo J, García S et al. Transforming big data into smart data: An insight on the use of the k-nearest neighbors algorithm to obtain quality data. In: Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 2019; 9 (2). doi: 10.1002/WIDM.1289
[18] Fernández A, García S, Galar M, Prati R, Krawczyk B et al. Learning from imbalanced data sets, Springer, Cham 2020, doi: 10.1007/978-3-319-98074-4
[19] Bayati A, Nguyen KK, Cheriet M. Multiple-Step-Ahead Traffic Prediction in High-Speed Networks. IEEE Communications Letters 2018; 22 (12): 2447-2450. doi: 10.1109/LCOMM.2018.2875747
[20] Li Z, Fang H, Huang M, Wei Y, Zhang L. Data-driven bearing fault identification using improved hidden Markov model and self-organizing map. Computers & Industrial Engineering 2018; 116: 37-46. doi: 10.1016/j.cie.2017.12.002
[21] Bigdeli E, Raahemi B, Mohammadi M, Matwin S. A fast noise resilient anomaly detection using GMM-based collective labelling. In: 2015 Science and Information Conference (SAI); 2015. pp. 337-344. doi: 10.1109/SAI.2015.7237166
[22] Xiang P, Zhou H, Li H, Song S, Tan W et al. Hyperspectral anomaly detection by local joint subspace process and support vector machine. International Journal of Remote Sensing 2020; 41 (10): 3798-3819. doi: 10.1080/01431161.2019.1708504
[23] Zhou P, Li Z, Snowling S, Baetz BW, Na D et al. A random forest model for inflow prediction at wastewater treatment plants. Stochastic Environmental Research and Risk Assessment 2019; 33 (10): 1781-1792. doi: 10.1007/S00477-019-01732-9
[24] Ieracitano C, Adeel A, Morabito FC, Hussain A. A novel statistical analysis and autoencoder driven intelligent intrusion detection approach. Neurocomputing 2020; 387: 51-62. doi: 10.1016/j.neucom.2019.11.016
[25] Hu X, Dai J, Huang Y, Yang H, Zhang L et al. A weakly supervised framework for abnormal behavior detection and localization in crowded scenes. Neurocomputing 2020; 383: 270-281. doi: 10.1016/j.neucom.2019.11.087
[26] Singh K, Rajora S, Vishwakarma DK, Tripathi G, Kumar S et al. Crowd anomaly detection using Aggregation of Ensembles of fine-tuned ConvNets. Neurocomputing 2020; 371: 188-198. doi: 10.1016/j.neucom.2019.08.059
[27] Fan J, Zhang Q, Zhu J, Zhang M, Yang Z et al. Robust deep auto-encoding Gaussian process regression for unsupervised anomaly detection. Neurocomputing 2020; 376: 180-190. doi: 10.1016/j.neucom.2019.09.078
[28] Che Z, Purushotham S, Cho K, Sontag D, Liu Y. Recurrent Neural Networks for Multivariate Time Series with Missing Values. Scientific Reports 2018; 8 (1): 1-12. doi: 10.1038/s41598-018-24271-9
[29] Malhotra P, Vig L, Shroff G, Agarwal P. Long short term memory networks for anomaly detection in time series. In: 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, ESANN 2015, Bruges, Belgium
[30] Nguyen HD, Tran KP, Thomassey S, Hamad M. Forecasting and Anomaly Detection approaches using LSTM and LSTM Autoencoder techniques with the applications in supply chain management. International Journal of Information Management 2021; 57: 102282. doi: 10.1016/j.ijinfomgt.2020.102282
[31] Wang M, Wang Z, Lu J, Lin J, Wang Z. E-LSTM: An Efficient Hardware Architecture for Long Short-Term Memory. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 2019; 9 (2): 280-291. doi: 10.1109/JETCAS.2019.2911739
[32] Jiang K, Wang W, Wang A, Wu H. Network Intrusion Detection Combined Hybrid Sampling with Deep Hierarchical Network. IEEE Access 2020; 8: 32464-32476. doi: 10.1109/ACCESS.2020.2973730
[33] Yuan J, Tian Y. An Intelligent Fault Diagnosis Method Using GRU Neural Network towards Sequential Data in Dynamic Processes. Processes 2019; 7: 152-155. doi: 10.3390/PR7030152
[34] Afrasiabi S, Afrasiabi M, Parang B, Mohammadi M. Designing a composite deep learning based differential protection scheme of power transformers. Applied Soft Computing Journal 2020; 87: 105975. doi: 10.1016/J.ASOC.2019.105975
[35] Kimotho JK, Sextro W. An approach for feature extraction and selection from non-trending data for machinery prognosis. In: PHM Society European Conference 2014; 2: 1. doi: 10.36001/PHME.2014.V2I1.1462
[36] Kingma DP, Ba J. Adam: A Method for Stochastic Optimization. In: 3rd International Conference on Learning Representations 2015. doi: 10.48550/arXiv.1412.6980
[37] Lee C, Choi E. Bayes error evaluation of the Gaussian ML classifier. IEEE Transactions on Geoscience and Remote Sensing 2000; 38 (3): 1471-1475. doi: 10.1109/36.843045
[38] Choi E, Lee C. Feature extraction based on the Bhattacharyya distance. Pattern Recognition 2003; 36 (8): 1703- 1709. doi: 10.1016/S0031-3203(03)00035-9
[39] Schuster M, Paliwal KK. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing 1997; 45 (11): 2673-2681. doi: 10.1109/78.650093
[40] Salehinejad S, Sankar S, Barfett J, Colak E, Valaee S. Recent Advances in Recurrent Neural Networks. Neural and Evolutionary Computing 2021. doi: 10.48550/arXiv.1801.01078
[41] Graves A, Schmidhuber J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks 2005; 18 (5): 602-610. doi: 10.1016/j.neunet.2005.06.042
[42] Jain LC, Medsker LR. Recurrent Neural Networks: Design and Applications. USA: CRC 1999
[43] Elsayed N, Maida A, Bayoumi M. Effects of different activation functions for unsupervised convolutional LSTM spatiotemporal learning. Advances in Science, Technology and Engineering Systems 2019; 4 (2): 260-269. doi: 10.25046/AJ040234
[44] Graves A, Jaitly N, Mohamed AR. Hybrid speech recognition with Deep Bidirectional LSTM. In: IEEE Workshop on Automatic Speech Recognition and Understanding 2013; 273-278. doi: 10.1109/ASRU.2013.6707742
[45] Cho K, Merrienboer BV, Gulcehre C, Bahdanau D, Bougares F et al. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. Computation and Language 2014. doi: 10.48550/arXiv.1406.1078
[46] Chung J, Gulcehre C, Cho K, Bengio Y. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. Neural and Evolutionary Computing 2014; 18 (5): 602-610. doi: 10.48550/arXiv.1412.3555
[47] Pahl M, Aubet F. All Eyes on You: Distributed Multi-Dimensional IoT Microservice Anomaly Detection. 14th International Conference on Network and Service Management (CNSM), 2018, pp. 72-80.
[48] Diro AA, Chilamkurti N. Distributed attack detection scheme using deep learning approach for Internet of Things. Future Generation Computer Systems 2018; 82: 761-768. doi: 10.1016/J.FUTURE.2017.08.043
[49] Azumah SW, Elsayed N, Adewopo V, Zaghloul S, Li C. A deep lstm based approach for intrusion detection iot devices network in smart home. In: IEEE 7th World Forum on Internet of Things (WF-IoT) 2021; 836-841. doi: 10.1109/WF-IoT51360.2021.9596033
[50] Alrashdi I, Alqazzaz A, Aloufi E, Alharthi R, Zohdy M et al. AD-IoT: Anomaly detection of IoT cyberattacks in smart city using machine learning. In: IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC) 2019;pp. 0305-0310. doi: 10.1109/CCWC.2019.8666450.
[51] McDermott CD, Majdani F, Petrovski AV. Botnet Detection in the Internet of Things using Deep Learning Approaches. In: International Joint Conference on Neural Networks (IJCNN) 2018;pp. 1-8. doi: 10.1109/IJCNN.2018.8489489.