Improved online sequential extreme learning machine: OS-CELM

: Online learning methods (OLM) have been gaining traction as a solution to classification problems because of rapid renewal and fast growth in volume of available data. ELM-based sequential learning (OS-ELM) is one of the most frequently used online learning methodologies partly due to fast training algorithm but suffers from inefficient use of its hidden layers due to the random assignment of the parameters of those layers. In this study, we propose an improved online learning model called online sequential constrained extreme learning machine (OS-CELM), which replaces the random assignment of those parameters with better generalization performance using the CELM method based on the distance between classes. We compare the performance and training times of OS-ELM, ELM, and the proposed models for four different data sets. The results indicate that the proposed model has better generalization and accuracy performance.

PDF

___

[1] Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review 1958; 65: 386-408.
[2] Huang GB, Zhu QY, Siew C. Extreme learning machine: a new learning scheme of feedforward neural networks. In: IEEE International Joint Conference on Neural Network; Budapest; 2004. pp. 985-990. doi: 10.1109/IJCNN.2004.1380068.
[3] Apaydin E. Introduction to Machine Learning. Cambridge: MIT Press, 2010.
[4] Liang N, Huang GB, Saratchandran P, Sundararajan N. A fast and accurate online sequential learning algorithm for feed forward networks. IEEE transactions on neural networks a publication of the IEEE Neural Networks Council 2006; 17: 1411-1123.
[5] Zhu W, Miao J, Qing L. Constrained extreme learning machine: a novel highly discriminative random feedforward neural network. In: International Joint Conference on Neural Networks (IJCNN); Beijing; 2014. pp. 800-807. doi: 10.1109/IJCNN.2014.6889761
[6] Chy TS, Rahaman MA. A comparative analysis by knn, svm & elm classification to detect sickle cell anemia. In: International Conference on Robotics, Electrical and Signal Processing Techniques (ICREST); Bangladesh; 2019. pp. 455-459. doi: 10.1109/ICREST.2019.8644410
[7] Huang GB, Ding X, Zhou H. Optimization method based extreme learning machine for classification. Neurocomputing 2010; 74: 155-163.
[8] Qiu S, Gao L, Wang J. Classification and regression of elm, lvq and svm for e-nose data of strawberry juice. Journal of Food Engineering 2015; 144: 77-85.
[9] Deng WY, Zheng Q, Wang Z. Cross-person activity recognition using reduced kernel extreme learning machine. Neural networks: the official journal of the International Neural Network Society 2014; 53: 1-7.
[10] Xu S, Wang J. A fast-incremental extreme learning machine algorithm for data streams classification. Expert Systems with Applications 2016; 65: doi: 10.1016/j.eswa.2016.08.052
[11] Wang Z, Xin J, Yang H, Tian S, Yu G et al. Distributed and weighted extreme learning machine for imbalanced big data learning. Tsinghua Science and Technology 2017; 22: 160-173.
[12] Garcia L, Saez JA, Luengo J, Lorena A, Carvalho A et al. Using the one-vs-one decomposition to improve the performance of class noise filters via an aggregation strategy in multi-class classification problems. KnowledgeBased Systems 2015; 90: 153-164. doi: 10.1016/j.knosys.2015.09.023
[13] Rohra JG, Perumal B, Narayanan SJ, Thakur P, Bhatt RB. User localization in an indoor environment using fuzzy hybrid of particle swarm optimization & gravitational search algorithm with neural networks. In: Proceedings of Sixth International Conference on Soft Computing for Problem Solving. Advances in Intelligent Systems and Computing; Singapore; 2015. doi:/10.1007/978-981-10-3322-3_27
[14] Bhatt RB, Dhall A, Sharma G, Chaudhury S. Efficient skin region segmentation using low complexity fuzzy decision tree model. In: Proceedings of INDICON An IEEE India Council Conference; India 2009. doi: 10.1109/INDCON.2009.5409447
[15] Detrano R, Janosi A, Steinbrunn W, Pfisterer M, Schmid J et al. International application of a new probability algorithm for the diagnosis of coronary artery disease. The American Journal of Cardiology 1989; 64 (5): 304-310.
[16] Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. In: Proceedings of the 1998 IEEE International Frequency Control Symposium; Pasadena, CA, USA; 1998. pp. 2278-2324.
[17] Van der Maaten L, Hinton G. Visualizing high-dimensional data using t-SNE. Journal of Machine Learning Research 2008; 9: 2579-2605.
[18] Grother P. NIST special database 19 hand printed forms and characters database. 1995.
[19] Sabanci K, Yigit E, Ustun D, Toktas A, Aslan MF. Wifi based indoor localization: Application and comparison of machine learning algorithms. In: XXIIIrd International Seminar Workshop on Direct and Inverse Problems of Electromagnetic and Acoustic Wave Theory (DIPED) ;Tbilisi,Georgia; 2018. pp. 246-251.
[20] Çatak FO. Classification with boosting of extreme learning machine over arbitrarily partitioned data. Soft Computing 2017; 21: 2269-2281.
[21] Amin M, Chiam Y, Varathan K. Identification of significant features and data mining techniques in predicting heart disease. Telematics and Informatics 2018. doi: 10.1016/j.tele.2018.11.007
[22] Khemphila A, Boonjing V. Heart disease classification using neural network and feature selection. In: International Conference on Systems Engineering; Las Vegas, USA; 2011. pp. 406-409.
[23] Verma L, Srivastava S, Negi P. A hybrid data mining model to predict coronary artery disease cases using noninvasive clinical data. Journal of Medical Systems 2016; 40: 178.15.
[24] Song G, Dai Q, Han X, Guo L. Two novel elm-based stacking deep models focused on image recognition. Applied Intelligence 2020; 50: 1345-1366.
[25] Chen J, Dong C, He G, Zhang X. A method for indoor Wi-Fi location based on improved back propagation neural network. Turkish Journal of Electrical Engineering & Computer Sciences 2019; 27: 2511-2525.
[26] Idoko J, Arslan M, Abiyev R. Fuzzy neural system application to differential diagnosis of erythemato-squamous diseases. Cyprus Journal of Medical Sciences 2018; 90-97.
[27] Ma CH, Shih H. Human skin segmentation using fully convolutional neural networks. In: IEEE 7th Global Conference on Consumer Electronics; Nara, Japan; 2018. pp. 168-170.
[28] Zhang J, Li Y, Xiao W. Adaptive online sequential extreme learning machine for dynamic modeling. Soft Computing 2021; 25: 2177-2189. doi: 10.1007/s00500-020-05289-6
[29] Zhang P, Huang Y, Li M, Wang Y, Wen P et al. Fault diagnosis method of analog circuit based on GA-OS-ELM. In: 2020 International Conference on Sensing, Diagnostics, Prognostics, and Control (SDPC); Beijing, China; 2020. pp. 273-278. doi: 10.1109/SDPC49476.2020.9353152