Human Activity Recognition with Smartwatch Data by using Mahalanobis Distance-Based Outlier Detection and Ensemble Learning Methods

Recognition of human activities is part of smart healthcare applications. In this context, the detection of human actions with high accuracy has been a field that has been working for many years. With the increase in the usage of smart devices, smartphones and smartwatches have become the constant equipment of these studies thanks to their internal sensors. Sometimes abnormal data are included in data sets due to the way the data were collected and for reasons arising from the sensors. For this reason, it becomes important to detect outlier data. In this study, step counter and heart rate sensors were used in addition to an accelerometer and gyroscope in order to detect human activities. Afterward, the outliers were detected and cleared with a Mahalanobis distance-based approach. With the aim of achieving a better classification performance, machine learning methods were used by strengthening them with ensemble learning methods. The obtained results showed that step counter, heart rate sensors, and ensemble learning methods positively affect the success of the classification. In addition, it was found that the Mahalanobis distance-based outlier detection method increased the classification accuracy significantly.

Keywords:

activity recognition, smartwatch, mahalanobis distance ensemble learning, sensor data,

PDF

___

S. Tian, W. Yang, J.M. Le Grange, P. Wang, W. Huang, and Z Ye, “Smart healthcare: making medical care more intelligent”, Global Health Journal, vol. 3, no. 3, pp. 62-65, 2019. https://doi.org/10.1016/j.glohj.2019.07.001
B. Mortazavi, S. Nyamathi, S.I. Lee, T. Wilkerson, H. Ghasemzadeh, and M. Sarrafzadeh, “Near-realistic mobile exergames with wireless wearable sensors”, IEEE Journal of Biomedical and Health Informatics, vol. 18, no. 2, pp. 449-456, 2013. https://doi.org/10.1109/JBHI.2013.2293674
M.C. Limas, J.B.O. Meré, F.J.M. de Pisón Ascacibar, and E.P.V. González, “Outlier detection and data cleaning in multivariate non-normal samples: the PAELLA algorithm”, Data Mining and Knowledge Discovery, vol. 9, no. 2, pp. 171-187, 2004. https://doi.org/10.1023/B:DAMI.0000031630.50685.7c
M. Gjoreski, H. Gjoreski, M. Luštrek, and M. Gams, “How accurately can your wrist device recognize daily activities and detect falls?” Sensors, vol. 16, no. 6, pp. 800, 2016. https://doi.org/10.3390/s16060800
E.A. Sağbaş, and S. Ballı, “Transportation mode detection by using smartphone sensors and machine learning”, Pamukkale University Journal of Engineering Sciences, vol. 22, no. 5, pp. 376-383, 2016. https://doi.org/10.5505/pajes.2015.63308
M.C. Kwon, and S. Choi, S. “Recognition of daily human activity using an artificial neural network and smartwatch”, Wireless Communications and Mobile Computing, 2018. https://doi.org/10.1155/2018/2618045
R.A. Voicu, C. Dobre, L. Bajenaru, and R.I. Ciobanu, “Human physical activity recognition using smartphone sensors”, Sensors, vol. 19, no. 3, pp. 458, 2019. https://doi.org/10.3390/s19030458
S. Ballı, E.A. Sağbaş, and M. Peker, “A mobile solution based on soft computing for fall detection”. In Mobile Solutions and Their Usefulness in Everyday Life, Ed. Sara Paiva, 2019, ch. 14, pp. 275-294.
M. Peker, S. Ballı, and E.A. Sağbaş, “Predicting human actions using a hybrid of ReliefF feature selection and kernel-based extreme learning machine”, In Cognitive Analytics: Concepts, Methodologies, Tools, and Applications, Ed.Mehdi Khosrow-Pour , 2020, ch. 17, pp. 307-325.
N. Ahmed, J.I. Rafiq, and M.R. Islam, “Enhanced human activity recognition based on smartphone sensor data using hybrid feature selection model”, Sensors, vol. 20, no. 1, pp. 317, 2020. https://doi.org/10.3390/s20010317
S.W. Yahaya, A. Lotfi, and M. Mahmud, “Detecting anomaly and its sources in activities of daily living”, SN Computer Science, vol. 2, no. 1, pp. 1-18, 2021. https://doi.org/10.1007/s42979-020-00418-2
Y. Chen, and C. Shen, “Performance analysis of smartphone-sensor behavior for human activity recognition” IEEE Access, vol. 5, pp. 3095-3110, 2017. https://doi.org/10.1109/ACCESS.2017.2676168
R. Li, H. Li, and W. Shi, “Human activity recognition based on LPA”, Multimedia Tools and Applications, vol. 79, no. 41, pp. 31069-31086, 2020. https://doi.org/10.1007/s11042-020-09150-8
M.O. Gani, T. Fayezeen, R.J. Povinelli, R.O. Smith, M. Arif, A.J. Kattan, and S.I. Ahamed, “A light weight smartphone based human activity recognition system with high accuracy”, Journal of Network and Computer Applications, vol. 141, pp. 59-72, 2019. https://doi.org/10.1016/j.jnca.2019.05.001
A. Elsts, N. Twomey, R. McConville, and I. Craddock, “Energy-efficient activity recognition framework using wearable accelerometers”, Journal of Network and Computer Applications, vol. 168, pp. 102770, 2020. https://doi.org/10.1016/j.jnca.2020.102770
H. Gjoreski, J. Bizjak, M. Gjoreski, and M. Gams, “Comparing deep and classical machine learning methods for human activity recognition using wrist accelerometer”, In Proceedings of the IJCAI 2016 Workshop on Deep Learning for Artificial Intelligence, vol. 10, pp. 970, 2016.
A. Ignatov, “Real-time human activity recognition from accelerometer data using Convolutional Neural Networks”, Applied Soft Computing, vol. 62, pp. 915-922, 2018. https://doi.org/10.1016/j.asoc.2017.09.027
B. Zhou, J. Yang, and Q. Li, “Smartphone-based activity recognition for indoor localization using a convolutional neural network”, Sensors, vol. 19, no. 3, pp. 621, 2019. https://doi.org/10.3390/s19030621
S. Wan, L. Qi, X. Xu, C. Tong, and Z. Gu, “Deep learning models for real-time human activity recognition with smartphones”, Mobile Networks and Applications, vol. 25, no. 2, pp. 743-755, 2020. https://doi.org/10.1007/s11036-019-01445-x
X. Zhou, W. Liang, I. Kevin, K. Wang, H. Wang, L.T. Yang, and Q. Jin, “Deep-learning-enhanced human activity recognition for Internet of healthcare things”, IEEE Internet of Things Journal, vol. 7, no. 7 , pp. 6429-6438, 2020. https://doi.org/10.1109/JIOT.2020.2985082
M. Altuve, P. Lizarazo, and J. Villamizar, “Human activity recognition using improved complete ensemble EMD with adaptive noise and long short-term memory neural networks”, Biocybernetics and Biomedical Engineering, vol. 40, no. 3, pp. 901-909, 2020. https://doi.org/10.1016/j.bbe.2020.04.007
D. Mukherjee, R. Mondal, P.K. Singh, R. Sarkar, and D. Bhattacharjee, “EnsemConvNet: a deep learning approach for human activity recognition using smartphone sensors for healthcare applications”, Multimedia Tools and Applications, vol. 79, no. 41, pp. 31663-31690, 2020. https://doi.org/10.1007/s11042-020-09537-7
C. Catal, S. Tufekci, E. Pirmit, and G. Kocabag, “On the use of ensemble of classifiers for accelerometer-based activity recognition”, Applied Soft Computing, vol. 37, pp. 1018-1022, 2015. https://doi.org/10.1016/j.asoc.2015.01.025
I. Elamvazuthi, L.I. Izhar, and G. Capi, “Classification of human daily activities using ensemble methods based on smartphone inertial sensors”, Sensors, vol. 18, no. 12, pp. 4132, 2018. https://doi.org/10.3390/s18124132
S. Balli, E.A. Sağbaş, and M. Peker, “Human activity recognition from smart watch sensor data using a hybrid of principal component analysis and random forest algorithm”, Measurement and Control, vol. 52, no. 1-2, pp. 37-45, 2019. https://doi.org/10.1177/0020294018813692
O. Herrera-Alcántara, A.Y. Barrera-Animas, M. González-Mendoza, and F. Castro-Espinoza, “Monitoring student activities with smartwatches: On the academic performance enhancement”, Sensors, vol. 19, no. 7, pp. 1605, 2019. https://doi.org/10.3390/s19071605
N. Irvine, C. Nugent, S. Zhang, H. Wang, and W.W. Ng, “Neural network ensembles for sensor-based human activity recognition within smart environments”, Sensors, vol. 20, no. 1, pp. 216, 2020. https://doi.org/10.3390/s20010216
S. Brajesh, and I. Ray, “Ensemble approach for sensor-based human activity recognition”, In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers, pp. 296-300, 2020. https://doi.org/10.1145/3410530.3414352
R. Sekiguchi, K. Abe, T. Yokoyama, M. Kumano, and M. Kawakatsu, “Ensemble learning for human activity recognition”, In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers, pp. 335-339, 2020. https://doi.org/10.1145/3410530.3414346
A. Subasi, D.H. Dammas, R.D. Alghamdi, R.A. Makawi, E.A., Albiety, T. Brahimi, and A. Sarirete, “Sensor based human activity recognition using adaboost ensemble classifier”, procedia computer science, vol. 140, pp. 104-111, 2018. https://doi.org/10.1016/j.procs.2018.10.298
N. Dwivedi, D.K. Singh, and D.S. Kushwaha, “Orientation invariant skeleton feature (oisf): a new feature for human activity recognition”, Multimedia Tools and Applications, vol. 79, no. 29, pp. 21037-21072, 2020. https://doi.org/10.1007/s11042-020-08902-w
S. Balli, and E.A. Sağbaş, “The usage of statistical learning methods on wearable devices and a case study: activity recognition on smartwatches” Advances in statistical methodologies and their application to real problems, Ed. Tsukasa Hakimoto, 2017, ch. 13, pp. 259-277.
G.H. John, P. Langley, “Estimating continuous distributions in Bayesian classifiers”, In Proceedings of the Eleventh conference on Uncertainty in artificial intelligence, pp. 338-345, August 1995.
B.K. Alsberg, R. Goodacre, J.J. Rowland, and D.B. Kell, “Classification of pyrolysis mass spectra by fuzzy multivariate rule induction-comparison with regression, K-nearest neighbour, neural and decision-tree methods”, Analytica Chimica Acta, vol. 348, no. 1-3, pp. 389-407, 1997. https://doi.org/10.1016/S0003-2670(97)00064-0
E.A. Sağbaş, and S. Ballı, “Estimation of Human Activities by Using Wrist Movements”, 3. International Management Information Systems Conference, pp. 52-58, October 2016.
S. Ballı, E.A. Sağbaş, “Diagnosis of transportation modes on mobile phone using logistic regression classification”, IET Software, vol. 12, no. 2, pp.142-151, 2018. https://doi.org/10.1049/iet-sen.2017.0035
L. Breiman, “Random forests”, Machine learning, vol. 45, no. 1, pp. 5-32, 2001. https://doi.org/10.1023/a:1010933404324
S. Ballı, and E.A. Sağbaş, “Classification of Human Motions with Smartwatch Sensors”, Süleyman Demirel University Journal of Natural and Applied Sciences, vol. 21, no. 3, pp. 980-990, 2017. https://doi.org/10.19113/sdufbed.32689
L. Rokach, “Ensemble-based classifiers”, Artificial intelligence review, vol. 33, no. 1, pp. 1-39, 2010. https://doi.org/10.1007/s10462-009-9124-7
L. Breiman, “Bagging predictors”, Machine learning, vol. 24, no. 2, pp. 123-140, 1996. https://doi.org/10.1007/BF00058655
L.I. Kuncheva, “Combining pattern classifiers: methods and algorithms”, John Wiley & Sons, 2014.
A. Onan, S. Korukoğlu, and H. Bulut, “Ensemble of keyword extraction methods and classifiers in text classification”, Expert Systems with Applications, vol. 57, pp. 232-247, 2016. https://doi.org/10.1016/j.eswa.2016.03.045
E.A. Sağbaş, and S. Balli, “Activity recognition by voting method via motion sensors”, In 2017 International Conference on Computer Science and Engineering (UBMK), 2017. https://doi.org/10.1109/UBMK.2017.8093557
E.A. Sağbaş, and S. Ballı, “Usage of the smartphone sensors and accessing raw sensor data”, Academic Computing Conferences, pp. 180-186, 2015.
P.C. Mahalanobis, “On tests and measures of groups divergence”, IJ Asiatic Soc. Bengal, vol. 26, pp. 541, 1930.
C. Leys, O. Klein, Y. Dominicy, and C. Ley, “Detecting multivariate outliers: Use a robust variant of the Mahalanobis distance”, Journal of Experimental Social Psychology, vol. 74, pp. 150-156, 2018. https://doi.org/10.1016/j.jesp.2017.09.011
K. Polat, and S. Güneş, “Classification of epileptiform EEG using a hybrid system based on decision tree classifier and fast Fourier transform”, Applied Mathematics and Computation, vol. 187, no. 2, pp. 1017-1026, 2007. https://doi.org/10.1016/j.amc.2006.09.022
I.H. Witten, and E. Frank, “Data mining: practical machine learning tools and techniques with Java implementations”, Acm Sigmod Record, vol. 31, no.1, pp. 76-77, 2002.