Geometrical Modification of Learning Vector Quantization Method for Solving Classification Problems

In this paper, a geometrical scheme is presented to show how to overcome an encountered problem arising from the use of generalized delta learning rule within competitive learning model. It is introduced a theoretical methodology for describing the quantization of data via rotating prototype vectors on hyper-spheres.The proposed learning algorithm is tested and verified on different multidimensional datasets including a binary class dataset and two multiclass datasets from the UCI repository, and a multiclass dataset constructed by us. The proposed method is compared with some baseline learning vector quantization variants in literature for all domains. Large number of experiments verify the performance of our proposed algorithm with acceptable accuracy and macro f1 scores.

Anahtar Kelimeler:

Machine learning, Learning vector quantization; Geometrical learning approach

PDF

___

[1] Watanabe, S. 2009. Algebraic Geometry and Statistical Learning Theory. Cambridge Monographs on Applied and ComputationalMathematics, Cambridge University Press.
[2] John, G. 1993. Geometry-Based Learning Algorithms.
[3] Kim, J.H., and Park, S. 1995. The geometrical learning of binary neural networks. IEEE Transactions on Neural Networks, 6(1), 237-247.
[4] Cabrelli, C., Molter, U. and Shonkwiler, R. 2000. A constructive algorithm to solve “convex recursive deletion" (CoRD) classification problems via twolayer perceptron networks. EEE Transactions on Neural Networks Learning Systems, 11(3), 811-816.
[5] Wang, D. and Chaudhari, N.S. 2004. An approach for construction of Boolean neural networks based on geometrical expansion. Neurocomputing, 57, 455-461.
[6] Shoujue, W. and Jiangliang, L. 2005. Geometrical learning, descriptive geometry, and biomimetic pattern recognition. Neurocomputing, 67, 9-28.
[7] Bayro-Corrochano, E. and Anana-Daniel, N. 2005. MIMO SVMs for classification and regression using the geometric algebra framework. Proceedings of the International Joint Conference on Neural Networks, 895–900.
[8] Zhang, D., Chan, X. and Lee, WS. 2005. Text classificitaion with kernels on the multinomial manifold. Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, 266-273.
[9] Delogu, R., Fanni, A. and Montisci, A. 2008. Geometrical synthesis of MLP neural networks. Neurocomputing, 71 (4-6), 919-930.
[10] Liu, Z., Liu, J.G., Pan, C. and Wang, G. 2009. A novel geometric approach to binary classification based on scaled convex hulls. IEEE Transactions on Neural Networks, 20(7), 1215-1220.
[11] Nova, D. and Est´evez, P.A. 2014. A review of learning vector quantization classifiers. Neural Computing and Applications, 25(3-4), 511-524.
[12] Kaden, M., Lange M., Nebel, D., Riedel, M. , Geweniger, T. and Villmann, T. 2014. Aspects in Classification Learning - Review of Recent Developments in Learning Vector Quantization. Foundations of Computing and Decision Sciences, 39(2), 79-105.
[13] Kohonen T. 1990. The self-organizing map. Proceedings of the IEEE, 78(9), 1464-1480.
[14] Kohonen, T.1990. Improved versions of learning vector quantization. Proceedings of the International Joint Conference on Neural Networks, 1, 545-550.
[15] Kohonen T., Hynninen, J., Kangas, J., Laaksonen, J. and Torkkola, K. 1996. LVQ_PAK: The learning vector quantization program package. Helsinki University of Technology, Laboratory of Computer and Information Science, Finland.
[16] Sato, A.S. and Yamada, K. 1995. Generalized learning vector quantization. In G. Tesauro, D. Touretzky and T. Leen (Eds.): Advances in Neural Information Processing Systems. MIT Press, 423-429.
[17] Hammer, B. and Villmann, T.2002. Generalized relevance learning vector quantization. Neural Networks - New developments in self-organizing maps, 15(8-9), 1059-1068.
[18] Seo, S. and Obermayer, K. 2003. Soft learning vector quantization. Neural Computation, 15(7), 1589- 1604.
[19] Biehl, M., Ghosh,A., Hammer,B. and Bengio,Y. 2006. Dynamics and Generalization Ability of LVQ Algorithms. The Journal of Machine Learning Research, 8, 323–360.
[20] Schneider, P., Hammer,B. and Biehl,M. 2009. Adaptive relevance matrices in learning vector quantization. Neural Computation, 21, 3532-3561.
[21] Kaestner, M., Hammer, B., Biehl, M. and Villmann, T. 2012. Functional relevance learning in generalized learning vector quantization. Neurocomputing, 90, 79-105.
[22] Hammer, B., Hofmann, D., Schleif, F.M. and Zhu, X. 2014. Learning vector quantization for (dis-)similarities. Neurocomputing, 131, 43-51.
[23] Bohnsack, A., Domaschke, K., Kaden, M., Lange, M. and Villmann, T. Learning matrix quantization and relevance learning based on Schatten-p-norms. Neurocomputing, 192, 104-114.
[24] Bohnsack, A., Domaschke, K., Kaden, M., Lange, M. and Villmann, T. 2016. Learning matrix quantization and relevance learning based on Schatten-pnorms. Neurocomputing, 192, 104-114.
[25] Buchala, S., Davey, N., Gale, T.M. and Frank, R.J. 2005. Analysis of linear and nonlinear dimensionality reduction methods for gender classification of face images. International Journal of Systems Science, 36(14), 931-942.
[26] Maeda, S. and Ishii, S. 2009. Learning a multidimensional companding function for lossy source coding. Neural Networks, 22(7), 998-1010.
[27] Denil, M., Shakibi, B., Dinh, L., Ranzato, M. and Freitas, N. 2013. Predicting Parameters in Deep Learning. In CJC. Burges, L. Bottou , M. Welling, Z. Ghahramani and KQ. Weinberger (Eds.): Advances in Neural Information Processing Systems, 26, 2148–2156.
[28] Bache, K. and Lichman, M. 2014. UCI Repository of Machine Learning Databases. Irvine CA: University of California. School of Information and Computer Science. Available on:
[29] Fisher, R.A. 1936. The use of multiple measurements in taxonomic problems. Annals of Eugenics, 2, 179-188.
[30] Wolberg,W.H., Street,W.N., Heisey, D.M. andMangasarian, O.L. 1995. Computer-derived nuclear features distinguish malignant from benign breast cytology. Human Pathology, 26, 792-796.
[31] Alpaydin, E. and Kaynak, C. 1998. Cascaded Classifiers. Kybernetika.
[32] Kaynak, C. 1995. Methods of Combining Multiple Classifiers and Their Applications to Handwritten Digit Recognition. MSc Thesis. Institute of Graduate Studies in Science and Engineering. Bogazici University.