Gazi Erkan BOSTANCI, Mehmet Serdar GÜZEL, Şifa ÖZSARI, Ayhan AYDIN

Adaptation of metaheuristic algorithms to improve training performance of an ESZSL model

Zero-shot learning (ZSL) is a recent promising learning approach that is similar to human vision systems. ZSL essentially allows machines to categorize objects without requiring labeled training data. In principle, ZSL proposes a novel recognition model by specifying merely the attributes of the category. Recently, several sophisticated approaches have been introduced to address the challenges regarding this problem. Embarrassingly simple approach to zeroshot learning (ESZSL) is one of the critical of those approaches that basically proposes a simple but efficient linear code solution. However, the performance of the ESZSL model mainly depends on parameter selection. Metaheuristic algorithms are considered as one the most sophisticated computational intelligence paradigms that allows to approximate optimization problems with high success. This paper addresses this problem by adapting leading metaheuristic algorithms to automatically train the parameters of a linear ESZSL model. The model is statistically validated by performing a series of experiments with benchmark datasets.

PDF

___

[1] Romera-Paredes B,Torr P. An embarrassingly simple approach to zero-shot learning. In: International Conference on Machine Learning; Lille, France; 2015. pp. 2152-2161.
[2] Li J, Jing M, Lu K, Ding Z, Zhu L et al. Leveraging the invariant side of generative zero-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2019. pp. 7402-7411.
[3] Ding Z, Shao M, Fu Y. Low-rank embedded ensemble semantic dictionary for zero-shot learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017. pp. 2050-2058
[4] Zhang L, Xiang T, Gong S. Learning a deep embedding model for zero-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2017. pp. 2021-2030.
[5] Xian Y, Lorenz T, Schiele B, Akata Z. Feature generating networks for zero-shot learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2018. pp. 5542-5551.
[6] Mishra A, Krishna RS, Mittal A, Murthy HA. A generative model for zero shot learning using conditional variational autoencoders. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops; 2018. pp. 2188-2196.
[7] Dietterich TG, Bakiri G. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research 1994; 2: pp. 263-286. doi: 10.1613/jair.105
[8] Bart E, Ullman S. Cross-generalization: Learning novel classes from a single example by feature replacement. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05); 2005. pp. 672-679.
[9] Lampert CH, Nickisch H, Harmeling S. Attribute-based classification for zero-shot visual object categorization. IEEE Transactions on Pattern Analysis and Machine Intelligence 2013; (36)3: pp. 453-465. doi: 10.1109/TPAMI.2013.140
[10] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems; 2012. pp. 1097-1105.
[11] Soysal OA, Guzel MS. An introduction to zero-shot learning: an essential review. In: 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA); Ankara, Turkey; 2020. pp. 1-4.
[12] Kankuekul P, Kawewong A, Tangruamsub S, Hasegawa O. Online incremental attribute-based zero-shot learning. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition; 2012. pp. 3657-3664.
[13] Norouzi M, Mikolov T, Bengio S, Singer Y, Shlens J et al. Zero-shot learning by convex combination of semantic embeddings. arXiv preprint arXiv:1312.5650 2013.
[14] Jayaraman D, Grauman K. Zero-shot recognition with unreliable attributes. In: Advances in Neural Information Processing Systems; 2014. pp. 3464-3472.
[15] Sumbul G, Cinbis RG, Aksoy S. Fine-grained object recognition and zero-shot learning in remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing 2017; 56(2): 770-779. doi: 10.1109/TGRS.2017.2754648.
[16] Lawrence ND, Platt JC. Learning to learn with the informative vector machine. In: Proceedings of The Twenty-first International Conference on Machine Learning; 2004. pp. 65.
[17] Croonenborghs T, Driessens K, Bruynooghe M. Learning relational options for inductive transfer in relational reinforcement learning. In: International Conference on Inductive Logic Programming; 2007. pp. 88-97.
[18] Raykar VC, Krishnapuram B, Bi J, Dundar M, Rao RB. Bayesian multiple instance learning: automatic feature selection and inductive transfer. In: Proceedings of The 25th International Conference on Machine Learning; 2008. pp. 808-815.
[19] Rückert U, Kramer S. Kernel-based inductive transfer. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases; 2008. pp. 220-233.
[20] Akata Z, Perronnin F, Harchaoui Z, Schmid C. Label-embedding for attribute-based classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2013. pp. 819-826.
[21] Lampert CH, Nickisch H, Harmeling S. Learning to detect unseen object classes by between-class attribute transfer. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition; 2009. pp. 951-958.
[22] Patterson G, Hays J. Sun attribute database: Discovering, annotating, and recognizing scene attributes. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition; 2012. pp. 2751-2758.
[23] Farhadi A, Endres I, Hoiem D, Forsyth D. Describing objects by their attributes. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition; 2009. pp. 1778-1785.
[24] Wah C, Branson S, Welinder P, Perona P, Belongie S. The caltech-ucsd birds-200-2011 dataset; 2011.
[25] Xian Y, Lampert CH, Schiele B, Akata Z. Zero-shot learning—A comprehensive evaluation of the good, the bad and the ugly. IEEE Transactions on Pattern Analysis and Machine Intelligence 2018; 41(9): 2251-2265. doi: 10.1109/TPAMI.2018.2857768.
[26] Holland JH et al. Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence: MIT press, 1992.
[27] ÖZSARI Ş, UĞUZ H, ÇAY T. Interview in land consolidation using genetic algorithm. Communications Faculty of Sciences University of Ankara Series A2-A3 2018; 60 (2): 129-146. doi: 10.1501/commua1-2_0000000119.
[28] Kennedy J, Eberhart R. Particle swarm optimization. In: Proceedings of ICNN’95-International Conference on Neural Networks; 1995. pp. 1942-1948.
[29] Shi Y, Eberhart RC. Parameter selection in particle swarm optimization. In: International Conference on Evolutionary Programming; 1998. pp. 591-600.
[30] Sengupta S, Basak S, Peters RA. Particle swarm optimization: a survey of historical and recent developments with hybridization perspectives. Machine Learning and Knowledge Extraction 2019; 1(1): 157-191. doi: 10.3390/make1010010.
[31] Güzel MS, Kara M, Beyazkılıç MS. An adaptive framework for mobile robot navigation. Adaptive Behavior 2017; 25(1): 30-39. doi: 10.1177/1059712316685875.
[32] Neoh SC, Morad N, Marzuki A, Lim CP, Aziz ZA. A multi-resolution GA-PSO layered encoding cascade optimization model. In: Innovations in Swarm Intelligence 248th ed. Berlin, Heidelberg, Germany: Springer, 2009, pp. 121-140.
[33] Xie B, Chen S, Liu F. Biclustering of gene expression data using PSO-GA hybrid. In: 2007 1st International Conference on Bioinformatics and Biomedical Engineering; 2007. pp. 302-305.
[34] Robinson J, Rahmat-Samii Y. Particle swarm optimization in electromagnetics. IEEE transactions on antennas and propagation 2004; 52(2): 397-407. doi: 10.1109/TAP.2004.823969.
[35] Gidemen G, Furat M. PID denetleyici optimizasyonu Üzerine uygulamasi. In: 2017 International Artificial Intelligence and Data Processing Symposium (IDAP); 2017. pp. 1-6.
[36] Simon D. Evolutionary Optimization Algorithms. Hoboken, NJ, USA: John Wiley & Sons, 2013.