ABC-based stacking method for multilabel classification

Multilabel classification is a supervised learning problem wherein each individual instance is associatedwith multiple labels. Ensemble methods are effective in managing multilabel classification problems by creating aset of accurate, diverse classifiers and then combining their outputs to produce classifications. This paper presents anovel stacking-based ensemble algorithm, ABC-based stacking, for multilabel classification. The artificial bee colonyalgorithm, along with a single-layer artificial neural network, is used to find suitable meta-level classifier configurations.The optimization goal of the meta-level classifier is to maximize the average accuracy of classification of all the instancesinvolved. We run an experiment on 10 benchmark datasets of varying domains and compare the proposed approach tofive other ensemble algorithms to demonstrate the feasibility and effectiveness of ABC-based stacking.

PDF

___

[1] Trohidis K, Tsoumakas G, Kalliris G, Vlahavas IP. Multi-label classification of music into emotions. In: 9th International Conference on Music Information Retrieval; Philadelphia, PA, USA; 2008. pp. 325-330.
[2] Sun F, Tang J, Li H, Qi GJ, Huang TS. Multi-label image categorization with sparse factor representation. IEEE Transactions on Image Processing 2014; 23 (3): 1028-1037. doi: 10.1109/TIP.2014.2298978
[3] Maruthupandi J, Devi KV. Multi-label text classification using optimised feature sets. International Journal of Data Mining, Modelling and Management 2017; 9 (3): 237-248. doi: 10.1504/IJDMMM.2017.10007699
[4] Büyükçakir A, Bonab H R, Can F. A novel online stacked ensemble for multi-label stream classification. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management; Torino, Italy; 2018. pp. 1063-1072.
[5] Tsoumakas G, Katakis I. Multi-label classification: an overview. International Journal of Data Warehousing and Mining 2007; 3 (3): 1-13. doi: 10.4018/jdwm.2007070101
[6] Read J, Pfahringer B, Holmes G, Frank E. Classifier chains for multi-label classification. In: Machine Learning and Knowledge Discovery in Databases, European Conference; Bled, Slovenia; 2009. pp. 254-269
[7] Read J, Pfahringer B, Holmes G, Frank E. Classifier chains for multi-label classification. Machine Learning 2011; 85 (3): 333-359. doi: 10.1007/s10994-011-5256-5
[8] Madjarov G, Kocev D, Gjorgjevikj D, Dzeroski S. An extensive experimental comparison of methods for multi-label learning. Pattern Recognition 2012; 45 (9): 3084-3104. doi: 10.1016/j.patcog.2012.03.004
[9] Tsoumakas G, Vlahavas IP. Random k-labelsets: an ensemble method for multilabel classification. In: 18th European Conference on Machine Learning; Warsaw, Poland; 2007. pp. 406-417.
[10] Read J, Pfahringer B, Holmes G. Multi-label classification using ensembles of pruned sets. In: Proceedings of the 8th IEEE International Conference on Data Mining; Pisa, Italy; 2008. pp. 995-1000.
[11] Wu Q, Tan M, Song H, Chen J, Ng MK. ML-FOREST: a multi-label tree ensemble method for multilabel classification. IEEE Transactions on Knowledge and Data Engineering 2016; 28 (10): 2665-2680. doi: 10.1109/TKDE.2016.2581161
[12] Kocev D, Vens C, Struyf J, Dzeroski S. Tree ensembles for predicting structured outputs. Pattern Recognition 2013; 46 (3): 817-833. doi: 10.1016/j.patcog.2012.09.023
[13] Lorente MPS, Ledezma A, Sanchis A. Generating ensembles of heterogeneous classifiers using Stacked Generalization. Wiley Interdisciplinary Review: Data Mining and Knowledge Discovery 2015; 5 (1): 21-34. doi: 10.1002/widm.1143
[14] Zhang C, Yan J, Li C, Bie R. Contour detection via stacking random forest learning. Neurocomputing 2018; 275: 2702-2715. doi: 10.1016/j.neucom.2017.11.046
[15] Nguyen TT, Nguyen MP, Pham XC, Liew AWC. Heterogeneous classifier ensemble with fuzzy rule-based meta learner. Information Sciences 2018; 422: 144-160. doi: 10.1016/j.ins.2017.09.009
[16] Xia Y, Liu C, Da B, Xie F. A novel heterogeneous ensemble credit scoring model based on bstacking approach. Expert Systems with Applications 2018; 93 (1): 182-199. doi: 10.1016/j.eswa.2017.10.022
[17] Shunmugapriya P, Kanmani S. Optimization of stacking ensemble configurations through artificial bee colony algorithm. Swarm and Evolutionary Computation 2013; 12: 24-32. doi: 10.1016/j.swevo.2013.04.004
[18] Kohavi R, John GH. Wrappers for feature subset selection. Artificial Intelligence 1997; 1 (2): 273-324. doi: 10.1016/S0004-3702(97)00043-X
[19] Ledezma A, Aler R, Sanchis A, Borrajo D. GA-stacking: Evolutionary stacked generalization. Intelligent Data Analysis 2010; 14 (1): 89-119. doi: 10.3233/IDA-2010-0410
[20] Chen Y, Wong ML, Li H. Applying ant colony optimization to configuring stacking ensembles for data mining. Expert Systems with Applications 2014; 41 (6): 2688-2702. doi: 10.1016/j.eswa.2013.10.063
[21] Tran B, Xue B, Zhang M, Nguyen S. Investigation on particle swarm optimization for feature selection on high-dimensional data: local search and selection bias. Connection Science 2016; 28 (3): 270-294. doi: 10.1080/09540091.2016.1185392
[22] Xue B, Zhang M, Browne WN. Particle swarm optimization for feature selection in classification: Novel initialization and updating mechanisms. Applied Soft Computing 2014; 18 (3): 261-276. doi: 10.1080/09540091.2016.1185392
[23] Tang B, Chen Q, Wang X, Wang X. Reranking for stacking ensemble learning. In: Neural Information Processing Theory and Algorithms - 17th International Conference; Sydney, Australia; 2010. pp. 575-584.
[24] Karaboga D. An Idea Based on Honey Bee Swarm for Numerical Optimization. Tech Rep-TR06. Kayseri, Turkey: Computer Engineering Department, Erciyes University.
[25] Cai D, Yu Y, Wei J. A modified artificial bee colony algorithm for parameter estimation of fractional-order nonlinear systems. IEEE Access 2018; 6: 48600-48610. doi: 10.1109/ACCESS.2018.2859978
[26] Schiezaro M, Pedrini H. Data feature selection based on artificial bee colony algorithm. EURASIP Journal on Image and Video Processing 2013; 47 (1): 1-20. doi: 10.1186/1687-5281-2013-47
[27] Xiang W, Li Y, He R, Gao M, An M. A novel artificial bee colony algorithm based on the cosine similarity. Computers & Industrial Engineering 2018; 115: 54-68. doi: 10.1016/j.cie.2017.10.022
[28] Shi C, Kong X, Yu PS, Wang B. Multi-label ensemble learning. In: Machine Learning and Knowledge Discovery in Databases - European Conference; Athens, Greece; 2011. pp. 223-239.
[29] Xu J, Liu J, Yin J, Sun C. A multi-label feature extraction algorithm via maximizing feature variance and featurelabel dependence simultaneously. Knowledge Based Systems 2016; 98: 172-184. doi: 10.1016/j.knosys.2016.01.032
[30] Xu J. Fast multi-label core vector machine. Pattern Recognition 2013; 46 (3): 885-898. doi: 10.1016/j.patcog.2012.09.003
[31] Xioufis ES, Tsoumakas G, Vlahavas IP. An empirical study of lazy multilabel classification algorithms. In: Artificial Intelligence: Theories, Models and Applications, 5th Hellenic Conference on AI; Syros, Greece; 2008. pp. 401-406.
[32] Madjarov G, Gjorgjevikj D, Dzeroski S. Two stage architecture for multi-label learning. Pattern Recognition 2012; 45 (3): 1019-1034. doi: 10.1016/j.patcog.2011.08.011
[33] Gjorgjevikj D, Madjarov G. Two stage classifier chain architecture for efficient pair-wise multi-label learning. In: 2011 IEEE International Workshop on Machine Learning for Signal Processing; Beijing, China; 2011. pp. 1-6
[34] Cheng W, Hüllermeier E. Combining instance-based learning and logistic regression for multilabel classification. Machine Learning 2009; 76 (2-3): 211-225. doi: 10.1007/s10994-009-5127-5
[35] Fürnkranz J, Hüllermeier E, Mencía EL, Brinker K. Multilabel classification via calibrated label ranking. Machine Learning 2008; 73 (2): 133-153. doi: 10.1007/s10994-008-5064-8
[36] Dems̆ar J. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 2006; 7: 1-30. doi: 10.1007/s10846-005-9016-2