A hybrid search method of wrapper feature selection by chaos particle swarm optimization and local search

Finding a subset of features from a large dataset is a problem that arises in many fields of study. Since the increasing number of features has extended the computational cost of a system, it is necessary to design and implement a system with the least number of features. The purpose of feature selection is to find the best subset of features from the original ones. The result of the best selection is improving the computational cost and the accuracy of the prediction. A large number of algorithms have been proposed for feature subset selection. In this paper, we propose a wrapper feature selection algorithm for a classification that is based on chaos theory, binary particle swarm optimization, and local search. In the proposed algorithm, the nearest neighbor algorithm is used for the evaluation phase

PDF

___

[1] Jain A, Zongker D. Feature selection: evaluation, application, and small sample performance. IEEE T Pattern Anal 1997; 19: 153-158.
[2] Mitra S, Kundu PP, Pedrycz W. Feature selection using structural similarity. Inform Sciences 2012; 198: 48-61.
[3] Gheyas IA, Smith LS. Feature subset selection in large dimensionality domains. Pattern Recogn 2010; 43: 5-13.
[4] Wang Y, Dahnoun N, Achim AA. Novel system for robust lane detection and tracking. Signal Process 2010; 92: 319-334.
[5] Sikonja MR, Kononenko I. Theoretical and empirical analysis of ReliefF and RReliefF. Mach Learn 2003; 53: 23-69.
[6] Lutu PEN, Engelbrecht AP. A decision rule based method for feature selection in predictive data mining. Expert Syst Appl 2010; 37: 602-609.
[7] Miller A. Subset Selection in Regression. 2nd ed. Boca Raton, FL, USA: CRC Press, 2002.
[8] Tsai JS, Huang WB, Kuo YH, Horng MF. Joint robustness and security enhancement for feature based image watermarking using invariant feature regions. Signal Process 2012; 92: 1431-1445.
[9] Choi SI, Oh J, Choi CH, Kim C. Input variable selection for feature extraction in classification problem. Signal Process 2012; 92: 636-648.
[10] Kabira MM, Shahjahan M, Murase K. A new hybrid ant colony optimization algorithm for feature selection. Expert Syst Appl 2012; 39: 3747-3763.
[11] Navot A. On the role of feature selection in machine learning. PhD, Hebrew University, Jerusalem, Israel, 2006.
[12] Unler A, Murat A. A discrete particle swarm optimization method for feature selection in binary classification problems. Eur J Oper Res 2010; 206: 528-539.
[13] Hsu HH, Hsieh CW, Lu MD. Hybrid feature selection by combining filters and wrappers. Expert Syst Appl 2011; 38: 8144-8150.
[14] Vieira SM, Sousa JMC, Kaymak U. Fuzzy criteria for feature selection. Fuzzy Set Syst 2012; 189: 1-18.
[15] Tahir MA, Smith J. Creating diverse nearest-neighbor ensembles using simultaneous metaheuristic feature selection. Pattern Recogn Lett 2010; 31: 1470-1480.
[16] Whitney AW. A direct method of nonparametric measurement selection. IEEE T Comput 1971; 20: 1100-1103.
[17] Marill T, Green DM. On the effectiveness of receptors in recognition systems. IEEE T Inform Theory 1963; 9: 11-17.
[18] Breaban M, Luchian H. A unifying criterion for unsupervised clustering and feature selection. Pattern Recog 2011; 44: 854-865.
[19] Webb A. Statistical Pattern Recognition. 2nd ed. New York, NY, USA: Wiley, 2002.
[20] Li YM, Zhang SJ, Zeng XP. Research of multi population agent genetic algorithm for feature selection. Expert Syst Appl 2009; 36: 11570-11581.
[21] Vieira SM, Sousa JMC, Runkler TA. Two cooperative ant colonies for feature selection using fuzzy models. Expert Syst Appl 2010; 37: 2714-2723.
[22] Wang X, Yang J, Teng X, Xia W, Jensen R. Feature selection based on rough sets and particle swarm optimization. Pattern Recogn Lett 2007; 28: 459-471.
[23] Ronen M, Jacob Z. Using simulated annealing to optimize feature selection problem in marketing applications. Eur J Oper Res 2006; 71: 842-858.
[24] Chen B, Chen L, Chen Y. Efficient ant colony optimization for image feature selection. Signal Process 2013; 93: 1566-1576.
[25] Hua J, Tembe W, Dougherty ER. Feature selection in the classification of high dimension data. In: IEEE 2008 Genomic Signal Processing and Statistics; 810 June 2008; Phoenix, AZ, USA. pp. 1-2.
[26] Jin X, Xu A, Bie R, Guo P. Machine learning techniques and chi square feature selection for cancer classification using SAGE gene expression profiles. Lect Notes Comp Sci 2006; 3916: 106-115.
[27] Liao C, Li S, Luo Z. Gene selection using Wilcoxon rank sum test and support vector machine for cancer. Lect Notes Comp Sci 2007; 4456: 57-66.
[28] Peng H, Long F, Ding C. Feature selection based on mutual information criteria of max dependency, max relevance and min redundancy. IEEE T Pattern Anal 2005; 27: 1226-1238.
[29] Biesiada J, Duch W. Feature selection for high dimensional data: a Pearson redundancy based filter. Adv Soft Comp 2008; 45: 242-249.
[30] Rocchi L, Chiari L, Cappello A. Feature selection of stabilometric parameters based on principal component analysis. Med Biol Eng Comput 2004; 42: 7179.
[31] Kennedy J, Eberhart RC. Particle swarm optimization. In: IEEE 1995 Neural Networks; NovemberDecember 1995; Perth, Australia. pp. 1942-1948.
[32] Muneender E, Vinod KDM. Optimal real and reactive power dispatch for zonal congestion management problem for multi congestion case using adaptive fuzzy PSO. In: IEEE 2009 TENCON; 2326 January 2009; Singapore. pp. 1-6.
[33] Lee S, Soak S, Oh S, Pedrycz W, Jeon M. Modified binary particle swarm optimization. Prog Nat Sci 2008; 18: 1161-1166.
[34] Camci F. Analysis of velocity calculation methods in binary pso on maintenance scheduling. In: IEEE 2008 Applications of Digital Information and Web Technologies; 46 August 2008; Ostrava, Czech Republic. pp. 12- 17.
[35] Kennedy J, Eberhart RC. A discrete binary version of the particle swarm algorithm. In: IEEE 1997 Systems, Man, and Cybernetics; 1215 October 1997; Orlando, FL, USA. pp. 4104-4109.
[36] Chuang LY, Yang CH, Li JC. Chaotic maps based on binary particle swarm optimization for feature selection. Appl Soft Comput 2011; 11: 239-248.
[37] Yue LG, Xiao HA, Junmin L. A particle swarm optimization algorithm with logarithm decreasing inertia weight and chaos mutation. In: IEEE 2008 Computational Intelligence and Security; 1317 December 2008; Suzhou, China. pp. 61-65.