Olcay KURŞUN, C. Okan ŞAKAR, Oleg FAVOROV

Using covariates for improving the minimum redundancy maximum relevance feature selection method

Maximizing the joint dependency with a minimum size of variables is generally the main task of feature selection. For obtaining a minimal subset, while trying to maximize the joint dependency with the target variable, the redundancy among selected variables must be reduced to a minimum. In this paper, we propose a method based on recently popular minimum Redundancy-Maximum Relevance} (mRMR) criterion. The experimental results show that instead of feeding the features themselves into mRMR, feeding the covariates improves the feature selection capability and provides more expressive variable subsets.

Anahtar Kelimeler:

Mutual information; mRMR; unsupervised learning; support vector machines; SINBAD covariates

Using covariates for improving the minimum redundancy maximum relevance feature selection method

Keywords:

Mutual information; mRMR; unsupervised learning; support vector machines; SINBAD covariates,

PDF

___

J. G. Zhang, & H.W. Deng, “Gene selection for classiŞcation of microarray data based on the Bayes error”, BMC Bioinformatics, Vol. 8, pp. 370, 2007.
C. E. Shannon, “A mathematical theory of communication”, Bell System Technical Journal, Vol. 27, pp. 379-423, 656, 1948.
C. Ding & H. Peng, “Minimum redundancy feature selection from microarray gene expression data”, Journal of Bioinformatics and Computational Biology, Vol. 3(2), pp. 185-205, 2005.
N. Kwak & C. H. Choi, “Input feature selection by mutual information based on Parzen Window”, IEEE Transac- tions on Pattern Analysis and Machine Intelligence, Vol. 24(12), pp. 1667-1671, 2002.
H. Peng, F. Long, F., C. Ding, “Feature selection based on mutual information: Criteria of max-dependency, max- relevance, and min-redundancy”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27(8), pp. 1238, 2005.
UCI Learning Repository, http://archive.ics.uci.edu/ml/, June 2008.
H. Ozdemir, G. Demir, G. Altay, S. Albayrak, C. Bayat, “Prediction of tropospheric ozone concentration in Istanbul by employing artiŞcial neural network”, Environmental Engineering Science, Vol. 25(9), 2008.
P. A. Est´evez, M. Tesmer, C. A. Perez, J. M. Zurada, “Normalized mutual information feature selection”, IEEE Transactions on Neural Networks, Vol. 20(2), 2009.
O. Favorov & D. Ryder, “SINBAD: a neocortical mechanism for discovering environmental variables and regularities hidden in sensory input”, Biological Cybernetics , Vol. 90, pp. 191-202, 2004.
O. Kursun & O. Favorov, “Feature selection and extraction using an unsupervised biologically-suggested approxi- mation to Gebelein’s maximal correlation”, International Journal of Pattern Recognition and ArtiŞcial Intelligence, (to appear). O. Kursun & O. Favorov, “SINBAD automation of scientiŞc discovery: From factor analysis to theory synthesis”, Natural Computing, Vol. 3(2), pp. 207-233, 2004.
O. Kursun & O. Favorov, “What can SVMS teach each other?”, ArtiŞcial Neural Networks in Engineering (ANNIE ), St Louis, MO, USA, 2004.
A. Bal, “Widrow-cellular neural network and optoelectronic implementation”, Optik, Vol. 115, pp. 295-300, 2004.
C. Fyfe, Hebbian Learning and Negative Feedback Networks, Springer, 2005.
C. W. Hsu & C. J. Lin, “A comparison of methods for multi-class support vector machines”, IEEE Trans. Neural Networks, Vol. 13, pp. 415-425, (LIBSVM software available for downloading at http://www.csie.ntu.edu.tw/∼cjlin/libsvm). M. A. Little, P. E. McSharry, E. J. Hunter, L. O. Ramig, “Suitability of dysphonia measurements for telemonitoring of Parkinson’s disease”, IEEE Transactions on Biomedical Engineering, Vol. 56(2), pp. 1015-1022, 2009.
O. Sakar & O. Kursun, “Telediagnosis of Parkinson’s disease using measurements of dysphonia”, Journal of Medical Systems, 2009 (Online published).
A. C. Comrie, “Comparing neural networks and regression models for ozone forecasting”, J. Air Waste Manage, Vol. 47(6), pp. 653-663, 1997.
D. Wang, W. Z. Lu, “Interval estimation of urban ozone level and selection of inﬂuential factors by employing automatic relevance determination model”, Chemosphere, Vol. 62, pp. 1600-1611, 2006.
J. Gomes-Sanchis, J. D. Martin-Guerrero, E. Soria-Olivas, J. Villa-Frances, J. L. Carrasco, S. D. Valle-Tascon, “Neu- ral networks for analyzing the relevance of input variables in the prediction of tropospheric ozone concentration”, Atmospheric Environment, Vol. 40, pp. 6173-6180, 2006.
O.P. Barcenas, E. Soria-Olivas, J. D. Martin-Guerrero, G. Camps-Valls, J. L. Carrasco-Rodriguez, S. del Valle- Tascon, “Unbiased sensitivity analysis and pruning techniques in neural networks for surface ozone modeling”, Ecological Modelling, Vol. 182, pp. 149–158, 2005.
S. A. Abdul-Wahab, S. M. Al-Alawi, “Assessment and prediction of tropospheric ozone concentration levels using artiŞcial neural networks”, Environmental Modelling & Software, Vol. 17, pp. 219-228, 2002.
H. A. Guvenir, B. Acar, G. Demiroz, A. Cekin, “A supervised machine learning algorithm for Arrhythmia analysis”, Proceedings of the Computers in Cardiology Conference, Lund, Sweden, 1997.