Gürkan ÖZTÜRK, Gürhan CEYLAN

Revised polyhedral conic functions algorithm for supervised classification

In supervised classification, obtaining nonlinear separating functions from an algorithm is crucial for predic- tion accuracy. This paper analyzes the polyhedral conic functions (PCF) algorithm that generates nonlinear separating functions by only solving simple subproblems. Then, a revised version of the algorithm is developed that achieves better generalization and fast training while maintaining the simplicity and high prediction accuracy of the original PCF algo- rithm. This is accomplished by making the following modifications to the subproblem: extension of the objective function with a regularization term, relaxation of a hard constraint set and introduction of a new error term. Experimental results show that the modifications provide %12 better generalization on average and up to 10x faster training. This paper also contributes to the literature by providing detailed comparisons of the other classification algorithms that use polyhedral conic functions for the first time

PDF

___

1] Carrizosa E, Morales DR. Supervised classification and mathematical optimization. Computers & Operations Research 2013; 40 (1): 150-165.
[2] Kotsiantis SB, Zaharakis I, Pintelas P. Supervised machine learning: A review of classification techniques. Emerging Artificial Intelligence Applications in Computer Engineering 2007; 160: 3-24. [3] Silva APD. Optimization approaches to supervised classification. European Journal of Operational Research 2017; 261 (2): 772-788.
[4] Bennett KP, Parrado-Hernández E. The interplay of optimization and machine learning research. Journal of Machine Learning Research 2006; 7: 1265-1281.
[5] Üney F, Türkay M. A mixed-integer programming approach to multi-class data classification problem. European Journal of Operational Research 2006; 173 (3): 910-920.
[6] Jain P, Kar P. Non-convex optimization for machine learning. Foundations and Trends® in Machine Learning 2017; 10 (3-4): 142-336.
[7] Gasimov RN, Ozturk G. Separation via polyhedral conic functions. Optimization Methods and Software 2006; 21 (4): 527-540.
[8] Cortes C, Vapnik V. Support-vector networks. Machine Learning 1995; 20 (3): 273-297.
[9] Hofmann T, Schölkopf B, Smola AJ. Kernel methods in machine learning. The Annals of Statistics 2008; 36 (3): 171-1220.
[10] Gönen M, Alpaydın E. Multiple kernel learning algorithms. Journal of Machine Learning Research 2011; 12 (64): 2211-2268.
[11] Cimen E, Ozturk G, Gerek ON. Incremental conic functions algorithm for large scale classification problems. Digital Signal Processing 2018; 77: 187-194.
[12] Ozturk G, Ciftci MT. Clustering based polyhedral conic functions algorithm in classification. Journal of Industrial & Management Optimization 2015; 11 (3): 921.
[13] Hawkins DM. The problem of overfitting. Journal of Chemical Information and Computer Sciences 2004; 44 (1): 1-12.
[14] Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 2014; 15 (1): 1929-1958.
[15] Cevikalp H, Triggs B. Polyhedral conic classifiers for visual object detection and classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Honolulu, Hawaii, USA; 2017. pp. 261-269.
[16] Uylaş Satı N, Ordin B. Application of the polyhedral conic functions method in the text classification and comparative analysis. Scientific Programming 2018; 2018: 1-11. .
[17] Cimen E, Ozturk G. O-pcf algorithm for one-class classification. Optimization Methods and Software 2019; 2019: 1-15.
[18] Cevikalp H, Saglamlar H. Polyhedral conic classifiers for computer vision applications and open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 201; 1: 1-20.
[19] Öztürk G, Çimen E. Polyhedral conic kernel-like functions for svms. Turkish Journal of Electrical Engineering & Computer Sciences 2019; 27 (2): 1172-1180.
[20] Uylaş Satı N. A novel semisupervised classification method via membership and polyhedral conic functions. Turkish Journal of Electrical Engineering & Computer Sciences 2020; 28 (1): 80-92.
[21] Ozturk G, Bagirov AM, Kasimbeyli R. An incremental piecewise linear classifier based on polyhedral conic separation. Machine Learning 2015; 101 (1-3): 397-413
[22] Astorino A, Gaudioso M, Seeger A. Conic separation of finite sets I. the homogeneous case. Journal of Convex Analysis 2014; 53: 301-322.
[23] Kasimbeyli R. A nonlinear cone separation theorem and scalarization in nonconvex vector optimization. SIAM Journal on Optimization 2010; 20 (3): 1591-1619.
[24] Evgeniou T, Pontil M, Poggio T. Regularization networks and support vector machines. Advances in Computational Mathematics 2000; 13 (1): 1-50.
[25] Xu H, Caramanis C, Mannor S. Robustness and regularization of support vector machines. Journal of Machine Learning Research 2009; 10 (51): 1485-1510.
[26] Smola AJ, Schölkopf B, Müller KR. The connection between regularization operators and support vector kernels. Neural Networks 1998; 11 (4): 637-649.
[27] Chang CC, Lin CJ. Libsvm. A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2011; 2 (3): 1-27.
[28] Dua D, Graff C. UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences. Oakland, CA, USA: University of California, 2017.
[29] Rossum G. Python Reference Manual. Amsterdam, Netherlands: Centre for Mathematics and Computer Science, 1995.
[30] Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual. Beaverton, OR: USA: Gurobi Optimization, LLC, 2020.
[31] Pedregosa F. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 2011; 12: 2825-2830