The Effect of Derived Features on Art Genre Classification with Machine Learning

Classification of the artwork according to their genres is being done for years. Although this process was used to be done by art experts before, now artificial intelligence techniques may help people manage this classification task. The algorithms used for classification are already improved, and now they can make classifications and predictions for any kind of genre classification. In this study, two different machine learning algorithms are used on an artwork dataset for genre classification. The primary purpose of this study is to show that the derived features about the artwork have a remarkable effect on correct genre classification. These features are derived from the metadata of the dataset. This metadata contains information about the nationalities and the period that the artist lived. Image filters are also applied to the images but the results show that applying only image filters on the dataset used in the study did not perform well. Instead, adding derived features extracted from the metadata increased the classification performances dramatically.

PDF

___

[1] Sengupta, N., Sahidullah, M., Saha, G., 2016. "Lung sound classification using cepstral-based statistical features". Computers in Biology and Medicine. Vol. 75(1), pp.118–129. doi:10.1016/j.compbiomed.2016.05.013.

[2] Işık, E., İnallı, M., 2018. “Artificial neural networks and adaptive neuro-fuzzy inference systems approaches to forecast the meteorological data for HVAC: The case of cities for Turkey”. Energy, Vol. 154, pp. 7-16.

[3] Işık, E., İnallı, M., Celik, E., 2019. “ANN and ANFIS Approaches to Calculate the Heating and Cooling Degree Day Values: The Case of Provinces in Turkey”. Arab J Sci Eng, Vol.44, pp. 7581–7597. https://doi.org/10.1007/s13369-019-03852- 4.

[4] Maitra, D.S., Bhattacharya, U., Parui, S.K., 2015. "CNN based common approach to handwritten character recognition of multiple scripts". 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 1021–1025. doi:10.1109/ICDAR.2015.7333916.

[5] Zujovic, J., Gandy, L., Friedman, S., Pardo, B., Pappas, T. N., 2009, “Classifying paintings by artistic genre: An analysis of features & classifiers”, 2009 IEEE International Workshop on Multimedia Signal Processing, 5-7 Oct. 2009, Rio de Janeiro, Brazil. DOI: 10.1109/MMSP.2009.5293271.

[6] Bayes, M., Price, M., 1763, "An Essay Towards Solving a Problem in The Doctrine of Chances. By the Late Rev. Mr. Bayes, F. R. S. Communicated by Mr. Price, in a Letter to John Canton, A. M. F. R. S.", Philosophical Transactions, Vol. 53, pp. 370–418.

[7] Fix, E., Hodges, J. L., 1951, “Discriminatory Analysis. Nonparametric Discrimination: Consistency Properties” USAF School of Aviation Medicine, Randolph Field, Texas.

[8] Caudill, M., 1989, "Neural Network Primer: Part I", AI Expert

[9] Cortes, C., Vapnik, V., 1995, "Support Vector Networks", Machine Learning, Vol. 20, pp. 273-297.

[10] Kégl, B., 2013, "The return of AdaBoost.MH: multi-class Hamming trees". arXiv:1312.6086.

[11] Lee, S. G., Cha, E. Y., 2016, “Style classification and visualization of art painting’s genre using self‑organizing maps”, Human-Centric Computing and Information Sciences, 6:07. DOI: 10.1186/s13673-016-0063-4.

[12] Lombardi, T. E., 2005, “The classification of style in fine-art painting”, PhD thesis, Pace University, New York.

[13] Jafarpour, S., Polatkan, G., Brevdo, E., Hughes, S., Brasoveanu, A., Daubechies, I., 2009, "Stylistic analysis of paintings using wavelets and machine learning," In 17th European Signal Processing Conference, pp. 1220–1224.

[14] Levy, E., David, O., Netanyahu, N. S., 2013, “Painter Classification Using Genetic Algorithms”, 2013 IEEE Congress on Evolutionary Computation, pp. 3027-3034, June 20-23, Cancún, México.

[15] Shamir, L., Macura, T., Orlov, N., Eckley, D.M., Goldberg, I.G., 2010, “Impressionism, expressionism, surrealism: automated recognition of painters and schools of art”, Transact Appl Percept 7:8.

[16] Cetinic, E., Grgic, S., 2016, “Genre Classification of Paintings”, Proceedings of 58th International Symposium ELMAR, pp. 201-204, 12-14 September 2016, Zadar, Croatia.

[17] Saleh, B., Elgammal, A., 2015, “Large- scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Feature”, arXiv:1505.00855. DOI: 10.11588/dah.2016.2.23376.

[18] Fukushima, K., 2007, "Neocognitron", Scholarpedia, 2(1), 1717. DOI:10.4249/scholarpedia.1717.

[19] Crowley, E. J., 2016, “Visual Recognition in Art using Machine Learning”, PhD Thesis, Department of Engineering Science, University of Oxford.

[20] Oomen, E., 2018, “Classification of painting style with transfer learning”, Master Thesis, School of Humanities and Digital Science of Tilburg University, Tilburg, The Netherlands.

[21] Zhao, W., Zhou, D., Qiu, X., Jiang, W., 2021, “How to Represent Paintings: A Painting Classification Using Artistic Comments”, Sensors, 21, 1940. https://doi.org/10.3390/s21061940.

[22] Bar, Y., Levy, N., Wolf, L., “Classication of Artistic Styles using Binarized Features Derived from a Deep Neural Network”, 2015, Workshop at the European Conference on Computer Vision. DOI: 10.1007/978-3-319-16178-5_5.

[23] Lecoutre, A., Negrevergne, B, Yger, F., 2017, “Recognizing Art Style Automatically in painting with deep learning”, Proceedings of Machine Learning Research, 77, pp. 327-342.

[24] Sandoval, C., Pirogova, E., Lech, A. M., 2019, “Two-Stage Deep Learning Approach to the Classification of Fine-Art Paintings”, IEEE Access, 7. DOI: 10.1109/ACCESS.2019.2907986.

[25] Kaggle, Best Artworks of All Time, https://www.kaggle.com/ikarus777/best- artworks-of-all-time. Accessed 08.10.2019.

[26] Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I. H., 2009, “The WEKA data mining software: An update,” SIGKDD Explorations, vol. 11, 2009. http://www.cs.waikato.ac.nz/ml/weka/

[27] SciKit Learn, Label Encoder, https://scikit- learn.org/stable/modules/generated/sklearn .preprocessing.LabelEncoder.html. Accessed 10.10.2019.

[28] Voutsakis, E., Petrakis, E.G.M., and Milios, E. 2006. IntelliSearch: Intelligent Search for Images and Text on the Web. In Proceedings of International Conference on Image Analysis and Recognition (Povoa de Varzim, Portugal, September 18-20, 2006). ICIAR 2006, 697-708.

[29] Abidin, D., “Effects of Image Filters on Various Image Datasets”. Proceedings of the 2019 5th International Conference on Computer and Technology Applications. April 2019. Pages 1–5. https://doi.org/10.1145/3323933.3324056

[30] Balasubramani, R., Kannan, V., 2009, “Efficient use of MPEG-7 Color Layout and Edge Histogram Descriptors in CBIR Systems”, Global Journal of Computer Science and Technology. 9, 5 (2009), 157- 163.

[31] SimpleColorHistogramFilter, WEKA unsupervised instance image filter, URL:https://github.com/mmayo888/Image Filter/blob/master/ImageFilter/src/weka/filt ers/unsupervised/instance/imagefilter/Simp leColorHistogramFilter.java.

[32] Shim S. O., Choi, T. S., 2002, “Edge color histogram for image retrieval”, In Proceedings of International Conference on Image Processing (Rochester, NY, USA, September 22-25, 2002). IEEE, pp. 957- 960. DOI: 10.1109/ICIP.2002.1037942.

[33] Ho, T. K., 1995, "Random Decision Forests". Proceedings of the 3rd International Conference on Document Analysis and Recognition, 278-282, Montreal, QC, Canada.

[34] Fontana, F.A., Mäntylä, M.V., Zanoni, M., Marino, A., 2016, "Comparing and Experimenting Machine Learning Techniques for Code Smell Detection", Empirical Software Engineering, Vol. 21, pp. 1143-1191.

[35] Breiman, L., 2001, "Random Forests". Machine Learning, 5–32. doi: 10.1023/A:1010933404324.

[36] Donges N., 2018, “The Random Forest Algorithm”. https://machinelearning- blog.com/2018/02/06/the-random-forest- algorithm/#more-375 (03.09.2018).

[37] Quinlan, J. R., 1986, "Induction of Decision Trees". Machine Learning 1, 81-106.

[38] Witten, L. H., Frank, E., 2005, Data Mining: Practical Machine Learning Tools and Techniques. 2nd ed. San Francisco, USA. Morgan Kaufmann Publishers.

[39] Quinlan, J. R., 1993, C4.5 Programs for Machine Learning. San Francisco, USA. Morgan Kaufmann Publishers.

[40] Salama, A. A., Eisa, M., ELhafeez, S. A., Lotfy, M. M., 2015, “Review of Recommender Systems Algorithms Utilized in Social Networks based e- Learning Systems & Neutrosophic System”, Neutrosophic Sets and Systems, Vol. 8, pp. 33.