Baidaa AL-BANDER, Rwayda KH. S.AL-HAMD, Qahtan M. YAS, Hussain MAHDI

Benchmarking of deep learning algorithms for skin cancer detection based on a hybrid framework of entropy and VIKOR techniques

Skin cancer is one of the most common cancers worldwide caused by excessive development of skin cells. Considering the rapid growth of the use of deep learning algorithms for skin cancer detection, selecting the optimal algorithm has become crucial to determining the efficiency of computer-aided diagnosis (CAD) systems developed for the healthcare sector. However, a sufficient number of criteria and parameters must be considered when selecting an ideal deep learning algorithm. A generally accepted method for benchmarking deep learning models for skin cancer classification is unavailable in the current literature. This paper presents a multi-criteria decision-making framework for evaluating and benchmarking deep learning models for skin cancer detection based on hybridisation of entropy and VIseKriterijumska Optimizacija I Kompromisno Resenje (VIKOR) methods. Twelve well-known convolution networks are evaluated and tested on eleven publicly available image datasets to achieve the target of the study. Several criteria related to deep convolutional neural networks (CNNs) architectures, including optimisation technique, transfer learning, class balancing, transfer learning, data augmentation, and network complexity, have been considered in the multi-criteria evaluation. The decision matrix (DM) is designed based on a crossover of the five evaluation criteria and twelve (CNNs) classification models on different datasets. Subsequently, in the benchmarking and ranking of deep learning classification models, multi-criteria decision making (MCDM) techniques are used. The MCDM uses a scheme that involves the integration of entropy with VIKOR approaches. For the weight calculations of evaluation criteria, entropy is applied, while VIKOR is used to benchmark and rank the models. The obtained results reveal that the InceptionResNetV2 model gained the first rank and is selected as the optimal architecture for skin cancer detection considering the five criteria investigated in our study. The presented framework achieves a significant performance in selecting the best algorithm, which could provide substantial guidance to the researcher working in the field.

PDF

___

[1] Siegel R, Miller K, Jemal A. Cancer statistics, 2020. CA: A Cancer Journal for Clinicians 2020; 70 (1):7-30.
[2] Glazer A, Rigel D. Analysis of trends in geographic distribution of us dermatology workforce density. JAMA Dermatology 2017; 153 (5):472-473. doi: 10.1001/jamadermatol.2016.6032
[3] Sarvamangala D, Kulkarni R. Convolutional neural networks in medical image understanding: a survey. Evolutionary Intelligence 2021; 1-22. doi: 10.1007/s12065-020-00540-3
[4] Naeem A, Farooq M, Khelifi A, Abid A. Malignant melanoma classification using deep learning: datasets, performance measurements, challenges and opportunities. IEEE Access 2020; 8:110575-110597. doi: 10.1109/ACCESS.2020.3001507
[5] Esteva A, Kuprel B, Novoa R, Ko J, Swetter S et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017; 542 (7639):115-118. doi:10.1038/nature21056
[6] Haenssle H, Fink C, Schneiderbauer R, Toberer F, Buhl T et al. Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Annals of Oncology 2018; 29 (8):1836-1842. doi: 10.1093/annonc/mdy166
[7] Fujisawa Y, Otomo Y, Ogata Y, Nakamura Y, Fujita R at al. Deep-learning-based, computer-aided classifier developed with a small dataset of clinical images surpasses board-certified dermatologists in skin tumour diagnosis. British Journal of Dermatology 2019; 180 (2): 373-381. doi: 10.1111/bjd.16924
[8] Tschandl P, Rosendahl C, Akay B, Argenziano G, Blum A et al. Expert-level diagnosis of nonpigmented skin cancer by combined convolutional neural networks. JAMA Dermatology 2019; 155 (1): 58-65. doi: 10.1001/jamadermatol.2018.4378
[9] Gessert N, Nielsen M, Shaikh M, Werner R, Schlaefer A. Skin lesion classification using ensembles of multi-resolution efficientnets with meta data. MethodsX 2020; 7: 100864. doi:10.1016/j.mex.2020.100864
[10] Ha Q, Liu B, Liu F. Identifying melanoma images using efficientnet ensemble: Winning solution to the SIIM-ISIC melanoma classification challenge. arXiv Preprint; 2020. arXiv:2010.05351.
[11] Pérez E, Reyes O, Ventura S. Convolutional neural networks for the automatic diagnosis of melanoma: An extensive experimental study. Medical Image Analysis 2021; 67: 101858. doi:10.1016/j.media.2020.101858
[12] Zeleny M. Multiple criteria decision making Kyoto 1975. Springer Science & Business Media, 2012.
[13] Opricovic S, Tzeng G. Extended vikor method in comparison with outranking methods. European Journal of Operational Research 2007; 178 (2): 514-529. doi:10.1016/j.ejor.2006.01.020
[14] Codella N, Gutman D, Celebi M, Helba B, Marchetti M et al. Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging. In: IEEE 15th International Symposium on Biomedical Imaging (ISBI ); 2018. pp. 168-172.
[15] Gutman D, Codella N, Celebi E, Helba B, Marchetti M et al. Skin lesion analysis toward melanoma detection: A challenge at the international symposium on biomedical imaging (ISBI). arXiv Preprint; 2016. arXiv:1605.01397.
[16] Tschandl P, Rosendahl C, Kittler H. The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data 2018; 5 (1): 1-9. doi: 10.1038/sdata.2018.161
[17] Mendonça T, Ferreira P, Marques J, Marcal A, Rozeira J. Ph 2-a dermoscopic image database for research and benchmarking. In: 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); 2013. pp. 5437-5440.
[18] Giotis I, Molders N, Land S, Biehl M, Jonkman M et al. Med-node: A computer-assisted melanoma diagnosis system using non-dermoscopic images. Expert Systems with Applications 2015; 42 (19):6578-6585. doi: 10.1016/j.eswa.2015.04.034
[19] LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. In: Proceedings of the IEEE; 1998. pp. 2278-2324.
[20] Krizhevsky A, Sutskever I, Hinton G. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 2012; 25:1097-1105. doi:10.1145/3065386
[21] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv Preprint 2014. arXiv:1409.1556.
[22] Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016. pp. 2818-2826.
[23] Szegedy C, Ioffe S, Vanhoucke V, Alemi A. Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the AAAI Conference on Artificial Intelligence; 2017. pp. 1-31.
[24] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016. pp. 770-778.
[25] Huang G, Liu Z, Maaten L, Weinberger K. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2017. pp. 4700-4708.
[26] Chollet F. Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2017. pp. 1251-1258.
[27] Howard A, Zhu M, Chen B, Kalenichenko D, Wang W et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv Preprint 2017. arXiv:1704.04861.
[28] Zoph B, Vasudevan V, Shlens J, Le Q. Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2018. pp. 8697-8710.
[29] Goodfellow I, Bengio Y, Courville A, Bengio Y. Deep learning. MIT Press Cambridge, 2016.
[30] Cui Y, Jia M, Lin T, Song Y, Belongie S. Class-balanced loss based on effective number of samples. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2019. pp. 9268-9277.
[31] Yosinski J, Clune J, Bengio Y, Lipson H. How transferable are features in deep neural networks? arXiv Preprint 2014. arXiv:1411.1792.
[32] Wong S, Gatt A, Stamatescu V, McDonnell M. Understanding data augmentation for classification: when to warp? In: 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA), IEEE; 2016. pp. 1-6.
[33] He K, Sun J. Convolutional neural networks at constrained time cost. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2015. pp. 5353-5360.
[34] Hainmueller J. Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies. Political Analysis; 2012, 25-46. doi:10.1093/pan/mpr025
[35] Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics; 2010. pp. 249-256.
[36] Boughorbel S, Jarray F, El-Anbari M. Optimal classifier for imbalanced data using matthews correlation coefficient metric. PloS One 2017; 12 (6): e0177678. doi: 10.1371/journal.pone.0177678