Adi ALHUDHAİF, Kemal POLAT, Şaban ÖZTÜRK

Attention-based end-to-end CNN framework for content-based X-ray image retrieval

The widespread use of medical imaging devices allows deep analysis of diseases. However, the task of examining medical images increases the burden of specialist doctors. Computer-assisted systems provide an effective management tool that enables these images to be analyzed automatically. Although these tools are used for various purposes, today, they are moving towards retrieval systems to access increasing data quickly. In hospitals, the need for content-based image retrieval systems is seriously evident in order to store all images effectively and access them quickly when necessary. In this study, an attention-based end-to-end convolutional neural network (CNN)framework that can provide effective access to similar images from a large X-ray dataset is presented. In the first part of the proposed framework, a fully convolutional network architecture with attention structures is presented. This section contains several layers for determining the saliency points of X-ray images. In the second part of the framework, the modified image with X-ray saliency map is converted to representative codes in Euclidean space by the ResNet-18 architecture. Finally, hash codes are obtained by transforming these codes into hamming spaces. The proposed study is superior in terms of high performance and customized layers compared to current state-of-the-art X-ray image retrieval methods in the literature. Extensive experimental studies reveal that the proposed framework can increase the current precision performance by up to 13

PDF

___

[1] Domingues I, Pereira G, Martins P, Duarte H, Santos J et al. Using deep learning techniques in medical imaging: a systematic review of applications on CT and PET. Artificial Intelligence Review 2019; 53(6): 4093-160. doi: 10.1007/s10462-019-09788-3
[2] Gates M, Wingert A, Featherstone R, Samuels C, Simon C et al. Impact of fatigue and insufficient sleep on physician and patient outcomes: a systematic review. BMJ Open 2018; 8 (9). doi: 10.1136/bmjopen-2018-021967
[3] Anwar SM, Majid M, Qayyum A, Awais M, Alnowami M et al. Medical Image Analysis using Convolutional Neural Networks: A Review. Journal of Medical Systems 2018; 42 (11). doi: 10.1007/s10916-018-1088-1
[4] Öztürk Ş. Stacked auto-encoder based tagging with deep features for content-based medical image retrieval. Expert Systems with Applications 2020; 161. doi: 10.1016/j.eswa.2020.113693
[5] Pan Z, Wang L, Wang Y, Liu Y. Product quantization with dual codebooks for approximate nearest neighbor search. Neurocomputing 2020; 401: 59-68. doi: 10.1016/j.neucom.2020.03.016
[6] Shen HT, Liu L, Yang Y, Xu X, Huang Z et al. Exploiting Subspace Relation in Semantic Labels for Cross-modal Hashing. IEEE Transactions on Knowledge and Data Engineering. 2020. doi: 10.1109/tkde.2020.2970050
[7] Öztürk Ş. Comparison of Pairwise Similarity Distance Methods for Effective Hashing. IOP Conference Series: Materials Science and Engineering 2021; 1099 (1). doi: 10.1088/1757-899x/1099/1/012072
[8] Öztürk Ş. Class-driven content-based medical image retrieval using hash codes of deep features. Biomedical Signal Processing and Control 2021; 68. doi: 10.1016/j.bspc.2021.102601
[9] Nie X, Zhou X, Shi Y, Sun J, Yin Y. Classification-enhancement deep hashing for large-scale video retrieval. Applied Soft Computing 2021. doi: 10.1016/j.asoc.2021.107467
[10] Bhatt C, Kumar I, Vijayakumar V, Singh KU, Kumar A. The state of the art of deep learning models in medical science and their challenges. Multimedia Systems 2020. doi: 10.1007/s00530-020-00694-1
[11] Nowaková J, Prílepok M, Snášel V. Medical Image Retrieval Using Vector Quantization and Fuzzy S-tree. Journal of Medical Systems 2016; 41(2). doi: 10.1007/s10916-016-0659-2
[12] Shamna P, Govindan VK, Abdul Nazeer KA. Content based medical image retrieval using topic and location model. Journal of Biomedical Informatics 2019; 91. doi: 10.1016/j.jbi.2019.103112
[13] Shamna P, Govindan VK, Abdul Nazeer KA. Content-based medical image retrieval by spatial matching of visual words. Journal of King Saud University - Computer and Information Sciences 2018. doi: 10.1016/j.jksuci.2018.10.002
[14] Khatami A, Babaie M, Tizhoosh HR, Khosravi A, Nguyen T et al. A sequential search-space shrinking using CNN transfer learning and a Radon projection pool for medical image retrieval. Expert Systems with Applications 2018; 100: 224-33. doi: 10.1016/j.eswa.2018.01.056
[15] Ahmad J, Muhammad K, Baik SW. Medical Image Retrieval with Compact Binary Codes Generated in Frequency Domain Using Highly Reactive Convolutional Features. Journal of Medical Systems 2017; 42 (2). doi: 10.1007/s10916-017-0875-4
[16] Song T, Camalan S, Niazi MKK, Moberly AC, Teknos T et al. OtoMatch: Content-based eardrum image retrieval using deep learning. Plos One 2020; 15 (5). doi: 10.1371/journal.pone.0232776
[17] Dureja A, Pahwa P. Medical image retrieval for detecting pneumonia using binary classification with deep convolutional neural networks. Journal of Information and Optimization Sciences 2020; 41 (6): 1419-31. doi: 10.1080/02522667.2020.1809096
[18] Wang S-H, Govindaraj VV, Górriz JM, Zhang X, Zhang Y-D. Covid-19 classification by FGCNet with deep feature fusion from graph convolutional network and convolutional neural network. Information Fusion 2021; 67: 208-29. doi: 10.1016/j.inffus.2020.10.004
[19] Babaie M, Tizhoosh HR, Zhu S, Shiri MEJae-p. Retrieving similar X-ray images from big image data using radon barcodes with single Projections2017 January 01, 2017: [arXiv: 1701.00449 p.]. Available from: https: //ui.adsabs.harvard.edu/abs/2017arXiv170100449B.
[20] Chung Y-A, Weng W-HJae-p. Learning deep representations of medical images using siamese CNNs with application to content-based image Retrieval2017 November 01, 2017: [arXiv: 1711.08490 p.]. Available from: https: //ui.adsabs.harvard.edu/abs/2017arXiv171108490C.
[21] Haq NF, Moradi M, Wang ZJ. A deep community based approach for large scale content based X-ray image retrieval. Medical Image Analysis 2021; 68. doi: 10.1016/j.media.2020.101847
[22] Guan Q, Huang Y, Zhong Z, Zheng Z, Zheng L et al. Thorax disease classification with attention guided convolutional neural network. Pattern Recognition Letters 2020; 131: 38-45. doi: 10.1016/j.patrec.2019.11.040
[23] Jin S, Yao H, Sun X, Zhou S, Zhang L et al. Deep saliency hashing for fine-grained retrieval. IEEE Transactions on Image Processing 2020; 29: 5336-51. doi: 10.1109/tip.2020.2971105
[24] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. 2016 ieee conference on computer vision and pattern recognition (CVPR)2016. p. 770-778. doi: 10.1109/cvpr.2016.90
[25] Cao Z, Long M, Wang J, Yu PS. HashNet: Deep Learning to Hash by Continuation. 2017 IEEE International Conference on Computer Vision (ICCV)2017. p. 5609-5618. doi: 10.1109/iccv.2017.598
[26] Wang X, Peng Y, Lu L, Lu Z, Bagheri M et al. ChestX-Ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)2017. p. 3462-3471. doi: 10.1109/cvpr.2017.369
[27] Wang Y, Ji S, Lu M, Zhang Y. Attention boosted bilinear pooling for remote sensing image retrieval. International Journal of Remote Sensing 2019; 41(7): 2704-2724. doi: 10.1080/01431161.2019.1697010
[28] Chen Z, Cai R, Lu J, Feng J, Zhou J. Order-sensitive deep hashing for multimorbidity medical image retrieval. Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. Lecture Notes in Computer Science2018. p. 620-628. doi: 10.1007/978-3-030-00928-1_70
[29] Jun W, Kumar S, Shih-Fu C. Semi-supervised hashing for large-scale search. IEEE Transactions on Pattern Analysis and Machine Intelligence 2012; 34 (12): 2393-2406. doi: 10.1109/tpami.2012.48
[30] Gong Y, Lazebnik S. Iterative quantization: A procrustean approach to learning binary codes. Cvpr 20112011. p. 817-824. doi: 10.1109/cvpr.2011.5995432
[31] Liong VE, Jiwen L, Gang W, Moulin P, Jie Z. Deep hashing for compact binary codes learning. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)2015. p. 2475-2483. doi: 10.1109/cvpr.2015.7298862
[32] Liu H, Wang R, Shan S, Chen X. Deep Supervised Hashing for Fast Image Retrieval. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)2016. p. 2064-2072. doi: 10.1109/cvpr.2016.227
[33] Lan R, Zhong S, Liu Z, Shi Z, Luo X. A simple texture feature for retrieval of medical images. Multimedia Tools and Applications 2017; 77 (9): 10853-10866. doi: 10.1007/s11042-017-5341-2