Patrick Kwabena MENSAH, Faiza Umar BAWAH, Mighty Abra AYIDZOE, Yongbin YU, Jingye CAI

Visual Interpretability of Capsule Network for Medical Image analysis

Deep learning (DL) models are currently not widely deployed for critical tasks such as in health. This is attributable to the ”black box,” making it difficult to gain the trust of practitioners. This paper proposes the use of visualizations to enhance performance verification, improve monitoring, ensure understandability, and improve interpretability needed to gain practitioners’ confidence. These are demonstrated through the development of a CapsNet model for the recognition of gastrointestinal tract infection. The gastrointestinal tract comprises several organs joined in a long tube from the mouth to the anus. It is susceptive to diseases that are difficult for medics to diagnose, since it is not easy to have physical access to the sick regions. Consequently, manual access and analysis of images of the unhealthy parts requires the skills of an expert, as it is tedious, prone to errors, and costly. Experimental results show that visualizations in the form of post-hoc interpretability can demonstrate the reliability and interpretability of the CapsNet model applied to gastrointestinal tract diseases. The outputs can also be explained to gain practitioners’ confidence in paving the way for its adoption in critical areas of society.

PDF

___

[1] Tjoa E, Guan C. A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI. IEEE Transaction on Neural Networks Learning Systems 2020.
[2] Holzinger A, Langs G, Denk H, Zatloukal K, Müller H. Causability and explainability of artificial intelligence in medicine. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 2019; 9 (4): 1–13.
[3] Samek W, Müller R. Towards Explainable Artificial Intelligence. In: Explainable AI: interpreting, explaining and visualizing deep learning. Springer, 2019, pp. 5-22.
[4] Samek W, Wiegand T, Müller R. Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint 2017; arXiv:1708.08296 2017.
[5] Lipton ZC. The mythos of model interpretability. ACM Queue 2018; 61 (10): 35–43.
[6] Ayidzoe MA, Yu Y, Mensah PK, Cai J, Adu K et al. Gabor Capsule Network with Preprocessing Blocks for the Recognition of Complex Images. Machine Vision Applications 2021; 32 (4): 1-6. doi:10.1007/s00138-021-01221-6
[7] Krizhevsky A, Sutskever L, Hinton GE. ImageNet Classification with Deep Convolutional Neural Networks. Advances in neural information processing systems 2012; 25: 1097-1105.
[8] Van Der Maaten L, Hinton G. Visualizing Data using t-SNE. Journal of machine learning research 2008; 9 (11).
[9] Baehrens D, Schroeter T, Harmeling S, Kawanabe M, Hansen K et al. Müller. How to explain individual classification decisions. The Journal of Machine Learning Research 2010; 11: 1803–1831.
[10] Bach S, Binder A, Montavon G, Klauschen F, Müller KR et al. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS One 2015; 10 (7). doi:org/10.1371/journal.pone.0130140
[11] Ribeiro MT, Singh S, Guestrin C. Why should i trust you? Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining 2016. pp. 1135–1144. doi:10.1145/2939672.2939778
[12] Ribeiro E, Uhl A, Hafner M. Colonic polyp classification with convolutional neural networks. In: 2016 IEEE 29th International Symposium on Computer-Based Medical Systems (CBMS) 2016. pp. 253-258.
[13] Shahroudnejad A, Afshar P, Plataniotis KN, Mohammadi A. Improved Explainability of Capsule Networks: Relevance Path by Agreement. In: 2018 ieee global conference on signal and information processing (globalsip). 2018. pp. 549–553.
[14] Sabour S, Frosst N, Hinton GE. Dynamic Routing Between Capsules. Advances in neural information processing systems 2017; 3856–3866.
[15] Afshar P, Mohammadi A, Plataniotis KN. Brain Tumor Type Classification via Capsule Networks. In: 2018 25th IEEE International Conference on Image Processing (ICIP). 2018. pp.3129–3133. doi:10.1109/ICIP.2018.8451379.
[16] Iesmantas T, Alzbutas R. Convolutional Capsule Network for Classification of Breast Cancer Histology Images. In: International Conference Image Analysis and Recognition 2018. pp. 853–860. doi:10.1007/978-3-319-93000-8_97
[17] Jiménez-Sánchez A, Albarqouni S, Mateus D. Capsule networks against medical imaging data challenges. In: Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, 2018. pp. 150–160. doi:10.1007/978-3-030-01364-6_17
[18] Afshar P, Oikonomou A, Naderkhani F, Tyrrell PN, Plataniotis KN et al. 3D-MCN: A 3D Multi-scale Capsule Network for Lung Nodule Malignancy Prediction. Scientific reports 2020; 10 (1): 1–11. doi:10.1038/s41598-020- 64824-5
[19] Zeiler MD, Fergus R. Visualizing and Understanding Convolutional Networks. In: European conference on computer vision, 2012.
[20] Yosinski J, Clune J, Fuchs T. Understanding Neural Networks Through Deep Visualization. ArXiv preprint 2015, arXiv:1506.06579.
[21] Mensah Kwabena P, Benjamin Asubam W, Abra Mighty A. Exploring the performance of LBP-capsule networks with K-Means routing on complex images. Journal of King Saud University-Computer and Information Sciences 2020. doi:10.1016/j.jksuci.2020.10.006
[22] Kwabena PM, Weyori BA, Mighty AA. Gabor Capsule Network for Plant Disease Detection. International Journal of Advanced Computer Science and Applications 2020; 10 (11): 388–395.
[23] Cao S, Yao Y, G An. E2-Capsule Neural Networks for Facial Expression Recognition Using AU-Aware Attention. IET Image Processing 2019; 14 (11): 2417-2424.
[24] Zhu Z, Peng G, Chen Y, Gao H. A convolutional neural network based on a capsule network with strong generalization for bearing fault diagnosis. Neurocomputing 2019; 323: 62–75. doi:10.1016/j.neucom.2018.09.050
[25] Fei H, Ji D, Zhang Y, Ren Y. Topic-Enhanced Capsule Network for Multi-Label Emotion Classification. IEEE/ACM Transactions on Audio, Speech, and Language Processing 2020; 28:1839–1848. doi:10.1109/TASLP.2020.3001390
[26] Bae J, Kim DS. End-to-end speech command recognition with capsule network. In: Interspeech, 2018, pp. 776–780. doi:10.21437/Interspeech.2018-1888
[27] Shi K, Gong C, Lu H, Zhu Y, Niu Z. Wide-grained capsule network with sentence-level feature to detect meteorological event in social network. Future Generation Computer Systems 2020; 102:323–332. doi:10.1016/j.future.2019.08.013
[28] Liu J, Lin H, Yang L, Xu B, Wen D. Multi-Element Hierarchical Attention Capsule Network for Stock Prediction. IEEE Access 2020; 8: 143114–143123. doi:10.1109/ACCESS.2020.3014506
[29] Yao H, Gao P, Wang J, Zhang P, Jiang C et al. Capsule Network Assisted IoT Traffic Classification Mechanism for Smart Cities. IEEE Internet Things Journal 2019; 6: 7515–7525. doi:10.1109/JIOT.2019.2901348
[30] Afshar P, Plataniotis KN, Mohammadi A. Capsule networks’ interpretability for brain tumor classification via radiomics analyses. In: IEEE International Conference on Image Processing (ICIP) 2019. pp. 3816–3820.
[31] Peer D, Stabinger S, Rodr A. Increasing the adversarial robustness and explainability of capsule networks with γ - capsules. ArXiv preprint 2018, arXiv:1812.0970.
[32] LaLonde R, Torigian D, Bagci U. Encoding Visual Attributes in Capsules for Explainable Medical Diagnoses. In: International Conference on Medical Image Computing and Computer-Assisted Intervention; 2020. pp. 294–304. doi:10.1007/978-3-030-59710-8_29
[33] Pogorelov C, Konstantin R, Kristin R, Griwodz SL, Eskeland C et al. Kvasir: A Multi-Class Image Dataset for Computer Aided Gastrointestinal Disease Detection. In: Proc. 8th ACM Multimed. Systems Conference, ACM; 2017. pp. 164-169. doi:10.1017/CBO9781107415324.004
[34] Brahimi M, Arsenovic M, Laraba S, Sladojevic S, Boukhalfa K et al. Deep learning for plant diseases: detection and saliency map visualisation. Human and machine learning 2018. doi:10.1007/978-3-319-90403-0_6
[35] Rahim T, Hassan SA, Shin SY. A deep convolutional neural network for the detection of polyps in colonoscopy images. Biomedical Signal Processing and Control 2021; 68: 102654. doi:10.1016/j.bspc.2021.102654
[36] Yogapriya J, Chandran V, Sumithra MG, Anitha P, Jenopaul P et al. Gastrointestinal Tract Disease Classification from Wireless Endoscopy Images Using Pretrained Deep Learning Model. Computational and Mathematical Methods in Medicine 2021; 12.