Hasan ERBAY, Kemal ÜRETE, Hadi Hakan MARAŞ

Detection of hand osteoarthritis from hand radiographs using convolutional neural networks with transfer learning

Osteoarthritis is the most common type of arthritis. Hand osteoarthritis leads to specific structural changes in the joints, such as asymmetric joint space narrowing and osteophytes (bone spurs). Conventional radiography has traditionally been the primary method of visualizing these structural changes and diagnosing osteoarthritis. We aimed to develop a computerized method that is capable of determining the structural changes seen in radiography of the hand and to assist practitioners in interpreting radiographic changes and diagnosing the disease. In this retrospective study, transfer-learning-based convolutional neural networks were trained on a randomly selected dataset containing 332 radiography images of hands from an original set of 420 and were validated with the remaining 88. Multilayer convolutional neural network models were designed based on a transfer learning method using pretrained AlexNet, GoogLeNet, and VGG-19 networks. The accuracies of the models were 93.2% for AlexNet, 94.3% for GoogLeNet, and 96.6% for VGG-19. The sensitivities of these models were 0.9167 for AlexNet, 0.9184 for GoogLeNet, and 0.9574 for VGG-19, while the specificity values were 0.9500, 0.9744, and 0.9756, respectively. The performance metrics, including accuracy, sensitivity, specificity, and precision, of our newly developed automated diagnosis methods are promising in the diagnosis of hand osteoarthritis. Our computer-aided detection systems may help physicians in interpreting hand radiography images, diagnosing osteoarthritis, and saving time.

PDF

___

1] Zhang Y, Jordan JM. Epidemiology of osteoarthritis. Clinics in Geriatric Medicine 2010; 26(3): 355-369.
[2] Hayashi D, Roemer FW, Guermazi A. Imaging for osteoarthritis. Annals of Physical and Rehabilitation Medicine 2016; 59(3): 161-169.
[3] Leung GL, Rainsford KD, Kean WF. Osteoarthritis of the hand i: aetiology and pathogenesis, risk factors, investigation and diagnosis. Journal of Pharmacy and Pharmacology 2014; 66(3): 339-346.
[4] Ramonda R, Frallonardo P, Musacchio E, Vio S, Punzi L. Joint and bone assessment in hand osteoarthritis. Clinical Rheumatology 2014; 33(1): 11-19.
[5] Schaefer LF, McAlindon TE, Eaton CB, Roberts MB, Haugen IK et al. The associations between radiographic hand osteoarthritis definitions and hand pain: data from the osteoarthritis initiative. Rheumatology International 2018; 38(3): 403-413.
[6] Rossignol M, Leclerc A, Allaert FA, Rozenberg S, Valat JP et al. Primary osteoarthritis of hip, knee, and hand in relation to occupational exposure. Occupational and Environmental Medicine 2005; 62(11): 772-777.
[7] Mathiessen A, Cimmino MA, Hammer HB, Haugen IK, Iagnocco A et al. Imaging of osteoarthritis (oa): What is new? Best Practice & Research Clinical Rheumatology 2016; 30(4): 653-669.
[8] Roemer FW, Eckstein F, Hayashi D, Guermazi A. The role of imaging in osteoarthritis. Best practice & Research Clinical Rheumatology 2014; 28(1): 31-60.
[9] Kellgren JH, Lawrence JS. Radiological assessment of osteo-arthrosis. Annals of The Rheumatic Diseases 1957; 16(4): 494.
[10] Greenspan H, Ginneken BV, Summers RM. Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique. IEEE Transactions on Medical Imaging 2016; 35(5): 1153-1159.
[11] Li H, Giger ML, Huynh BQ, Antropova NO. Deep learning in breast cancer risk assessment: evaluation of convolutional neural networks on a clinical dataset of full-field digital mammograms. Journal of Medical Imaging 2017; 4(4): 041304.
[12] Pereira S, Pinto A, Alves V, Silva CA. Brain tumor segmentation using convolutional neural networks in mri images. IEEE Transactions on Medical Imaging 2016; 35(5): 1240-1251.
[13] Setio AAA, Ciompi F, Litjens G, Gerke P, Jacobs C et al. Pulmonary nodule detection in ct images: false positive reduction using multi-view convolutional networks. IEEE Transactions on Medical Imaging 2016; 35(5): 1160-1169.
[14] Üreten K, Erbay H, Maraş HH. Detection of rheumatoid arthritis from hand radiographs using a convolutional neural network. Clinical Rheumatology 2020; 39(4): 969-974
[15] Dreyer KJ, Geis JR. When machines think: radiology’s next frontier. Radiology 2017; 285(3): 713-718.
[16] Shen D, Wu G, Suk HI. Deep learning in medical image analysis. Annual Review of Biomedical Engineering 2017; 19:221-248.
[17] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems; Red Hook, New York, USA; 2012. pp. 1097-1105.
[18] Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ et al. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and< 0.5 mb model size. arXiv preprint arXiv: 2016; 1602.07360.
[19] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; Las Vegas, NV, USA; 2016. pp. 770-778.
[20] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 2014; 1409.1556.
[21] Szegedy C, Liu W, Jia Y, Sermanet P, Reed S et al. Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition; Boston, MA, USA; 2015. pp. 1-9.
[22] Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017; 542(7639): 115.
[23] Rawat, W and Wang, Z. Deep convolutional neural networks for image classification: A comprehensive review In: Neural Computation 2017; 29(9): 2352-2449.
[24] Kim DH, MacKinnon T. Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks. Clinical Radiology 2018; 73(5): 439-445.
[25] Anwar SM , Majid M, Qayyum A, Awais M, Alnowami M et al. Medical image analysis using convolutional neural networks: a review. Journal of Medical Systems 2018; 42(11): 226.
[26] Xue Y, Zhang R, Deng Y, Chen K, Tao Jiang T. A preliminary examination of the diagnostic value of deep learning in hip osteoarthritis. PloS one 2017; 12(6): e0178992.
[27] Tiulpin A, Thevenot J, Rahtu E, Lehenkari P, Saarakkala S. Automatic knee osteoarthritis diagnosis from plain radiographs: A deep learning-based approach. Scientific Reports 2018; 8(1): 1727
[28] Górriz M, Antony J, McGuinness K, Giró-i-Nieto X, O’Connor NE. Assessing Knee OA Severity with CNN attention-based end-to-end architectures, arXiv preprint arXiv: 2019; 1908.08856.
[29] Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 2014; 15(1): 1929-1958.
[30] Wong SC, Gatt A, Stamatescu V, McDonnell MD. Understanding data augmentation for classification: when to warp? In: 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA); New York, USA; 2016. pp. 1-6.
[31] Tobias H, Barros P, Wermter S. The effects of regularization on learning facial expressions with convolutional neural networks. In : International Conference on Artificial Neural Networks; Barcelona, Spain; 2016. pp. 80-87.
[32] Phaisangittisagul, E. An analysis of the regularization between L2 and dropout in single hidden layer neural network In : 2016 7th International Conference on Intelligent Systems, Modelling and Simulation (ISMS); Bangkok (Thailand); 2016. pp:174-179.
[33] Kohn MD, Sassoon AA, Fernando ND. Classifications in brief: Kellgren-Lawrence classification of osteoarthritis. Clinical Orthopaedics and Related Research 2016; 474(8): 1886-93