Devrim ÜNAY, Özden YALÇIN ÖZUYSAL, Devrim PESEN OKVUR, Behçet Uğur TÖREYİN, Aydin AYANZADEH, Sevgi ÖNAL

Improved cell segmentation using deep learning in label-free optical microscopy images

The recently popular deep neural networks (DNNs) have a significant effect on the improvement of segmentation accuracy from various perspectives, including robustness and completeness in comparison to conventional methods. We determined that the naive U-Net has some lacks in specific perspectives and there is high potential for further enhancements on the model. Therefore, we employed some modifications in different folds of the U-Net to overcome this problem. Based on the probable opportunity for improvement, we develop a novel architecture by using an alternative feature extractor in the encoder of U-Net and replacing the plain blocks with residual blocks in the decoder. This alteration makes the model superconvergent yielding improved performance results on two challenging optical microscopy image series: a phase-contrast dataset of our own (MDA-MB-231) and a brightfield dataset from a well-known challenge (DSB2018). We utilized the U-Net with pretrained ResNet-18 as the encoder for the segmentation task. Hence, following the modifications, we redesign a novel skip-connection to reduce the semantic gap between the encoder and the decoder. The proposed skip-connection increases the accuracy of the model on both datasets. The proposed segmentation approach results in Jaccard Index values of 85.0% and 89.2% on the DSB2018 and MDA-MB-231 datasets, respectively. The results reveal that our method achieves competitive results compared to the state-of-the-art approaches and surpasses the performance of baseline approaches.

PDF

___

[1] Jaccard N, Griffin LD, Keser A, Macown RJ, Super A et al. Automated method for the rapid and precise estimation of adherent cell culture characteristics from phase contrast microscopy images. Biotechnology and Bioengineering 2014; 11 (3): 504-517.
[2] Chalfoun J, Majurski M, Peskin A, Breen C, Bajcsy P et al. Empirical gradient threshold technique for automated segmentation across image modalities and cell lines. Journal of Microscopy 2015; 260 (1): 86-99.
[3] Ambühl ME, Brepsant C, Meister JJ, Verkhovsky AB, Sbalzarini IF. High‐resolution cell outline segmentation and tracking from phase‐contrast microscopy images. Journal of Microscopy 2012; 245 (2): 161-170.
[4] Seroussi I, Veikherman D, Ofer N, Yehudai‐Resheff S, Keren K. Segmentation and tracking of live cells in phase‐contrast images using directional gradient vector flow for snakes. Journal of Microscopy 2012; 247 (2): 137-146.
[5] Chowdhury S, Kandhavelu M, Yli-Harja O, Ribeiro AS. Cell segmentation by multi-resolution analysis and maximum likelihood estimation (MAMLE). BMC Bioinformatics 2013; 14 (10): 1-3.
[6] Essa E, Xie X, Errington RJ, White N. A multi-stage random forest classifier for phase contrast cell segmentation. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); Milan, Italy; 2015. pp. 3865-3868.
[7] Su H, Yin Z, Huh S, Kanade T. Cell segmentation in phase contrast microscopy images via semi-supervised classification over optics-related features. Medical Image Analysis 2013; 17 (7): 746-765
[8] Falk T, Mai D, Bensch R, Çiçek Ö, Abdulkadir A et al. U-Net: deep learning for cell counting, detection, and morphometry. Nature Methods 2019; 16 (1): 67-70.
[9] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention; Munich, Germany; 2015. pp. 234-241. Springer, Cham.
[10] Milletari F, Navab N, Ahmadi SA. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV); Stanford, CA, USA; 2016. pp. 565-571.
[11] Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M et al. Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999.
[12] Leclerc S, Smistad E, Grenier T, Lartizien C, Ostvik A et al. RU-Net: A refining segmentation network for 2D echocardiography. In 2019 IEEE International Ultrasonics Symposium (IUS) 2019, pp. 1160-1163. IEEE.
[13] Alom MZ, Yakopcic C, Taha TM, Asari VK. Nuclei segmentation with recurrent residual convolutional neural networks based U-Net (R2U-Net). In: NAECON 2018-IEEE National Aerospace and Electronics Conference; Dayton, OH, USA; 2018. pp. 228-233.
[14] Badrinarayanan V, Kendall A, Cipolla R. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2017; 39 (12): 2481-2495.
[15] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. 2014.
[16] Iglovikov V, Shvets A. Ternausnet: U-net with vgg11 encoder pre-trained on imagenet for image segmentation. arXiv preprint arXiv:1801.05746. 2018.
[17] Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J. Unet++: A nested u-net architecture for medical image segmentation. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pp. 3-11. Springer, Cham, 2018.
[18] Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Honolulu, HI, USA; 2017. pp. 4700-4708.
[19] Azad R, Asadi-Aghbolaghi M, Fathy M, Escalera S. Bi-directional ConvLSTM U-Net with densley connected convolutions. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops 2019.
[20] Tsai HF, Gajda J, Sloan TF, Rares A, Shen AQ. Usiigaci: Instance-aware cell tracking in stain-free phase contrast microscopy enabled by machine learning. SoftwareX 2019; 9: 230-237.
[21] Jha D, Riegler MA, Johansen D, Halvorsen P, Johansen HD. Doubleu-net: A deep convolutional neural network for medical image segmentation. In: 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS) 2020, pp. 558-564. IEEE.
[22] Cao X, Lin Y. Caggnet: Crossing aggregation network for medical image segmentation. In: 2020 25th International Conference on Pattern Recognition (ICPR); Milan, Italy; 2021. pp. 1744-1750. IEEE.
[23] Ayanzadeh A, Yağar HO, Özuysal ÖY, Okvur DP, Töreyin BU et al. Cell segmentation of 2D phase-contrast microscopy images with deep learning method. In: 2019 Medical Technologies Congress (TIPTEKNO) 2019, pp. 1-4. IEEE.
[24] Ayanzadeh A, Özuysal ÖY, Okvur DP, Önal S, Töreyin BU et al. Deep learning based segmentation pipeline for label-free phase-contrast microscopy images. In: 2020 28th Signal Processing and Communications Applications Conference (SIU) 2020, pp. 1-4. IEEE.
[25] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Las Vegas, NV, USA; 2016. pp. 770-778.
[26] Russakovsky O, Deng J, Su H, Krause J, Satheesh S et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision. 2015;115 (3):211-52.
[27] Binici RC, Şahin U, Ayanzadeh A, Töreyin BU, Önal S et al. Automated segmentation of cells in phase contrast optical microscopy time series images. In2019 Medical Technologies Congress (TIPTEKNO) 2019; pp. 1-4. IEEE.
[28] Schneider CA, Rasband WS, Eliceiri KW. NIH Image to ImageJ: 25 years of image analysis. Nature Methods 2012; 9 (7): 671-675.
[29] Caicedo JC, Goodman A, Karhohs KW, Cimini BA, Ackerman J et al. Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl. Nature Methods 2019; 16 (12): 1247-1253.
[30] Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014.
[31] Tomar NK, Jha D, Ali S, Johansen HD, Johansen D et al. DDANet: Dual decoder attention network for automatic polyp segmentation. arXiv preprint arXiv:2012.15245. 2020.