Shahbaz KHAN, Zubair Ahmad KHAN, Muhammad TUFAIL, Muhammad Tahir KHAN, Shahzad ANWAR

Deep-learning-based spraying area recognition system for unmanned-aerial-vehicle-based sprayers

Unmanned aerial vehicle (UAV)-based spraying system employing machine learning techniques is a recent advancement in precision agriculture for precise spraying, promoting saving chemicals (pesticide/herbicide), and enhanc- ing their effectiveness. This study aims to develop an eﬀicient deep learning system for UAV-based sprayers, which has the capability to accurately recognize spraying areas. A deep learning system is proposed and developed incorporating a faster region-based convolutional neural network (R-CNN) for the imagery collected. In order to develop a classifier for identifying spraying areas from nonspraying areas, four different agriculture croplands and orchards were considered. All the experiments were performed in agriculture fields through DJI Spark with an RGB camera. During experimentation, heights of 2.5 m and 6 m were attained for cropland and orchard image collection. The developed recognition system achieved 87.77% and 88.57% accuracy for recognizing spraying areas in crops and orchards, respectively, for a limited dataset and variable target sizes. The developed deep learning system on comparison outperformed other machine learning and deep learning systems in the literature. The developed system could be easily integrated into real-time UAV-based sprayers for precision agriculture.

PDF

___

[1] Puri V, Nayyar A, Raja L. Agriculture drones: A modern breakthrough in precision agriculture. Journal of Statistics and Management Systems 2017 Jul 4; 20(4): 507-18. doi: 10.1080/09720510.2017.1395171
[2] Long S. Drones and Precision Agriculture: The Future of Farming. 2019.
[3] Hayat S, Yanmaz E, Muzaffar R. Survey on unmanned aerial vehicle networks for civil applications: A communications viewpoint. IEEE Communications Surveys & Tutorials 2016 Apr 29; 18(4): 2624-2661. doi: 10.1109/COMST.2016.2560343
[4] Shakhatreh H, Sawalmeh AH, Al-Fuqaha A, Dou Z, Almaita E et al. Unmanned aerial vehicles (UAVs): A survey on civil applications and key research challenges. IEEE Access 2019 Apr 9; 7: 48572-48634. doi: 10.1109/AC- CESS.2019.2909530
[5] Sampedro C, Rodriguez-Ramos A, Bavle H, Carrio A, De la Puente P et al. A fully-autonomous aerial robot for search and rescue applications in indoor environments using learning-based techniques. Journal of Intelligent & Robotic Systems 2019; 95(2): 601-627. doi: 10.1007/s10846-018-0898-1
[6] Silvagni M, Tonoli A, Zenerino E, Chiaberge M. Multipurpose UAV for search and rescue operations in mountain avalanche events. Geomatics, Natural Hazards and Risk 2017; 8(1): 18-33. doi: 10.1080/19475705.2016.1238852
[7] Bejiga MB, Zeggada A, Nouﬀidj A, Melgani F. A convolutional neural network approach for assisting avalanche search and rescue operations with UAV imagery. Remote Sensing 2017; 9(2): 100. doi: 10.3390/rs9020100
[8] Tuyishimire E, Bagula A, Rekhis S, Boudriga N. Cooperative data muling from ground sensors to base stations using UAVs. In: 2017 IEEE Symposium on Computers and Communications (ISCC); Heraklion, Greece; 2017. pp. 35-41.
[9] Quaritsch M, Kruggl K, Wischounig-Strucl D, Bhattacharya S, Shah M, Rinner B. Networked UAVs as aerial sensor network for disaster management applications. e & i Elektrotechnik und Informationstechnik 2010;127(3): 56-63. doi: 10.1007/s00502-010-0717-2
[10] Liu P, Chen AY, Huang YN, Han JY, Lai JS et al. A review of rotorcraft unmanned aerial vehicle (UAV) developments and applications in civil engineering. Smart Structures and Systems 2014; 13(6): 1065-1094. doi: 10.12989/sss.2014.13.6.1065
[11] Deng C, Wang S, Huang Z, Tan Z, Liu J. Unmanned aerial vehicles for power line inspection: A cooperative way in platforms and communications. Journal of Communication 2014; 9(9): 687-692.
[12] Mohamadi F, inventor. Vertical takeoff and landing (VTOL) small unmanned aerial system for monitoring oil and gas pipelines. United States patent US 8, 880, 241. 2014 Nov 4.
[13] Haddal CC, Gertler J. Homeland security: Unmanned aerial vehicles and border surveillance. In: Library of Congress Washington DC Congressional Research Service; 2010. pp. 1-11.
[14] Finn RL, Wright D. Unmanned aircraft systems: Surveillance, ethics and privacy in civil applications. Computer Law & Security Review 2012; 28(2): 184-94. doi: 10.1016/j.clsr.2012.01.005
[15] Huang Y, Thomson SJ, Hoffmann WC, Lan Y, Fritz BK. Development and prospect of unmanned aerial vehicle tech- nologies for agricultural production management. International Journal of Agricultural and Biological Engineering 2013; 6(3): 1-10.
[16] Muchiri N, Kimathi S. A review of applications and potential applications of UAV. In: Proceedings of Sustainable Research and Innovation Conference; 2016. pp. 280-283.
[17] Kazmi W, Bisgaard M, Garcia-Ruiz FJ, Hansen KD, La Cour-Harbo A. Adaptive surveying and early treatment of crops with a team of Autonomous Vehicles. In: European Conference on Mobile Robots (ECMR); Örebro, Sweden; 2011. pp. 253-258.
[18] Gonzalez-Dugo V, Zarco-Tejada P, Nicolás E, Nortes PA, Alarcón JJ et al. Using high resolution UAV thermal imagery to assess the variability in the water status of five fruit tree species within a commercial orchard. Precision Agriculture 2013; 14(6): 660-678. doi: 10.1007/s11119-013-9322-9
[19] Garcia-Ruiz F, Sankaran S, Maja JM, Lee WS, Rasmussen J, Ehsani R. Comparison of two aerial imaging platforms for identification of Huanglongbing-infected citrus trees. Computers and Electronics in Agriculture 2013; 91: 106- 115. doi: 10.1016/j.compag.2012.12.002
[20] Mogili UR, Deepak BB. Review on application of drone systems in precision agriculture. Procedia Computer Science 2018; 133: 502-509. doi: 10.1016/j.procs.2018.07.063
[21] Hung C, Xu Z, Sukkarieh S. Feature learning based approach for weed classification using high resolution aerial im- ages from a digital camera mounted on a UAV. Remote Sensing 2014; 6(12): 12037-12054. doi: 10.3390/rs61212037
[22] Rebetez J, Satizábal HF, Mota M, Noll D, Büchi L et al. Augmenting a convolutional neural network with local histograms- A case study in crop classification from high-resolution UAV imagery. In: European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN); Bruges, Belgium; 2016. pp. 515-520.
[23] Milioto A, Lottes P, Stachniss C. Real-time blob-wise sugar beets vs weeds classification for monitoring fields using convolutional neural networks. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 2017; 4: 41. doi: 10.5194/isprs-annals-IV-2-W3-41-2017
[24] Lottes P, Khanna R, Pfeifer J, Siegwart R, Stachniss C. UAV-based crop and weed classification for smart farming. In: 2017 IEEE International Conference on Robotics and Automation (ICRA); Singapore; 2017. pp. 3024-3031.
[25] Bah MD, Hafiane A, Canals R. Deep learning with unsupervised data labeling for weed detection in line crops in UAV images. Remote Sensing 2018; 10(11): 1690. doi: 10.3390/rs10111690
[26] Huang H, Lan Y, Deng J, Yang A, Deng X et al. A semantic labeling approach for accurate weed mapping of high resolution UAV imagery. Sensors 2018; 18(7): 2113. doi: 10.3390/s18072113
[27] Fan Z, Lu J, Gong M, Xie H, Goodman ED. Automatic tobacco plant detection in UAV images via deep neural networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 2018; 11(3): 876-87. doi: 10.1109/JSTARS.2018.2793849
[28] Wei H, Mao J. The recognition of rice area images by UAV based on deep learning. In: MATEC Web of Conferences; Shangai, China; 2018. pp. 02057.
[29] Li Y, Qian M, Liu P, Cai Q, Li X et al. The recognition of rice images by UAV based on capsule network. Cluster Computing 2019 ; 22(4): 9515-9524. doi:10.1007/s10586-018-2482-7.
[30] Valente J, Doldersum M, Roers C, Kooistra L. Detecting rumex obtusifolius weed plants in grasslands from uav rgb imagery using deep learning. ISPRS Annals of Photogrammetry, Remote Sensing & Spatial Information Sciences 2019; 4: 179-185. doi:10.5194/isprs-annals-IV-2-W5-179-2019.
[31] Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Las Vegas, USA; 2016. pp. 779-788.
[32] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S et al. SSD: Single shot multibox detector. In: European Conference on Computer Vision; Amsterdam, the Netherlands; 2016 Oct 8 pp. 21-37.
[33] Alganci U, Soydas M, Sertel E. Comparative research on deep learning approaches for airplane detection from very high-resolution satellite images. Remote Sensing 2020; 12(3): 458. doi: 10.3390/rs12030458
[34] Tian Y, Yang G, Wang Z, Wang H, Li E et al. Apple detection during different growth stages in orchards using the improved YOLO-V3 model. Computers and Electronics in Agriculture 2019; 157: 417-426. doi: 10.1016/j.compag.2019.01.012
[35] Olaniyi OM, Daniya E, Abdullahi IM, Bala JA, Olanrewaju EA. Weed recognition system for low-land rice precision farming using deep learning approach. In: International Conference on Artificial Intelligence & Industrial Applications; Meknes, Morocco; 2020 pp. 385-402.
36] Gao P, Zhang Y, Zhang L, Noguchi R, Ahamed T. Development of a recognition system for spraying areas from unmanned aerial vehicles using a machine learning approach. Sensors 2019; 19(2): 313. doi: 10.3390/s19020313
[37] Ren S, He K, Girshick R, Sun J. Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems; Montreal, Canada; 2015. pp. 91-99.
[38] Chao YW, Vijayanarasimhan S, Seybold B, Ross DA, Deng J et al. Rethinking the faster r-cnn architecture for temporal action localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Salt Lake City, Utah, United States; 2018. pp. 1130-1139.
[39] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Las Vegas, USA; 2016. pp. 770-778.
[40] Redmon J, Farhadi A. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 2018.
[41] Zheng YY, Kong JL, Jin XB, Wang XY, Su TL et al. CropDeep: The crop vision dataset for deep-learning-based classification and detection in precision agriculture. Sensors 2019 ; 19(5): 1058. doi: 10.3390/s19051058
[42] Rohan A, Rabah M, Kim SH. Convolutional neural network-based real-time object detection and tracking for parrot AR drone 2. IEEE Access 2019; 7: 69575-69584. doi: 0.1109/ACCESS.2019.2919332