Orchard Mapping with Deep Learning Semantic Segmentation

Abstract

The goal of orchard mapping with deep learning semantic segmentation is to automatic detection and localization of the orchard tree canopy under varied situations like multiple seasons, various tree ages, and varying levels of weed covering. The accuracy of segmentation depends on many factors like season, canopy size, and presence of weeds, background soil condition, and untreated soil. The proposed research have several combinations of training & test data based on orchard conditions The proposed research work contains deep learning convolutional neural network variant. The algorithm U- net network will be used. This helps to automatic detect and localize the tree canopy size using UAV images. Image segmentation will help all farmers by automatically segmenting tree canopies from aerial images for the development of integrated tools. After measurement of these parameters, tree canopy volume can be detected of targeted tree.

Introduction

I. INTRODUCTION

Using information and communication technology, precision agriculture (PA) is a contemporary management approach that focuses on efficiently managing the temporal and spatial variability of field and crop attributes. Here, a range of sensors and technologies are combined to provide a comprehensive dataset of field characteristics, capturing both temporal and spatial variability and looking for the precise causes of their occurrence. The non-contact monitoring of agricultural properties using the radiation reflected by the plants is known as remote sensing (RS). It is widely employed in agricultural systems for mapping chores. It makes use of aerial or ground-based imagery.

Farmers can measure the size of the canopy and the volume of plants by segmenting the orchard canopy in order to make informed crop decisions. Sematic segmentation aims to automatically identify and locate the orchard tree canopy in a variety of conditions, such as different seasons, different tree ages, and different weed cover percentages. All of the Unmanned Aerial Vehicle (UAV) photos are gathered to create the test and train data set for the sematic segmentation database. Semantic segmentation, which automatically separates the tree canopies from aerial photographs, benefits the farming industry by offering a tool for the on-the-spot evaluation of orchard ecosystems. Using UAV images, farmers can assess the size, shape, and gaps in the canopy as well as the suitable tree age and the potential yield output of the orchard. They can also estimate the amount of water and pesticides used.

II. LITERATURE REVIEW

1. Messina, G.; Modica, G. Applications of UAV Thermal Imagery in Precision Agriculture: State of the Art and Future Research Outlook. Remote. Sens. 2020

Precision agriculture (PA) utilizes unmanned aerial vehicles (UAVs) for low-altitude remote sensing (RS), which is a potent instrument. Thermal RS can be used in a variety of ways in such situation. Thermal RS is a great method for real-time identification of plant stress situations since the surface temperature of plants varies quickly under stress. UAV thermal RS is being used for a variety of tasks, including as phenotyping plants, identifying plant illnesses, and monitoring plant water stress. However, understanding the fundamental characteristics of thermal radiation is necessary for the proper use and interpretation of thermal data. This paper aimed to review the state of the art of UAV thermal RS in agriculture, outlining an overview of the latest applications and providing a future research outlook. 't fit properly and data is not visible).

2. Ofori, M.; El-Gayar, O. Towards deep learning for weed detection: Deep convolutional neural network architectures for plant seedling classification. August 2020

The majority of traditional weed management methods on farms rely on manual labor. This technique is time-consuming, expensive, and causes significant yield losses. However, the traditional use of chemical weed management is in opposition to the pursuit of sustainability.

Precision agriculture researchers have employed remote sensing weed maps to tackle this using computer vision, however this has mostly proven ineffectual for early season weed removal because to issues including solar reflection and cloud cover in satellite imagery.

This study used deep convolutional neural networks' (DCNN's) automatic feature extraction capabilities to categorize plant seedlings in light of recent developments in artificial intelligence. DCNNs can successfully classify crops and weeds in a variety of phonological growth stages, according to a comparative research.

3. Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation.  October 2015

There is broad agreement that thousands of annotated training samples are necessary for successful deep network training. This research presented a network and training technique that heavily depends on data augmentation to make better use of the existing annotated samples. The architecture comprised of a symmetric expanding path that permits exact localization and a contracting path to capture context. The results of the study demonstrate that such a network may be trained end-to-end from a small number of images and exceeds the previous best technique using u-net architecture that achieves very good performance on very different biomedical segmentation applications.

4. Schiefer, F.; Kattenborn, T.; Frick, A.; Frey, J.; Schall, P.; Koch, B.; Schmidtlein, S. Mapping forest tree species in high resolution UAV-based RGB-imagery by means of convolutional neural networks. ISPRS J. Photogramm. Remote. Sens. 2020

The study demonstrated that high accuracy tree species mapping across heterogeneous temperate forests is possible using RGB imagery from consumer-grade UAVs in conjunction with a CNN-based semantic segmentation. 

The evaluated CNN-based tree species mapping included a range of spatial resolutions, various tile sizes, and the addition of height data (nDSM). In cases where there were sufficient reference data available, the tile size had little to no impact on the model's accuracy. 

The research highlighted the interactions between CNN-based segmentation techniques and high resolution UAV data. This study gave encouraging future prospects for applications in forestry or extensive and long-term ecological research by showcasing the possibility of coordinated usage of UAVs and CNNs.

5. Onishi, M.; Ise, T. Explainable identification and mapping of trees using UAV RGB image and deep learning. Sci. Rep. 2021

Research is now being done on the identification and mapping of trees using remotely sensed data for use in forest management.  In this study, they developed a convolutional neural network (CNN) using a Red-Green-Blue (RGB) image taken by an unmanned aerial vehicle (UAV) machine vision system for tree recognition and mapping. In this paper, they first determined the slope from the three-dimensional model obtained by the UAV, and then automatically segmented the UAV RGB photograph of the forest into several tree crown objects using color and three-dimensional information, as well as the slope model. Finally, object-based CNN classification was applied for each image of a crown.

The guided gradient-weighted class activation mapping (Guided Grad-CAM) revealed that the CNN classified trees based on their shapes and leaf contrasts, which increases the system's potential for classifying individual trees with similar colors in a practical way that is cost-effective

III. PROBLEM STATEMENT

The aim of the proposed research work is to develop deep learning based semantic segmentation of tree canopy

IV. PROPOSED WORK

The proposed research will built around computer vision methods and data-driven algorithms. An annotated dataset will be generated by hiding the tree canopies in a significant number of Unmanned Aerial Vehicle (UAV) collected images. This annotated dataset will then be used to generate a massive dataset for supervised learning.

The model's selected deep-learning algorithm will be taught to precisely identify tree canopies and separate them from the surrounding environment using this annotated dataset. Consequently, each target tree canopy's shape will be included in a mask image that is generated. The weighted average of each segmented mask will be used to determine its centroid. This will function as a respectable estimate of the tree trunk.

V. PROPOSED SYSTEM ARCHITECTURE

VI. METHODOLOGY

1. Module 1: Dataset

Imported data will be divided into train and test sets. At least one image from each use case must be included in the test set for the strategy to be implemented;

2. Module 2: Preprocessing

An essential component of computer vision tasks will be image pre-processing when using self-learning algorithms. For the numerical calculations to be performed, the images must be transformed into the appropriate sizes and forms. The initial process involve shrinking all of the images to 512 x 512 pixels and reducing their size and additionally color enhancement will be done.

3. Module 3: U-net algorithm

The training data will fed into the U-net and the model learns to create proper segmentations for each image. An evaluation metric is used across a randomly selected validation set comprising 10% of the training set, so that the trained model can learn to create better segmentation masks. The trained model produces segmentation masks for the test images and the evaluation metric is applied.

4. Module 4: Compare & Predict

False Positive (FP): Incorrect identification of trees at locations.

False Negatives (FN): Failing to identify trees.

In order to manually check for false positive or false negative segmentations, the segmentation masks will be compared to the real masks produced during annotation. The performance of overall model will be defined by accuracy of trained model on test data set and also ratio of false positive over false negative

Conclusion

The problem of accurately identifying and segmenting tree canopies in different orchards using UAS photos was the subject of semantic segmentation. Numerous uses for tree segmentation exist, including mapping orchard landscapes to locate tree trunks for autonomous ground vehicle navigation. The method can also be used to precisely measure the volume of tree canopies in orchards, which can help ascertain the trees\' age and prospective productivity.

References

[1] Messina, G.; Modica, G. Applications of UAV Thermal Imagery in Precision Agriculture: State of the Art and Future Research Outlook. Remote. Sens. 2020 [2] Ofori, M.; El-Gayar, O. Towards deep learning for weed detection: Deep convolutional neural network architectures for plant seedling classification. August 2020 [3] Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. October 2015 [4] Schiefer, F.; Kattenborn, T.; Frick, A.; Frey, J.; Schall, P.; Koch, B.; Schmidtlein, S. Mapping forest tree species in high resolution UAV-based RGB-imagery by means of convolutional neural networks. ISPRS J. Photogramm. Remote. Sens. 2020 [5] Onishi, M.; Ise, T. Explainable identification and mapping of trees using UAV RGB image and deep learning. Sci. Rep. 2021

Copyright

Copyright © 2023 Avadhut Shankar Salavi Kumbhar, Prof. Mrs. S.S. Patil. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.