Onion Weed Classification System through Deep Learning

Abstract

Agriculture is one of the most essential and highly critical aspects of human survival on Earth. Farming allows for creating large stockpiles of food that can feed and nourish the population effectively. India is largely based on farming and other consumable products that are even exported to the rest of world. This makes agriculture and important part of the Indian subcontinent as it allows for an effective growth and provides nourishment to the nation as well as the dependent countries. The main problems associated with the detection of presence of weed in the farm is usually done manually and takes a lot of the farmer’s time and energy and can be almost impossible for a single farmer if the farm is large. This takes up a lot of labor time which can be used for other and more important tasks. The improvement in the technology can be implemented to improve weed detection and identification. Therefore, for providing a solution for this problem this approach proposes an effective and useful mechanism for weed classification using Convolutional Neural Networks along with Decision Making. The approach has been evaluated effectively through experimentation which has been crucial in the realization of the performance of the methodology.

Introduction

I. INTRODUCTION

The paradigm of agriculture is one of the most critical and essential elements of developing a healthy population and society. The early humans have also relied on agriculture after years of hunting and gathering fruits and other useful grains. The hunting phase of the humans had smaller groups of individuals that were together and sustained on hunting other animals in groups for their sustenance. The groups also gathered berries and other fruits, by carefully selecting them from the poisonous ones and consuming them for nutrition and energy for other tasks. This went on for several years where the human race predominantly ate what was found and then subsist on what was available.

Most of the instances the food and other resources were not abundant or consistent. This was identified by the early humans, as well as, the humans were ingenious to pick up on the process of seed germination and the complex process of growing useful crops. This led to the slow transition of the humans towards an agrarian society which subsisted on growing the crops for feeding the now large groups of people which could not be sustained using hunting and gathering. This was crucial to the improvement of the lifestyle of the humans and allowing increased nutrition to be available to the individuals to perform a lot more complex intensive tasks.

Nowadays agriculture is being performed increasingly all over the globe, with a large amount of farmland and increasingly efficient farming techniques such as insecticides, pesticides and fertilizers to improve the yield considerably. This has been crucial as the dependence on the agriculture has increased significantly over the past few decades. The technological improvements have been essential for the marked improvement in the various facilities of the cultivation procedure, but have been insufficient in the elimination of weeds in the cropland. This is a problematic occurrence as it can be extremely difficult to identify and remove the weed plants manually.

The weed plants are undesirable as they can compete with the crops for the nutrients and can negatively impact the yield and can considerably reduce the health of the crops in the field. The weed plants are resilient and highly competitive which can be quite detrimental to the onion crops. Therefore, to improve the methodology for the weed classification this research paper defines an effective technique for the automatic weed classification through the use of image processing on a live camera feed of the farm. The Convolutional Neural Networks are being used for the purpose of achieving the detection through training and testing on frames that are pre-processed and normalized. The approach has the ability to reduce the farmer’s effort and also improve the yield by effectively identifying the weed plants.

The Literature Survey component of this research paper examines previous work. Section 3 delves into the approach in depth, while section 4 focuses on the outcomes evaluation. Finally, Section 5 brings this report to a close and gives some hints for future research.

II. LITERATURE SURVEY

Arif Sheeraz et al. [1] created a new methodology depending on convolutional LSTM for the categorization of weed plants. Although alternative techniques for plant categorization that rely on computer vision have been suggested, this study field still requires improved and autonomous classifiers. For weed plants, the suggested technique has excellent detection and classification capabilities. The algorithms have been tested extensively, and the results indicate that the proposed schemes are a far superior categorization technique. Commercial solutions are also required for weed plant identification.

C. Lammie et al. have constructed and presented the first FPGA-accelerated binarized DNN especially focused on weed species classification. Their fully FPGA-accelerated DNN runs on a single System On a Chip (SoC), requiring no additional hardware or a host computer for partial processing [2]. The authors investigate the effect of downsampling input photographs on DNN classification accuracy and discover that drastically lowering image resolution has just a tiny effect. They compare and contrast their GPU and FPGA implementations, revealing that their novel binarized FPGA-accelerated DNNs utilize substantially less power and have faster per-image inference times than their traditional GPU-accelerated counterparts.

M. Das et al. provides a new deep learning approach for pixel-level semantic segmentation of weed plant, canola plant, and canola flea beetle damages in the context of a complicated soil background. The model architecture and objective function were created to learn discriminative features to segment the imbalanced classes. Baseline models are also trained and evaluated in the same environment to analyze the performance of the proposed model [3]. Furthermore, data preprocessing has been enhanced to address the problem of class imbalance. All experiments and assessments are conducted using a four-class segmentation dataset.

M. H. Asad et al. offer a reliable and scalable approach for estimating LAI using multispectral remote sensing and proximate sensing data. Semantic segmentation is utilized to estimate crops and weeds from limited high-resolution ground image samples. Planet Lab RapidEye satellite imagery is being used. CropPro Consulting [4] provides SWAT and soil property maps. The authors discovered that WLAI taken from high-resolution ground photography is closely associated with SWAT zones. CLAI is more uniform throughout SWAT zones since soil-related concerns are handled with variable rate sowing and fertilizer application. The findings also show that local changes predicted with limited high-resolution ground imagery may be utilized to forecast LAIs for the entire field with acceptable accuracy.

H. S. Ullah et al. presented a pixel-level semi-automatic image labeling system. This technique is appropriate for distinguishing weeds and crops in agricultural photographs [5]. Field photos are divided into two categories: backdrop and vegetation, and then fine-tuned to reduce noise. The authors classify the minority class by drawing polygons and then using image subtraction to label the other class. They tweak the standard U-Net model to obtain an easily deployable, memory and response-efficient approach. The adjustment results in a comparable performance score. Using dilated convolution with three different dilation rates, they extract features from the supplied pictures.

In machine vision challenges T. Ilyas et al. suggested a unique and effective deep learning strategy for semantic segmentation of healthy and diseased/overgrown strawberries for harvesting purposes. The system includes adaptive receptive field and channel selection modules, which enable the network to handle varying-sized instances and associated feature maps. While transforming information from encoder to decoder, the bottleneck module computes the rich feature [6]. The authors provide a dataset with a significant degree of non-uniformity in the distribution due to photos from varied situations with varying camera sensors, lighting, and focal length. The depiction of the network's intermediate layers demonstrates the efficacy of the modules utilized.

Osorio Kavir et al. compared three weed estimate algorithms in lettuce farms, all of which rely on deep learning image analysis rather than expert eye evaluations. Support vector machines (SVM) are one approach that employs a scatter plot of oriented gradients (HOG) as a feature representation [7]. The second approach used YOLOV3 for object identification and Mask R-CNN for feature extraction for each person. An NDVI index was utilized as a background demultiplexer to remove non-photosynthetic objects in addition to these procedures. According to the defined criteria, the machine and deep learning algorithms received F1 scores of 88 percent, 94 percent, and 94 percent for crop detection, respectively. The observed crops were then translated to a binary mask and combined with the NDVI background subtractor to identify weeds indirectly. The covering % of the marijuana was assessed using a typical image processing technique after receiving the weed pictures.

S. Shorewala et al. describe a semi-supervised method for reliably estimating weed density and dispersion to help precision agriculture. Only color photographs are utilized as input in the suggested method. To begin, remove all of the backdrop pixels to create a binary vegetation mask. The pixels are clustered into either background or vegetation using an unsupervised network. The mask is then overlaid over the input color picture, which is then divided into smaller sections [8]. These smaller areas are then labeled as either weeds or crops.

The performance of two types of classifiers is investigated in this research: a) classifiers that use a pre-trained ResNet50 as a feature, such as SVM, Gaussian Naive Bayes, Neural Network, and Random Forest; and b) classifiers that use a pre-trained ResNet50 as a feature, such as SVM, Gaussian Naive Bayes, Neural Network, and Random Forest.

S. I. Moazzam et al. presented a new framework for weed identification, which uses a patch-depend classification technique rather than semantic segmentation to make real-time intelligent aerial spraying systems more realistic [9].

The authors created a novel VGG-Beet convolutional neural network (CNN) for classification that is dependent on a generic CNN (VGG) model with 11 convolutional layers. To gather sugar beet information for testing, they employed 3-channel multispectral sensors with a ground control points distance of 0.2 cm/pixel and an elevation of 4 meters. For a better comparison, the authors used different publicly accessible sugar beet crop aerial photography datasets, one taken with a 5-channel hyperspectral sensor and the other with a 4-channel hyperspectral sensor with a ground sample distance of 1cm and a height of 10 meters. The studies used three different multispectral sensor datasets, and the scientists revealed that the identical channels in these sensors had different wavelengths, requiring each sensor to have its trained model.

Deep learning and image processing were used by X. Jin et al. to suggest a technique for identifying weeds in vegetable plantations. The algorithm was broken down into two parts. To recognize veggies, a CenterNet model was trained. The trained CenterNet had 95.6 percent accuracy, 95.0 percent recall, and an F1 score of 0.953. Weeds were then assigned to the remaining green items in the color picture.

A color index was established and tested using Genetic algorithms (GAs) depending on Bayesian classification error to remove weeds from the background. Therefore, the model concentrates solely on detecting vegetables, eliminating the handling of numerous weed species [10].

III. PROPOSED METHODOLOGY

The proposed model for the identification of weed in the onion crop is explained in detail with below mentioned steps.

1. Step 1: Dataset preparation: This is the initial step of the proposed system where two plastic trays of size around 1 × 1 feet are being used to grow the onion crops. One tray is maintained with pure onion crop whereas the other one is added with some weeds. Around some 2000 images of these weed and non-weed crops are collected through the opencv cv library using the python programming language. The obtained images are segmented as the training and testing images to train the proposed system for the purpose of identification of the weeds.
2. Step 2: pre-processing  -  An image data generator object is created for the keras python library class using  some factors like rescaling with 1:255, a shear range of 0.2 and a zoom range of 0.2. A resized size of 150 × 150 is set for 64 batches with class mode binary for both training and testing objects for the respective dataset images. A 500 epochs are adjusted to train the dataset using the convolutional neural network as explained in the next step.
3. Step 3: Training with Convolution neural network ( CNN)- To train the obtained images a Convolution neural network is being employed using the keras and tensor flow libraries in python.  For the purpose of deployment a sequential neural network model is selected for the 3 layers of neurons. In the first layer a 32 kernels are set to the size 3×3 along with the activation function Relu for the set dimension and the color channel of 3.   First layer is added with a max pooling layer at the end. The second and the third layers are  also built similar to the first layer. After the 3 layers a flatten layer is  stopping the process of training with a dense layer of size 100 and with an activation function called Relu. Then finally the data is gathered using another dense layer of uni size along with the activation function sigmoid. An adam optimizer is used to optimize the precision of neuron values, this finally yields a trained data to store in a file with an extension of. H5. The Whole learning process through CNN is shown in the below mentioned architectural table.

4. Step 4: Decision Making- This is the step where testing process is conducted by feeding the live onion crop images, Where the crop is grown in the plastic tray or it may be from live fields.  The images are streamed using the open CV object and then finally the images are subjected to the CNN process along with the loading of the trained model stored with the extension h5. Obtained prediction is used to  make the decision for WEED or NON-WEED  crop using the if then rules to alert the respective former using the Whatspp along with a sample image.

IV. RESULTS AND DISCUSSIONS

The proposed methodology for the detection of the weeds in the onion crop is deployed using the python programming language through the anaconda distribution and spyder IDE. The proposed model deployed in windows based Core i5 processor machine with 8GB of primary memory. For the evolution of the proposed model live onion crop is cultivated in the plastic trays and then thousands of images streamed from the camera to form the testing and training images. The obtained images are used for the deployment of the model as explained in the prior step. The captured images from the live onion crops are shown in the below figure 3.

Conclusion

The proposed methodology for the purpose of achieving the weed classification on a onion crop has been elaborated in this research article. The presented technique has been facilitated on python programming language. The paradigm of weed classification is usually performed manually by the farmer on their fields. This is an intensely laborious and time consuming process. if the weed plants are not effectively eliminated from the farm, it could lead to a decreased crop yield. This is due to the weed crop competing with the onion crop for the resources on the field. This is an undesirable scenario that can be effectively circumvented through the use of computer vision as proposed by this approach, The Convolutional Neural Networks have been designed for the evaluation of the weed plants through effective training using the actual onion crop images. The testing is also performed for the approach which has identified the weed plants with reasonable accuracy. The extensive evaluation of the approach has also been executed which has resulted in a highly satisfactory performance. For the purpose of future research directions, the approach can be further augmented by training the neural network on images starting from the germination stage to the harvesting stage of the onion crop in different conditions and crop fields for improving the accuracy significantly.

References

[1] Arif Sheeraz, Kumar Rajesh, Abbasi Shazia, Mohammadani, Khalid & Dev Kapeel. (2021). Weeds Detection and Classification using Convolutional Long-Short-Term Memory. 10.21203/rs.3.rs-219227/v1. [2] C. Lammie, A. Olsen, T. Carrick, and M. Rahimi Azghadi, \"Low-Power and High-Speed Deep FPGA Inference Engines for Weed Classification at the Edge,\" in IEEE Access, vol. 7, pp. 51171-51184, 2019, DOI: 10.1109/ACCESS.2019.2911709. [3] M. Das and A. Bais, \"DeepVeg: Deep Learning Model for Segmentation of Weed, Canola, and Canola Flea Beetle Damage,\" in IEEE Access, vol. 9, pp. 119367-119380, 2021, DOI: 10.1109/ACCESS.2021.3108003. [4] M. H. Asad and A. Bais, \"Crop and Weed Leaf Area Index Mapping Using Multi-Source Remote and Proximal Sensing,\" in IEEE Access, vol. 8, pp. 138179-138190, 2020, DOI: 10.1109/ACCESS.2020.3012125. [5] H. S. Ullah, M. H. Asad, and A. Bais, \"End to End Segmentation of Canola Field Images Using Dilated U-Net,\" in IEEE Access, vol. 9, pp. 59741-59753, 2021, DOI: 10.1109/ACCESS.2021.3073715. [6] T. Ilyas, A. Khan, M. Umraiz, Y. Jeong and H. Kim, \"Multi-Scale Context Aggregation for Strawberry Fruit Recognition and Disease Phenotyping,\" in IEEE Access, vol. 9, pp. 124491-124504, 2021, DOI: 10.1109/ACCESS.2021.3110978. [7] Osorio, Kavir, Andrés Puerto, Cesar Pedraza, David Jamaica, and Leonardo Rodríguez. 2020. \"A Deep Learning Approach for Weed Detection in Lettuce Crops Using Multispectral Images\" Agri Engineering 2, no. 3: 471-488. [8] S. Shorewala, A. Ashfaque, R. Sidharth and U. Verma, \"Weed Density and Distribution Estimation for Precision Agriculture Using Semi-Supervised Learning,\" in IEEE Access, vol. 9, pp. 27971-27986, 2021, DOI: 10.1109/ACCESS.2021.3057912. [9] S. I. Moazzam et al., \"A Patch-Image Based Classification Approach for Detection of Weeds in Sugar Beet Crop,\" in IEEE Access, vol. 9, pp. 121698-121715, 2021, DOI: 10.1109/ACCESS.2021.3109015. [10] X. Jin, J. Che, and Y. Chen, \"Weed Identification Using Deep Learning and Image Processing in Vegetable Plantation,\" in IEEE Access, vol. 9, pp. 10940-10950, 2021, DOI: 10.1109/ACCESS.2021.3050296.

Copyright

Copyright © 2022 Nilesh Dagadu Navale, Ravindra Babasaheb Nagare, Pravin Babasaheb Khakale, Shubham Bapurao Gavhane. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.