Detection of License Plate If Helmet Not Found

Abstract

Motorbike accidents has been growing more in the current years, due to high usage of superbike which is nearly 300-1200cc, which may reach the speed of 200-300 kmph, in that hyper speed, there are more chances of accidents. In India not only over speeding causes accidents, due to bad roads and road bumps will cause accident even though people riding in low speed. Also, if raider violates the traffic rules cops may try to catch the riders, it might be bit difficult to catch the riders sometime they may escape, it will be too difficult to board a case on them by capturing the License Plate number manually, therefore its highly necessary to build an automated based helmet and license plate recognition based on Deep Learning system using YOLO algorithm.

Introduction

I. INTRODUCTION

Our purpose of this study is to develop a Non-Helmet Driver Detection Approach that will automate overall process of identifying traffic offences including not wearing helmets & retrieving its vehicle's license number plate. Object recognition using Deep Learning at 3 layers is the basic principle concerned. At first step, the item has been used to identify the people; & at second step, YOLOv3 has been used to identify the motorbike; & at third step, YOLOv3 has been used to identify the license plate; and towards the fourth step, YOLOv3 will be used to identify the license numberplate. Following that, OCR is used to retrieve the license registration numberplate (Optical Character Recognition). Each of these procedures, particularly the license number plate extracting aspect, are governed to predetermined parameters & limits. The efficiency with which this operation is completed is critical because it uses video clip as its source. We built a comprehensive solution include both headwear helmet detecting & license number plate extraction using the approaches mentioned earlier.

Artificial Intelligence (AI) is a technique whereby a trained model can work on its own based on the inputs it receives during the training period. It is also possible to build a helmet detection model by training with a specific dataset and analysing that data. These are also known as machine learning algorithms. Using helmet detection model, non-helmet riders can be easily identified. By cropping the image based on the detected class, the license plate of the rider can be seen. Those images are sent to OCR (Optical Character Recognition) model which is able to recognize the text gives the license plate as output. This process can also be performed using a webcam in real time.

In this study, we will examine the possibility of using CCTV cameras to enforce wearing of helmets. Though the implementation of the developed system,unsafe behaviours are changed and the total number of accidents and severity of their consequences is reduced.

II. RELATED WORK

A. Helmet Detection

Many methods have recently been proposed in the field of deep learning. [1] describes how to detect motorcycles from videos using the background subtraction method and the SMO classifier. To classify helmets and those without helmets, handcrafted features are used, along with CNN. A comparison of CNN and manual features shows that CNN is more accurate. A moving object is obtained from a video frame using adaptive background subtraction. The Convolutional Neural Network algorithm is being used to categories motorbikes into moving vehicles. Last, they proceed to categories the upper quarters region of motorbikes using CNN to confirm that riders are not wearing helmet [3]. The Convolution layer (GMM) is being used in to separate & identify foreground items. The method using a quicker Region-Based CNN (Faster R-CNN) to identify motorbikes among the indicated background items, ensuring that motorbikes are there. Thereafter, the R-CNN is implemented to determine motorbikes wearing or not wearing a helmet. Whereas the helmet detection [1], [2], & [3] uses deep convolutional neural network, the foreground object in the motorbike identification step is also still obtained using classical thresholding, which would be quite poor in busy scenario.

This is suggested in [4] & [5] to be using YOLOv3 [6] method is used to detect the motorcyclists wearing helmet however, no motorbike recognition is verified. In [7] and [8], we initially detected the motorbike & man inside the image using the YOLOv3 method, and afterwards they evaluated the overlapped volume of the grid cell between both the motorbike and man to identify the individual riding the motorbike. The YOLOv3 algorithms was then used to determine whether motorcyclists were wearing a helmet. Therefore, motorbikes & motorcyclist intersect significantly when traffic has been monitored, making it redundant to specifically recognize motorbikes. In [9], [10], and [11], this was suggested using the SSD / YOLOv3 algorithms to locate the motorbike region, then to remove the top part of an image and to use a classification algorithm to distinguish between a helmet & a non-helmet. Likewise, if there are multiple riders on the motorbike, this will not be an effective classification technique. They consider the motorcycles & the motorcyclists after all in [12], [13], & [14] before using the CNN model to determine if the driver is wearing helmets. The efficiency of this single-step, fine - grained detection methods is poor.

B. License Plate Detection

The use of object recognition CNNs as in LP detection phase has been explored by numerous writers. Silva and Jung [15] found that while recognizing LPs without previous vehicles detections, the Fast-YOLO model [16] only managed to achieve a poor recognition rate. In order to achieve high precision - recall rates on a dataset that contains Brazilian LPs, they employed the Fast-YOLO model structured in a cascaded way, first to detect the top view of the automobiles and afterwards its LPs in the identified areas.

For the sole purpose of LP identification, Hsu et al. [17] modified a YOLO & YOLOv2 models. Although the improved version of YOLO processed 54 FPS on just a powerful GPU, we think that LP detection techniques must be faster (i.e., 150+ FPS) as the LP elements still have to be identified. The image data was divided into smaller sections by Kurpiel et al. [18], creating an overlapped grid. A CNN was used to create a scores with each regional, & the Lp was found by examining the results of nearby sub-regions. On a real - world dataset they supplied, it takes 230 ms on a GT-740M GPU to identify Brazilian LPs in photos with several vehicle parts, with just an 83 percent accuracy rate.

In order to carry out character-based LP identification, Li et al. [19] training a CNN using characters that were taken out of ordinary texts. To create a text salience map, the networks was applied all across full image in a sliding-window method. Depending on how the characters clustered, text-like sections were identified. The initial selection boxes are then created using correlated component analysis (CCA). An LP/non-LP CNN were subsequently trained to exclude FPs. Even if the recall and precision rates attained were better than those attained in earlier research, the cost of using such a series of techniques makes them unsuitable for real world applications; on a Tesla K40c GPU, processing a simple picture takes more than two sec.

C. YOLO History

In their 2015 research paper titled "You Only Look Once: Unified, Real-Time Object Recognition," Joseph Redmon et al. proposed the 1st YOLO model. A most popular object recognition models up until that point were RCNN models. The RCNN type of model were accurate, but also because identifying the suggested regions for the bounding box, classifying these regions, and then performing post-processing to improve the output required several steps, they were quite sluggish. YOLO were developed with the intention of eliminating multistage detection and performing it in a single phase, hence lengthening the inference time.

III. EXISTING SYSTEM

As per research titled "The Global status report on road safety 2018" published by the World Health Organization, 50 million people are suffering and 1.35 million people are dying globally every year as a result of traffic accidents. It is almost difficult to imagine that walkers, bikers, & motorcyclists all bear an equal share of this burden. According to this paper, creating a thorough plan of action is necessary save the lives. The truth that India is ranked first in terms of fatalities from traffic accidents is concerning. According to analyses by specialists, this tendency is due in part to rapid urbanization and a lack of use of seat belts, helmets, or other safety equipment when driving.

IV. PROPOSED SYSTEM

The suggested an approach for extracting features utilizing hybrids descriptor based on LBP, HOG, & Hough transform descriptor. While LBP and a gray levels co-occurrence matrices were used by Xinhua Jiang et al. for extraction of features. The targeted items include motorcycles, motorcyclists, pedestrians, & laborers, & YOLOv3 and COCO datasets could be used to detect various types of objects and categorize them appropriately. Various aspects like the color of the helmets and the tires can then be used to identify motorcycles. proposed utilizing a microcontroller & accelerometer to recognize 2-wheeler crashes.

True victims of traffic deaths are frequently pedestrians, thus ensuring their safety is crucial. Je Li and others suggested technique uses a histogram of directed gradient information to categories pedestrians using SVM (HOG). Helmet detecting is the final phase. In addition to HOG descriptors, color-based and circle Hough transforms are employed to detect helmets. Identification of color features is also another possibility. For the purpose of locating the helmets, color space transformations and color feature discrimination were employed. To more accurately recognize helmets, Back-Propagation artificial neural network and GLCM statistical properties are utilized. The procedures in a helmet identification system include gathering dataset, detecting moving objects, removing backgrounds, classifying objects using neural networks, and extracting the license number if a rider is not wearing helmets.

V. METHODOLOGIES

The suggested vision-based framework combines three phases. The first step is to make a proper environment. There isn't a dataset for us to use to train our model because there isn't one. available on the market. The data preparation is next performed, which is divided into three parts: data capture, data augmentation, and data annotation are all steps in the process. To generate a more realistic setting, the images were taken with high resolution, many angles, and distinct backgrounds. To boost the diversity and complexity of the experimental dataset, enhancement techniques such as scaling, dropping, and adjusting brightness are used on the gathered photographs. Following picture enrichment, image labelling is performed, which entails drawing a boundary box around the items and labelling them as helmet or no helmet. Following the augmentation and annotation, a dataset of 25 photos was created, with 80 percent of the photos picked at random for the training sample and the remaining 20 percent for the testing selection. The image size will be reset even during training process, and the batch size will be fixed based on the GPU's memory limits. In the training, we'll utilize the optimizer with the learning rate set to 0.001 and the other parameters unchanged from the YOLO model. Following that will be the testing phase, during which a wide range of photos will be passed through the suggested solution and the results will be recorded.

A.  Data Collection

Firstly, we need to collect the images of people on motorcycle with helmet and without helmet along with license plate, from different source of internet.

B. Data Pre-Processing

By reducing undesirable distortions or boosting specific visual aspects that are important for later analysis, pre-processing of information is being used to enhance image data. If source and destination both are intensity images, "pre-processing" refers to transactions involving pictures at the most basic abstraction. Here, we're mapping the picture objects to use a free programmer named LabelIMG.

C. Algorithm

A technique for instantly identifying objects is the YOLO Convolutional Neural Network (CNN). CNNs are classi?er techniques that have the capacity to comprehend incoming pictures as well-organized collections of data and spot patterns. The advantage of YOLO is that it is significantly faster than that of other devices while maintaining accuracy & quality.

It gives the model the ability to analyses the real photo at testing time, enabling it to generate predictions representative of the entire context of the picture. Regions are "scored" by comparison with predefined categories using CNN algorithms like YOLO.

Remarkable observations are recognized in high-scoring areas, regardless of which class they more closely identify with. According to whether parts of the video compare favorably to specified vehicle categories, YOLO can, for instance, is being used to identify various car kinds in a real-time traffic stream.

VI. WORKING OF THE SYSTEM

1. Step 1: Upload an image from the interface provided.
2. Step 2: Uploaded image will be loaded to the helmet detection YOLO Model. Checks whether the image contains motorcycle and it has helmet or not.
3. Step 3: If image contains helmet it will draw the boundary box to the image.
4. Step 4: If image does not contains object helmet, it will be loaded to the license plate detection model.
5. Step 5: Loaded image will be evaluated by license plate detection model, if it recognizes the license plate from the loaded image, Later OCR image will be sent to OCR model to extract the text from the image.

VII.  RESULTS

We tested the system with both helmeted and non-helmeted bikers to ensure that the algorithm utilized in the system is robust. As predicted, the algorithm spotted the non-helmeted biker's number plate. The technology does not construct a bounding box for the number plate recognition for helmeted bikers.

VIII. FUTURE WORK

We utilized a jupyter notebook to construct the software, and it was a success. In Python, our project has been successfully tested. We also looked into the project's uses and future scope. Our solution can be linked to traffic cameras, and with minimal tweaking, it could be used to recognize non-helmet riders and license plate of the bike on real-time basis.

Conclusion

The results show that YOLO object recognition is capable of effectively categorizing and localizing all object categories and is well suited for real-time compute. The intended end-to-end model was created successfully & has all the elements required for deployment of automated and monitor. Techniques that are designed to handle the majority of cases are utilized to extract the license plates. These algorithms consider a range of scenarios, such as the presence of many motorcycle riders without helmets. All of the software & libraries used in our work are open source, which greatly increases its adaptability & affordability. The primary goal of the effort was to solve the problem of ineffective traffic control. As a result, we can draw the conclusion that any controller would embrace it.

References

[1] Shine, L.C.V.J.: Automated detection of helmet on motorcyclists from traffic surveillance videos: A comparative analysis using hand-crafted features and CNN. Multimedia Tools Appl 79(19–20), 14179– 14199 (2020) [2] Vishnu, C., et al.: Detection of motorcyclists without helmet in videos using convolutional neural network. In: 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA, 14–19 May 2017 [3] Yogameena, B.: Deep learning-based helmet wear analysis of a motorcycle rider for intelligent surveillance system. IET Intell. Transp. Syst. 13(7), 1190– 1198 (2019) [4] Allamki, L., et al.: Helmet detection using machine learning and automatic license plate recognition. International Research Journal of Engineering and Technology 06(12), 5 (2019) [5] Khan, F.A.: Helmet and number plate detection of motorcyclists using deep learning and advanced machine vision techniques. In: 2020 Second International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 15–17 July 2020 [6] Redmon, J., Farhadi, A.: YOLOv3: an Incremental Improvement. arXiv:1804.02767 [cs], (2018) [7] Saumya, A., et al.: Machine learning based surveillance system for detection of bike riders without helmet and triple rides. In: 2020 International Conference on Smart Electronics and Communication (ICOSEC), Trichy, India, 10–12 Sept. 2020 [8] Rohith, C.A., et al.: An efficient helmet detection for MVD using deep learning. In: 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI), Tirunelveli, India, 23–25 April 2019 [9] Chairat, A., et al.: Low cost, high performance automatic motorcycle helmet violation detection. In: 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass, CO, USA, 1–5 March 2020 [10] Santhosh, A.: Real-time helmet detection of motorcyclists without helmet using convolutional neural network\'IJRASET. International Journal for Research in Applied Science and Engineering Technology 8(7), 1112– 1116 (2020) [11] Dasgupta, M.: Automated helmet detection for multiple motorcycle riders using CNN. In: 2019 IEEE Conference on Information and Communication Technology, Allahabad, India, 6–8 Dec. 2019 [12] Lin, H., et al.: Helmet use detection of tracked motorcycles using CNN-based multi-task learning. IEEE Access 8, 162073– 162084 (2020) [13] Siebert, F.W., Lin, H.: Detecting motorcycle helmet use with deep learning. Accident. Anal. Prev. 134, 105319 (2020) [14] Boonsirisumpun, N.: Automatic detector for bikers with no helmet using deep learning. In: 2018 22nd International Computer Science and Engineering Conference (ICSEC), Chiang Mai, Thailand, 21–24 Nov. 2018 [15] S. M. Silva and C. R. Jung, “Real-time brazilian license plate detection and recognition using deep convolutional neural networks,” in Conference on Graphics, Patterns and Images (SIBGRAPI), Oct 2017, pp. 55–62. [16] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016, pp. 779–788.

Copyright

Copyright © 2022 Dr Minavathi, Dakshayani H N. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.