Advanced Helmet and License Plate Recognition: A Breakthrough in Safety and Surveillance Systems

Abstract

Helmet detection and license plate recognition have emerged as crucial components in ensuring safety and security in various domains, particularly in transportation and law enforcement. This review paper presents a comprehensive survey and analysis of methodologies, techniques, and advancements in the field of helmet detection and license plate recognition using computer vision and deep learning approaches. The paper investigates diverse datasets, algorithms, and architectures utilized for accurate detection and recognition of helmets worn by individuals in critical settings and the identification of license plates on vehicles. Furthermore, it explores the challenges, limitations, and future directions within these domains. The paper also evaluates the performance metrics, technological advancements, and potential applications in real-world scenarios, emphasizing the importance of these technologies in enhancing safety measures, traffic regulation, and security enforcement systems. Through this review, we aim to provide a consolidated overview and critical analysis of existing methodologies while identifying opportunities for future research and development in helmet detection and license plate recognition.

Introduction

I. INTRODUCTION

Road safety has become a critical global concern, with road accidents being a leading cause of unnatural deaths. Motorcycle accidents, in particular, contribute significantly to fatal injuries and fatalities worldwide. Many people in countries like India, Brazil, Thailand, and Malaysia rely on motorcycles for their daily commutes. While helmet laws vary from one country to another, India, for instance, mandates helmet use by law for all vehicle riders. Ensuring the safety of motorcyclists is paramount, and wearing a helmet is not just recommended but legally required. Efforts have been made to enact and enforce laws aimed at safeguarding motorcycle riders from accidents. Currently, traffic police bear the responsibility of preventing motorcycle-related injuries. However, the present traffic regulatory landscape in India faces several challenges that may benefit from innovative solutions. Riding a motorcycle without a helmet is considered a traffic violation and has contributed to an increase in accidents and fatalities. Traveling on a motorcycle carries a significantly higher risk of injury and fatality compared to driving a car. In fact, a staggering 60% of Traumatic Brain Injuries (TBIs) result from Road Traffic Accidents (RTAs). India ranks third in Asia in terms of TBI research output, underscoring the gravity of the accident situation. To mitigate the number of accidents, various measures have been implemented to monitor and address traffic law violations. However, many of these methods are manual, relying on human intervention and substantial manpower, making them inefficient and time-consuming. India's massive population, with over 1.4 billion people, results in a vast number of vehicles on the roads. Violations of traffic rules have become commonplace, despite the severe consequences. According to an NDTV report, approximately 400 motorcyclists lose their lives daily in India due to not wearing helmets. Wearing helmets can significantly increase the chances of survival by 42% and reduce injuries by up to 70%. As responsible citizens, it is imperative to adhere to laws and regulations, especially those related to road safety. However, there is room for improvement in the level of seriousness and commitment to these regulations. To better understand the importance of helmets, one can refer to a report by the World Health Organization (WHO) titled "Why are helmets needed?" This report provides a comprehensive explanation of what happens to the head in the event of an accident and how wearing a helmet effectively reduces the impact. In the event of a motorcycle accident, sudden deceleration can eject the rider from the vehicle. If the head collides with an object, its motion comes to an abrupt stop, but the brain continues to move due to its own mass until it impacts the inner part of the skull. Such head injuries can be fatal. Helmets serve as lifesavers in these situations by reducing the deceleration of the skull, almost eliminating the motion of the head. The cushioning inside the helmet absorbs the collision's impact, gradually bringing the head to a halt. Furthermore, it spreads the impact over a larger area, providing essential protection against severe injuries. Most importantly, helmets act as a mechanical barrier between the head and any objects the rider may come into contact with during the accident.

II. LITERATURE SURVEY

In this section, we present a comprehensive review of the existing literature concerning automatic helmet detection and number plate recognition. The examined papers contribute significant insights, methodologies, and advancements within this specific domain. Li et al. (2017) [1] focused on safety helmet-wearing detection based on image processing and machine learning. Their method involved image segmentation, feature extraction, and support vector machines (SVM) for classification. They tested their approach on a dataset of surveillance camera images and achieved accurate helmet detection.

Vishnu et al. (2017) [2] explored the detection of motorcyclists without helmets in videos utilizing a convolutional neural network (CNN) architecture. They constructed and evaluated a CNN model on a large-scale dataset, achieving high accuracy and highlighting the effectiveness of CNNs for helmet detection.

Silva et al. (2014) [3] proposed a method for helmet detection on motorcyclists using image descriptors and classifiers. They employed techniques such as colour histograms, edge orientation histograms, and Haar-like features. The authors evaluated their approach on a dataset of motorcycle images, demonstrating promising results.

Dahiya et al. (2016) [4] addressed the automatic detection of bike riders without helmets using real-time surveillance videos. Their system incorporated background subtraction, foreground segmentation, and head detection based on skin color and face detection. Experimental results showcased effective helmet detection capabilities.

Deng et al. (2009) [5] introduced the ImageNet database, a prominent resource used for training deep learning models in computer vision tasks.

Krizhevsky et al. (2012) [6] pioneered the use of deep convolutional neural networks (CNNs) for image classification, achieving state-of-the-art results on the ImageNet dataset.

Huang et al. (2017) [7] proposed DenseNet, a densely connected CNN architecture that enhances feature reuse and gradient flow, demonstrating superior performance in image classification tasks.

Adrian Rosebrock (2015) [8] discussed basic motion detection and tracking using Python and OpenCV, which laid the foundation for understanding motion-based detection approaches.

Mikolov et al. (2013) [9] introduced word embeddings and the Word2Vec model, which have been influential in representing words as dense vectors in natural language processing tasks, and potentially applicable to text-based aspects of number plate recognition.

LeCun et al. (2015) [10] provided an extensive overview of deep learning, covering its history, principles, and applications.

Rajkomar et al. (2018) [11] presented a scalable and accurate deep-learning approach for electronic health records, demonstrating the potential of deep learning in handling large-scale datasets.

By referencing these papers, we contextualize our research within the existing body of knowledge and draw upon established methodologies and advancements in the field of automatic helmet detection and number plate recognition.

III. PROBLEM STATEMENT

In recent times, there has been a growing trend in utilizing both social networks and road networks as valuable sources of information for detecting various events, particularly concerning road traffic dynamics such as congestion, traffic jams, and accidents. Given this context, there is a pressing need to develop a comprehensive system aimed at effectively addressing the challenges associated with helmet detection and subsequent alerting or fine notification processes. Therefore, our project endeavors to tackle the issue of helmet detection and number plate extraction through the innovative integration of advanced technologies. By leveraging computer vision techniques and machine learning algorithms, we aim to develop a robust system capable of accurately identifying the presence or absence of helmets on riders, as well as extracting pertinent information from vehicle license plates.

The primary objective of this initiative is to enhance road safety and streamline law enforcement efforts by providing real-time insights into helmet compliance among riders and ensuring the visibility of vehicle registration details. By tapping into data sources such as surveillance cameras positioned along roadways and intersections, our system will enable the seamless detection of non-compliant behavior, thereby facilitating prompt intervention and enforcement actions as necessary. Through the deployment of sophisticated algorithms trained on extensive datasets, we aim to achieve a high degree of accuracy and reliability in detecting helmets and extracting alphanumeric characters from license plates across diverse environmental conditions and scenarios. Additionally, our system will offer customizable settings and notification mechanisms, allowing for tailored responses to detected violations and facilitating effective monitoring and enforcement strategies. Overall, by addressing the problem of helmet detection and number plate extraction within the broader context of traffic management and law enforcement, our project seeks to contribute to the creation of safer and more efficient road networks, ultimately benefitting both motorists and the broader community.

IV. METHODOLOGY

For real-time helmet detection, there is a necessity for accuracy and speed. Hence a DNN based model called You Only Look Once (YOLO) was chosen. YOLO may be a state-of-the-art, real-time object detection system. YOLOv3 is extremely fast and accurate and will be an enormous improvement over the previous YOLO versions. It also makes predictions with one network evaluation unlike systems like R-CNN which require thousands for one image. This makes it extremely fast, over 1000x faster than R-CNN and 100x faster than Fast R-CNN. Object detection is the craft of detecting instances of a particular class, like animals, humans and lots of more in an exceedingly large picture or video. The Pre-Existing Object Detection API makes it easy to detect objects by using pre-trained object detection models. But these models detect several Objects which are of no use to us, therefore to detect the mandatory classes a custom object detector becomes necessary. To implement helmet detection and number plate recognition and extraction, 5 objects need to be detected. The objects are- Helmet, No Helmet, Motorbike, Person (sitting on the bike) and vehicle plate. There is a requirement to create a custom object detection model that's capable of detecting these objects. A group of images containing the objects of the classes to be detected are used as a dataset. This dataset is then used to train the custom model. Once the model has been trained, it'll be accustomed to detect these custom objects. The training is completed by feeding all the captured images with their annotations. The model extracts the features of each class from every image with the help of ground truth of the required classes. For extracting the features and storing them to acknowledge those features from other images, we use a deep learning classifier that supports the convolutional neural networks. When an image is given to the current trained model the detection of the pre-trained class is critical. Some images are taken as an example to point to the detection capability of the custom trained model.

V. MODULE DESCRIPTION

To achieve real-time and precise helmet detection, we've adopted the You Only Look Once (YOLO) Deep Neural Network (DNN) approach. YOLO stands as a pioneering real-time object detection system, with YOLOv3 marking a significant leap forward in speed and accuracy compared to its predecessors. YOLO's standout feature lies in its capability to generate predictions in a single network evaluation—a stark contrast to methodologies like R-CNN, which necessitate thousands of evaluations per image. This efficiency translates to YOLO being over 1000 times faster than R-CNN and 100 times faster than Fast R-CNN. Object detection is the process of identifying specific class instances, be it animals, humans, or various objects, within images or videos.

Leveraging a pre-existing Object Detection API streamlines the process through pre-trained models. However, these models often encompass a broad spectrum of objects, including ones unrelated to our objective. Hence, for detecting our specific classes, a custom object detector becomes imperative. In our context, the need extends to identifying five distinct objects: Helmets, No Helmets, Motorbikes, People (sitting on bikes), and Car Plates. Achieving helmet detection and number plate recognition involves developing a custom object detection model tailored to identify these objects. Commencing with the collection of a dataset comprising images containing instances of the targeted classes forms the foundation. This dataset becomes the core of training the custom model. During the training phase, the model is exposed to these images along with their annotated representations. Guided by these ground truth annotations, the model learns to discern unique features from each class. To encode and identify these distinct characteristics in other images, a deep learning classifier, predominantly based on convolutional neural networks (CNNs), is employed. Upon the model's completion of training, it becomes deployable for detecting these custom objects within real-world scenarios. When fed with an image, this trained model proficiently identifies the pre-defined classes. In order to illustrate the model's detection proficiency, a selection of example images showcasing its performance are presented. This meticulous methodology encapsulates the meticulous process of developing a custom object detection model for helmet detection and number plate recognition, underscoring the significance of YOLOv3's efficiency in real-time detection while highlighting the tailored approach necessary to detect specific classes in diverse visual data.

A. Data Collection

The dataset for the model is meticulously curated through a process of random frame extraction from video footage, followed by detailed manual annotation. Random frame selection ensures diversity in the dataset. Each frame was manually annotated, a painstaking task that involved the precise delineation of regions of interest (ROIs) around both helmets and number plates. The dataset encompasses a wide variety of scenarios and lighting conditions to ensure model robustness. Annotators ensured that the annotations were accurate, encompassing a range of helmet types and number plate styles. This carefully assembled dataset serves as the foundation for training a robust and reliable model capable of detecting and recognizing helmets and number plates in real-world environments.

B. Training the Model

Training the YOLOv3 model for helmet and number plate detection is a multifaceted process. Initially, a diverse dataset comprising bike, helmet, and vehicle images, including number plates, undergoes manual annotation. Data preprocessing follows, standardizing and enhancing dataset diversity. YOLOv3's real-time object detection prowess is chosen for its architecture. Leveraging pre-trained YOLOv3 weights from the COCO dataset as a foundation, the model is fine-tuned with a custom dataset emphasizing helmets and number plates. Training iterates data input, weight optimization, and accuracy improvement. Validation and parameter tuning refine performance. Exhaustive testing with unseen data and evaluation metrics ensures accuracy alignment. Once the model meets predefined criteria, the model becomes deployable for real-world use, excelling in identifying and recognizing helmets and number plates.

C. Helmet Detection

The processed model undergoes a conversion to the TensorFlow framework, enabling streamlined operation with a single GPU for enhanced efficiency and performance optimization. Employing the acquired weights from the training phase, the model is leveraged to predict bounding boxes and confidence scores, enabling precise localization. These bounding boxes are derived by selective cropping of frames, adjusting the crop rate dynamically as needed. In cases where the helmet category isn't identified within these cropped frames, the model then shifts its focus to recognize the license plate of the motorcycle. This sequence of operations represents a sequential, adaptive process where the model initially attempts to detect helmets and subsequently switches to license plate detection when helmets are not present in the given frames. This cascading approach ensures a comprehensive analysis of the visual data, aiming to accurately identify both helmets and license plates within the motorcycle imagery. The model's adaptability in prioritizing these distinct objects showcases its versatility and robustness in analyzing diverse visual scenarios related to motorcycle safety and compliance.

D. Vehicle Plate Recognition

To recognize vehicle license plates, the YOLOv3-detected registration code image is cropped and resized to extract a sub-image. An alternative approach involves utilizing a registration code recognition API or employing a deep text recognition Optical Character Recognition (OCR) model known for its high accuracy, approaching 100%. These methods facilitate the segmentation of individual characters and numbers present in the license plate. They analyze and identify characters, combining them to form a complete text string representing the registration code. The accurate recognition of each character within the vehicle plate holds paramount importance, ensuring indisputable identification of potential violators. This process guarantees that penalties or actions taken against offenders are based on precise and reliable information obtained from the license plate, emphasizing the critical role of accurate character recognition in maintaining the integrity and effectiveness of law enforcement systems related to traffic violations and safety.

VI. FUTURE SCOPE

This project will be further enhanced by implementing advanced safety measures, including collision detection testing and capturing images of vehicles exceeding speed limits, as well as capturing images of drivers using phones while driving. This initiative aims to reduce the workload of traffic police at an affordable cost. Once installed correctly and properly maintained, this method will consistently generate databases over an extended period. We utilized PyCharm to develop and successfully implement the program, which underwent successful testing in Python. Additionally, we conducted a study on potential applications and future prospects of the project. Our project can be seamlessly integrated with existing traffic cameras, and with appropriate modifications, it can be utilized to detect helmet usage in real-time. Moreover, we plan to combine the automated registration plate detection algorithm to create a system capable of issuing fines to individuals not wearing helmets.

Conclusion

In conclusion, this project presents a robust framework designed to detect motorcyclists without helmets using CCTV video feeds and automatically retrieve their vehicle license plate numbers. Leveraging Convolutional Neural Networks (CNNs) and the power of transfer learning, the system achieves remarkable accuracy in identifying riders disregarding helmet regulations. Beyond the detection of helmetless riders, the framework extends its functionality to recognize and store the license plate details of their motorcycles. This expanded capability serves as a crucial tool for enforcing helmet laws and ensuring road safety. The comprehensive nature of this approach makes the system an invaluable asset in promoting safer roads. Its ability to not only detect violations but also retain license plate information enables authorities to identify and take action against offenders effectively. The implementation of this framework within existing CCTV networks holds significant promise for enhancing road safety measures. Furthermore, its adaptability, owing to transfer learning techniques, enables seamless deployment across diverse environments, signifying its scalability and applicability in various locations. In essence, this project signifies the potential of artificial intelligence and machine learning in enhancing road safety measures. By offering a comprehensive solution for detecting helmetless riders and cataloging their vehicles through automated means, this framework stands as a potential lifesaver. Its deployment could potentially mitigate accidents and contribute significantly to saving lives on the roads.

References

[1] Li, J., Liu, H., Wang, T., Jiang, M., Wang, S., Li, K., Zhao, X. (2017, February). Safety helmet wearing detection based on image processing and machine learning. In Advanced Computational Intelligence (ICACI), 2017 Ninth International Conference on (pp. 201-205). IEEE. [2] C. Vishnu, D. Singh, C. K. Mohan and S. Babu, “Detection of motorcyclists without helmet in videos using convolutional neural network,” 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, 2017, pp. 3036- 3041. [3] R. R. V. e. Silva, K. R. T. Aires and R. d. M. S. Veras, “Helmet Detection on Motorcyclists Using Image Descriptors and Classifiers,” 2014 27th SIBGRAPI Conference on Graphics, Patterns and Images, Rio de Janeiro, 2014, pp. 141- 148. [4] K. Dahiya, D. Singh and C. K. Mohan “Automatic detection of bike-riders without helmet using surveillance videos in real-time,” 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, 2016, pp. 3046- 3051. [5] J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei, \\\"ImageNet: A Large-Scale Hierarchical Image Database,\\\" in IEEE Conference on Computer Vision and Pattern Recognition, 2009. [6] A. Krizhevsky, I. Sutskever, and G. Hinton, \\\"ImageNet Classification with Deep Convolutional Neural Networks,\\\" in Advances in Neural Information Processing Systems, 2012. [7] G. Huang, Z. Liu, K. Q. Weinberger, and L. van der Maaten, \\\"Densely Connected Convolutional Networks,\\\" in IEEE Conference on Computer Vision and Pattern Recognition, 2017. [8] Adrian Rosebrock, “Basic motion detection and tracking with Python and OpenCV”. https://www.pyimagesearch.com/2015/05/25/basicmotiondetectionandtracking-with-python-and-open. [9] T. Mikolov, K. Chen, G. Corrado, and J. Dean, \\\"Efficient Estimation of Word Representations in Vector Space,\\\" in Proceedings of the International Conference on Learning Representations, 2013 [10] Y. LeCun, Y. Bengio, and G. Hinton, \\\"Deep Learning,\\\" in Nature, 2015. [11] A. Rajkomar, E. Oren, K. Chen, et al., \\\"Scalable and Accurate Deep Learning for Electronic Health Records,\\\" in npj Digital Medicine, 2018.

Copyright

Copyright © 2024 Shruti Singh, Srashti Srivastava, Er. Himanshi Singh. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.