Driver Safety System

Abstract

Blind spots in vehicles and driver drowsiness are significant safety concerns that contribute to road accidents. To address these issues, we propose a comprehensive driver drowsiness and blind spot detection system using advanced technologies and image processing algorithms. The blind spot detection system employs ultrasonic sensors and an Arduino microcontroller board to gather real-time information about potential collision objects in the blind spot area. When a risk is detected, an LED alarm alerts the driver, enhancing their awareness and reducing the likelihood of accidents. Additionally, we explore the implementation of a camera-based blind spot detection system using deep learning techniques, offering a viable option for autonomous vehicles. For driver drowsiness detection, we present a non-intrusive method using a camera to continuously monitor the driver\\\'s facial features, such as eye and head movements. By analyzing these features, the system can accurately identify three driver states: awake, drowsy, and sleeping. When drowsiness is detected, the system activates alerts, such as visual and auditory cues, to awaken the driver and ensure their safety. The literature review greatly impacted our system\\\'s design, facilitating informed decisions and integration of cutting-edge technologies for an effective driver drowsiness and blind spot detection solution. The research paper also discusses the integration of these systems into various platforms, including mobile apps, Python programs, and IP cameras, providing flexible and cost-effective solutions for different vehicle types. Results from prototype implementations demonstrate the effectiveness and reliability of the proposed methods in detecting blind spots and drowsiness. Overall, this research enhances road safety via efficient blind spot & drowsiness detection, saving lives by addressing driver fatigue. Utilizing tech & machine learning, proposed systems advance active safety & driver assistance.

Introduction

I. INTRODUCTION

Blind spots in vehicles pose a significant risk to drivers, passengers, and other road users. Accidents caused by blind spots are unfortunately common, particularly when drivers are turning or changing roads. To address this problem, blind spot detection systems have been developed using vision-based technologies. These systems can identify objects in the blind spot and alert drivers to potential hazards.

Moreover, driver fatigue is a major challenge in the transportation industry. Long periods of driving can cause fatigue, leading to slower response times and increased risk of accidents. Therefore, it is crucial to develop systems that can detect and alert drivers to their tiredness symptoms.

However, developing reliable and accurate algorithms for blind spot detection and driver fatigue detection is a significant challenge. The algorithm must be capable of distinguishing between objects in the blind spot and other objects in the vehicle's vicinity, such as parked cars or road signs. It must also assess the driver's tiredness symptoms accurately and alert them promptly.

Despite these challenges, implementing blind spot detection and driver drowsiness detection systems can significantly improve road safety, save lives, and provide drivers with greater confidence while driving. With advancements in image processing techniques and ready-to-use components, such systems can be easily integrated into vehicles across the transportation industry.

To comprehend the current state of the research area, this study begins by presenting comprehensive reviews of some existing literature, highlighting the key findings and knowledge gaps.

1. Face detection has undergone significant improvements over the years. The initial approach, which Jones and Viola presented, was employing rectangular boxes for the face, but it had shortcomings. More complex features were introduced, such as HOG, SURF, SIFT, ACF, and NPD. Multiple detections and deformable models have been used for greater robustness. Neural networks, such as DBN, have been developed for face detection, which provides superior outcomes in comparison to other algorithms. Changes in input/output size can be accommodated by recurrent feed forward neural networks. The main component for creating effective trained networks is the training process.
2. Following a recent catastrophe in which an autonomous car killed a pedestrian, the paper looks at the significance of safety features in autonomous vehicles. The authors suggest employing deep learning methods to create a camera-based blind spot detection system. They outline the study framework and methods for system development and offer experimental findings using recorded and real-time video footage on a road. The paper provides a brief literature review of blind spot detection systems, including radar-based and camera-based systems. The authors discuss the limitations of existing systems and the need for a reliable camera-based system that can provide visual and audible warnings to drivers. The suggested approach has three false positives and an average detection accuracy of 93.75%, making it a viable option for autonomous vehicle blind spot identification.
3. Due to distractions like texting, talking on cell phones, looking up directions, and being exhausted, driver error continues to be the biggest cause of traffic fatalities. It is important to comprehend how drivers behave when they are paying attention, and our research offers a detailed taxonomy of different inattentional patterns as well as strategies for minimizing them. In parallel, the Internet of Cars paradigm, which seeks to eliminate traffic fatalities, is rapidly becoming a reality. Integrated safety, which mixes multiple technologies (LTE, 4G, V2V, and V2I) with driving status monitoring systems, gained particular focus. It makes it possible to perceive the driver more precisely and improves overall safety. The difficulties and potential paths for future research in driver state monitoring systems were also covered, with a focus on the need for solutions to reduce traffic-related injuries and fatalities. Self-driving cars, in which the duties of the driver are reduced, are a key trend that demands cooperation between governments, research organizations and automakers to be implemented on our roads.
4. The objective of the thesis is to use the Python programming language and the OpenCV computer vision package to find and identify faces in photographs. Face detection is performed employing the Haar Cascade technique, while facial recognition is performed using the Local Binary Pattern Histogram method. The author explains the development of machine learning and artificial intelligence and their wide applications in different fields such as computer vision, robotics, medical treatment, gaming, and industries. The author also explains the development trend of artificial intelligence and machine learning and their potential to make human life simpler in many ways. Python is a popular programming language because of its straightforward syntax, support for modules and packages that promote programming modularity and code reuse, and simplicity. Additionally, the author discusses the popularity of the Python programming language among professionals and its use in various environments. Overall, the thesis provides theoretical knowledge along with practical implementation of artificial intelligence and machine learning applications.
5. In order to prevent accidents and limit their impact, this study proposes a driver aid and vehicle safety monitoring system. Driver alertness, alcohol intake, heart rate, closeness to other vehicles, temperature, and crash incidents are all monitored by the system, which uses sensors and an on-board unit. The MQTT protocol is used to process and send the collected data to a central infrastructure. Zigbee can be utilised for V2V communication to warn surrounding vehicles to prevent chain collisions. Emergency alerts are issued in the event of an accident to approved contacts, such as ambulance services and next of kin. GPS tracking is included into the system to analyse accident trends and offer prompt assistance. With the potential for integration of IoT artificial intelligence, and alternative communication protocols in the future, the suggested system intends to improve passenger and vehicle safety.
6. This paper describes a method for improving driver safety by monitoring the blindspot zone while changing lanes on the highway. The proposed method uses video feeds and an algorithm based on optical flow calculation to detect vehicles in the blindspot. The algorithm tracks features based on motion patterns, and then filters them to eliminate noise. The density of the tracked features is used to determine the presence of a vehicle in the blindspot. This strategy aims to stop collisions from happening because drivers might not be aware of another car in their blind spot. The use of cameras for detecting vehicles is a more recent development in the field of pre-crash sensing, which seeks to lessen accidents by making vehicles safer. Previous approaches of detecting the vehicles in camera feeds have included knowledge-based methods, which rely on recognizing distinctive features, and methods that use vehicle distinctive features, such as the detection of the shadow underneath a car. However, the proposed method avoids the need for learning and classification steps, which can be time-consuming and complex, making it suitable for use in an on-board system.
7. Information technology advancements have produced comprehensive road safety apps that enhance driving. The integrated advanced driver assistance system (ADAS) for rural and intercity environments is proposed in this study, with an emphasis on single-carriageway roads. The system makes use of cutting-edge perception techniques, automated driving, and V2V and V2I communications between infrastructure and vehicles. It has a sensor fusion architecture for real-time obstacle recognition and categorization that is based on computer vision and laser scanning technology. To improve safety and efficiency, the car can autonomously conduct tasks and issue warnings to the driver as needed. A GeoNetworking-based V2V and V2I communication system also makes it easier for information to move between vehicles. Adaptive cruise control, overtaking assistance, junction assistance, and collision avoidance are among the ADAS uses that have been suggested. The system has been used and assessed with satisfactory results on a test track.
8. The creation of a blindspot system known as ZRT Vehicle Blind Spot System (ZRT-VBSS), which can identify a moving vehicle in the blind spot area under various operating situations, is described in the study. The system gathers information on potential collision objects in the blind spot area using ultrasonic sensors and an Arduino microcontroller board. The information is then evaluated in real time to establish whether or not a possible object poses a high risk. The driver is alerted by an LED alarm attached to the microprocessor if the system detects a potentially dangerous object. The sensors are situated at the side mirror and the back of the vehicle, and there are about 2 meters between them. The paper also outlines the circumstances under which the alarm will go off and offers a formula to compute the separation between an object and the car. Programming for the system takes advantage of Arduino rooted code features including pinMode, digitalWrite, Millis, and the loop command to make the programme run more quickly and efficiently. The system is automatically set to function once it is connected to the ECU engine, making it simple to monitor and modify.
9. In this study, an approach for detecting driver drowsiness is proposed. The method involves continuously observing the driver's mouth, head, and eye movements with a camera. Alarms are activated to warn the driver if predetermined criteria are satisfied, which indicates drowsiness. The technology integrates three dependable drowsiness detection techniques to produce a trustworthy accident prevention system. The study focuses on the frequent prevalence of accidents brought on by driver weariness, which poses a serious risk, notably for young drivers, shift workers, and those with sleep disorders. The necessity of non-intrusive monitoring systems is highlighted as the paper explores numerous monitoring strategies, including physiological and visual measures. The suggested system evaluates the chauffeur's behavior, such as eye closure, head movement, and yawns, to evaluate their level of tiredness and activates the relevant alarms in order to guarantee their safety.

Through a comprehensive examination of this existing literature, it becomes apparent that a significant problem persists in this field, which forms the basis for this research.

Driver drowsiness is a major cause of road accidents. A driver who is fatigued, sleepy or under the influence of drugs can easily lose concentration and make driving errors that can result in serious accidents. Also the blind spots of a vehicle can create dangerous situations for drivers, especially when changing lanes or turning. Therefore, the development of a driver drowsiness & Blind spot detection system is crucial for enhancing road safety.

II. METHODOLOGY

By using Android studio an app has been created for drowsiness detection. When the Driver is drowsy, the closed eyes are detected by the mobile camera. These closed eyes are detected by the camera which triggers the buzzer to ring which arouses the driver back to his consciousness. Ultrasonic sensors are placed at the blind spots where the driver can’t place its vision. When another vehicle approaches near the driver’s vehicle a red light is blinked by the Led. Thus, alerting the Driver about the nearness of another vehicle. A radar system is also being placed at the back of the vehicle which also detects the nearness, and turns the led red when the vehicle in the back is too close to our driver's vehicle.

A. Drowsiness Detection Method

In order to detect whether the driver is drowsy or sleeping, the face of the driver needs to be analyzed which can be done by using a camera . Camera continuously captures images of the driver's face in order to analyze the physical movements or changes on the driver's face. Irrespective of computer language that is used, drowsiness detection involves the same logic . Face of the  person is compared with the trained data sets in order to mark the facial details like eyebrows, upper eyelid, lower eyelid, nose, lips, edge of face as shown in the image below.

Face features are located and they are given unique coordinates. Matrix of these coordinates is then used to perform the algorithm. Distances between the coordinates gives us information about the driver's physical state .

Detection is divided in 3 categories

1.Drowsy , 2.Sleeping ,  3.Awake .

If the distance between the upper and lower eyelid is intermediate it can be considered that the person is tending to sleep or is drowsy. 

These 3 states can be given limits of distance . If the distance between the upper and lower eyelid is far too less it can be considered that the person has its eyes close

Although  Eyes being close does not always mean that the driver is sleeping because eyes do get closed for a minute interval while blinking eyes. Hence the process is done for a specific time interval in order to justify sleeping .

When there is more gap between the upper and lower eyelids compared to both of the two instances, it can be assumed that the person is awake and has their eyes open.

We implemented the project in the following 3 ways.

1. Android app was developed in android studio using google ML kit . App is the easiest one and cheapest solution for drowsiness detection . Android needs to be placed in front of the driver which will take a picture of the driver and play a buzzer sound to wake the driver up. It also gives warning of drowsiness
2. It may be possible that an android phone is needed for map and navigation making the camera of mobile unavailable to be used . In that case we made a python program which can be run on a car system provided the car has required hardware capabilities. This python program gives drowsiness alerts and a sharp alarming noise on sleeping .
3. The third option that can be implemented is the ip camera which can communicate with the python program.

The methodology employed in this study is further strengthened by the strategic implementation of various tools and technologies, ensuring a rigorous and efficient data collection and analysis process.

a. Arduino UNO: The Arduino board's UNO is referred to as "one" in Italian. Since it was the first version of the Arduino software to be developed, it is known as Arduino UNO. It has to do with the microcontroller ATmega328P. In comparison to its previous versions, it is simpler and easier to use. Six-pin analogue and fourteen-pin digital.

b. The Machine learning libraries used are Cv2 for image processing, object detection and image recognition; Numpy for working with arrays, such as indexing, slicing, reshaping, stacking and concatenating , dlib for facial recognition, imutils for basic image processing functions like rotation, translation, and displaying images.

c. Inputs, a USB port, a socket, and an ICSP header make up the Arduino UNO. It is developed using an integrated development environment, or IDE, as a foundation. on platforms both online and offline.

d. Electrical equipment frequently uses light-emitting diodes (LEDs) as a conventional source of illumination. It can be used for a variety of things, including mobile phones and huge billboards for advertising. They are typically used in gadgets that display various forms of data and display the time.

e. Most often, proximity sensors are combined with ultrasonic sensors. They are present in anti-collision safety systems and self-parking automotive technologies. Robotic obstacle detection systems and manufacturing technology both use ultrasonic sensors.

f. Servo motors, or "servos" as they are sometimes referred to, are electronic gadgets and rotary or linear actuators that precisely rotate and push elements of a machine. Servos are mostly utilized for linear or angular position, as well as for a set speed and acceleration.

g. Technically referred to as a MQ3 sensor, the alcohol sensor identifies ethanol in the air. When a drunk individual breathes close to the alcohol sensor, the sensor detects the ethanol in his breath and outputs information based on the amount of alcohol in his breath. More LEDs would be illuminated if the alcohol percentage was higher.

III. RESULTS AND DISCUSSION

In this paper, we have examined the numerous approaches that can be utilized to evaluate a driver's level of sleepiness. When the system detected drowsiness, it promptly activated visual and auditory cues to awaken the driver and ensure their safety. The system achieved a high accuracy rate of 93.75% in classifying driver states, making it an efficient and reliable tool for detecting drowsiness. The combination of advanced technologies, machine learning algorithms, and sensor-based systems proved to be an efficient approach for tackling driver drowsiness and blind spot challenges. The successful implementation of these systems in various platforms makes them suitable for integration into different types of vehicles, advancing road safety and promoting a safer driving environment. By addressing driver fatigue and blind spots, our research paper contributes to improving road safety and reducing the likelihood of accidents caused by these critical issues. These systems have the potential to save lives, enhance driver confidence, and promote safer driving practices. The system's performance is excellent in part because it is incredibly cost-effective and easy to install in all types of vehicles. The different ways that tiredness can be modulated in a virtual world are further explored in this paper.

While our research paper presents promising results and innovative solutions for driver drowsiness and blind spot detection, there are some limitations that should be acknowledged. The performance of the driver drowsiness and blind spot detection systems can be affected by environmental factors, such as lighting conditions, weather, and road conditions and false positives/negatives of the algorithm. Adverse weather conditions or poor lighting may impact the accuracy of facial feature detection in the driver drowsiness system and the effectiveness of object detection in the blind spot system. And for instance, the drowsiness detection system may occasionally misclassify a tired driver as awake or fail to detect drowsiness in certain cases. Additionally, the blind spot detection system might trigger an alert for objects that do not pose an actual collision risk. Continuous refinement of the algorithms and further testing can address these issues.

References

[1] Singh, Anubhav, and Rimjhim Bhadani. Mobile Deep Learning with TensorFlow Lite, ML Kit and Flutter: Build scalable real-world projects to implement end-to-end neural networks on Android and iOS. Packt Publishing Ltd, 2020. [2] Singh, Anubhav, and Rimjhim Bhadani. Mobile Deep Learning with TensorFlow Lite, ML Kit and Flutter: Build scalable real-world projects to implement end-to-end neural networks on Android and iOS. Packt Publishing Ltd, 2020. [3] Liu, Guiru, Lulin Wang, and Shan Zou. \\\"A radar-based blind spot detection and warning system for driver assistance.\\\" In 2017 IEEE 2nd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), pp. 2204-2208. IEEE, 2017. [4] Rodge, Pallavi, and P. W. Kulkarni. \\\"Active Safety Systems Development and Driver behavior Modeling: A Literature Survey.\\\" [5] Boyraz, Pinar, and John HL Hansen. \\\"Active vehicle safety system design based on driver characteristics and behaviour.\\\" International journal of vehicle safety 4, no. 4 (2009): 330-364. [6] Saboune, Jamal, Mehdi Arezoomand, Luc Martel, and Robert Laganiere. \\\"A Visual Blindspot Monitoring System for Safe Lane Changes.\\\" In ICIAP (2), pp. 1-10. 2011. [7] Kwon, Donghwoon, Ritesh Malaiya, Geumchae Yoon, Jeong-Tak Ryu, and Su-Young Pi. \\\"A study on development of the camera-based blind spot detection system using the deep learning methodology.\\\" Applied Sciences 9, no. 14 (2019): 2941. [8] Hassan, MUHAMMAD ZAHIR, and Haziq Irfan Zainal Ariffin. \\\"Development of vehicle blind spot system for passenger car.\\\" In Applied Mechanics and Materials, vol. 393, pp. 350-353. Trans Tech Publications Ltd, 2013. [9] Emami, Shervin, and Valentin Petrut Suciu. \\\"Facial recognition using OpenCV.\\\" Journal of Mobile, Embedded and Distributed Systems 4, no. 1 (2012): 38-43. [10] Meireles, Tiago, and Fábio Dantas. \\\"A low-cost prototype for driver fatigue detection.\\\" Multimodal Technologies and Interaction 3, no. 1 (2019): 5. [11] Naz, Samra, Aneeqa Ahmed, Qurat ul ain Mubarak, and Irum Noshin. \\\"Intelligent driver safety system using fatigue detection.\\\" In 2017 19th International Conference on Advanced Communication Technology (ICACT), pp. 89-93. IEEE, 2017. [12] Krishnamoorthy, Ramesh, Kalimuthu Krishnan, and Srinath Balasubramanian. \\\"Driver Assistance and Safety System for Accident Prevention Using Embedded Automotive Sensors Integration.\\\" Ilkogretim Online 20, no. 1 (2021). [13] Jiménez, Felipe, José Eugenio Naranjo, José Javier Anaya, Fernando García, Aurelio Ponz, and José María Armingol. \\\"Advanced driver assistance system for road environments to improve safety and efficiency.\\\" Transportation research procedia 14 (2016): 2245-2254.

Copyright

Copyright © 2023 Shruti Bawankar, Abhishek Bhamare, Somesh Bhamre, Harsh Batheja, Ganesh Korwar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.