Real-Time Driver Drowsiness Detection Using Images

Abstract

The suggested technology intends to reduce the amount of accidents caused by driver sleepiness and exhaustion, hence in-creasing transportation safety. In recent years, this has been a prevalent cause of accidents. Several expressions and body motions, including exhaustion in the eyes and yawning, are seen as symptoms of sleepiness and fatigue in drivers. These characteristics indicate that the driver\'s condition is poor. The EAR (Eye Aspect Ratio) calculates the distance ratio be-tween horizontal and vertical eye landmarks, which is essential for detecting sleepiness. A YAWN value is calculated for yawn detection utilising the distance between the lower and upper lips, and the distance is compared to a threshold value. We installed an eSpeak module (text to speech synthesiser) to provide suitable voice notifications when the driver becomes tired or yawns. The suggested method is intended to reduce the number of accidents and to contribute to technology in or-der to avoid fatalities caused by road accidents.

Introduction

I. INTRODUCTION

Drowsiness, defined as a condition of drowsiness when one needs to rest, can create symptoms that have a significant influence on work performance, such as reduced response time, occasional lack of consciousness, or microsleeps (blinks lasting more than 500 ms), to mention a few. In fact, chronic weariness can impair performance at levels comparable to those produced by drinking. These symptoms are particularly dangerous when driving since they increase the likelihood of drivers missing road signs or exits, drifting into other lanes, or even wrecking their vehicle and causing an accident. Existing solutions for detecting driver sleepiness are either highly expensive systems that apply to high-end automobile models or systems that are inexpensive but not robust. This research focuses on developing an efficient and cost-effective sleepiness detection system. The approach required in the current circumstance identifies tiredness based on geometric aspects of the eyes and lips. This research aims to accomplish the same goal by constructing a sleepiness detection system to monitor and avoid a negative consequence from tiredness neglect. There are a rising number of incidents on the highways nowadays, and driver tiredness is a major contributor, which has been widely recognised. The actual number of accidents caused by driver sleepiness is difficult to determine since it is frequently overestimated. The change from tiredness to nodding off is delicate and often passes unnoticed by the driver. This explains why it is critical to do more research in this area in order to minimise the occurrence of tiredness-related accidents and urge ourselves to build a driver drowsiness detection system.

A. Background and Related Work

There are two techniques of assessing a driver's sleepiness level, depending on the source of the data utilised for this measurement. On the one hand, there are systems that monitor the vehicle status to determine driver weariness, while on the other hand, there are systems that employ characteristics collected from the driver himself. (a) Vehicle-specific systems The most typical metrics evaluated in works that focus on the investigation of the vehicle state and its relationship to tiredness are steering wheel behaviours or lane deviations [11-13]. Other automotive metrics, such as vehicle position or steering wheel angle, are employed in [14], and data fusion on numerous measurements is used to accomplish a more reliable system. However, even if the driver's declining performance on skill-based activities is a result of sleepiness, it occurs later in the process and cannot be utilised to detect early indications of exhaustion [15]. (b) Driver-centered systems. The survey comprises the current technology and research available on the issue of our study. It is an attempt to have a better understanding of the efforts that have gone into this field of research, as well as to determine where our efforts should be directed when building this project. This research study focused on existing sleepiness detection systems such as facial landmark detection [7], blink detection, and yawn detection. Deep CNN [13], Computer Vision [15], behavioural measurements, and machine learning approaches all have various benefits, problems, and degrees of accuracy when it comes to detecting sleepiness. For blink detection and yawn detection, research has been conducted on EAR and MAR-based systems, respectively.

II. METHODOLOGY

1. OpenCV: OpenCV (Open-Source Computer Vision) is the Swiss Army Knife of Computer Vision. It offers a wide range of modules that may help us with various Computer Vision difficulties, but its architecture is likely the most valuable component of OpenCV. as well as memory management. It provides a framework for you to deal with images and videos as you want, using OpenCV algorithms or your own, without having to worry about allocating and reallocating memory for your images. optimised for use in real-time video and picture processing The author employs OPENCV's highly optimised image processing function for real-time image processing of live video coming from the camera.
2. DLib: Dlib is a contemporary C toolkit with machine learning techniques and tools for creating complicated C ++ applications to tackle real-world issues. It is employed in a broad range of industries and academic settings, including robots, embedded devices, mobile phones, and huge, high-performance computer systems. Lib's open source licences allow you to freely use it in any application.The author implements CNN (Neural Networks) using the free source Dib library. To detect facial characteristics, the author use highly optimised prediction algorithms and detectors of previously learnt face shapes.
3. EAR: EAR (Eye Aspect Ratio) The numerator of this equation calculates the distance between the vertical landmarks of the eye, while the denominator denotes. calculates distance between the horizontal eye reference points, weighting the denominator accordingly since there is only one. The aspect ratio of the eye is roughly constant when the eye is open, but quickly drops to zero when you blink.When the person blinks, the aspect ratio of the eyes drops dramatically and approaches zero. As shown in
   Figure 2, the aspect ratio of the eyes is constant, then quickly drops to zero and then increases again, suggesting that a single blink has occurred.

A. Face Recognition

The following sections describe the face recognition  algorithms Eigenface, Fisherface, Histogram of Local  Binary Pattern and their implementation in OpenCV: Histogram of Local Binary Pattern (LBPH) Local binary patterns were used as classifiers in Computer Vision and 1990 by Li. suggested Wang [4] The combination of LBP with histogram-oriented gradients was introduced in 2009, which improved the performance in certain data sets [5]. For feature coding, the image is divided into cells (4 x 4 pixels) using a surrounding pixel clockwise or counterclockwise. The values are compared with the central ones shown in Figure 6. The intensity or brightness value of each neighbor is compared to the central pixel. Depending on whether the difference is greater or less than 0, the location is assigned a 1 or 0. an 8-bit value for the cell.The advantage of this technique is that even if the brightness of the image.

In Figure , the result will be the same as before. in larger cells to determine the frequency of occurrence of values, which speeds up the process. By analyzing the results in the cell, edges can be identified as the values change. By calculating the values for all cells and concatenating the histograms, feature vectors can be obtained. The input images are classified according to the same procedure and compared with the data set, and the distance is determined. By setting a threshold, you can tell if the face is familiar or unfamiliar. Eigenface and Fisherface calculate the dominant features of the entire training set, while LBPH analyzes them individually.

When the driver's eyes are closed for more than the chosen threshold number of frames or when the motorist yawns [12], the system determines that the driver is weary. From now on, one of these notable situations will occur, and the associated effect will occur. When the face is properly oriented and no wearable barrier is present, the accuracy measured during the performance analysis phase is nearly 100%. When an obstruction (e.g., a hat) is present, accuracy suffers somewhat. Ambient lighting conditions are critical for achieving the best outcomes. If the user's eye closure and yawn occur at the same time, a voice alarm is produced, but the system responds incorrectly and unsynchronizedly. As a result, such a situation should be avoided to prevent any inconsistent results.

IV. LIMITATIONS AND ADVANTAGES

A. Limitations

The model's accuracy suffers if the eye frames are not recorded accurately owing to any form of obstruction (such as goggles or spectacles with reflections). In performing studies, camera activities such as auto adjustments for zoom and rotation are not taken into account. Once the eyeballs have been located, automatically zooming in will assist improve accuracy. When the driver is not facing the camera, the accuracy of detecting eyes and mouth decreases.

B. Advantages

In some cases, detecting drowsiness with OpenCV (Open Source Computer Vision Library) and dlib (a C++ library for machine learning) may be superior to using a CNN (Convolutional Neural Network). The following are some benefits of utilising OpenCV and dlib for sleepiness detection:                                                                                                                                                                  

1. Efficiency: OpenCV and dlib are highly optimised computer vision libraries. They are built to be quick and efficient, making them ideal for real-time applications like sleepiness detection. CNNs, on the other hand, can be computationally costly and may be inefficient for real-time processing, particularly on resource-constrained systems.                                                                                  
2. Face Detection and Landmark Estimation: Pre-trained models and techniques for face detection and facial landmark estimation are provided by OpenCV and dlib. These approaches may be used to precisely find and monitor face characteristics like as the eyes and lips, which are critical for detecting sleepiness. To reach comparable accuracy, CNNs often require a considerable quantity of labelled data and training, which may be time-consuming and resource-intensive.                                                                   
3. Image Processing and feature Extraction: OpenCV and dlib provide a diverse set of image processing and feature extraction methods. These algorithms may be used to extract significant face traits such as eye closure, blink rate, or head posture, which might indicate tiredness. While CNNs may extract features automatically, training them requires a large quantity of labelled data, which may not be easily available for sleepiness detection.                                                                                                                  
4. Real-time Monitoring: To give timely alarms or actions, drowsiness detection frequently necessitates real-time monitoring. The efficient processing capabilities of OpenCV and dlib enable for faster frame rates, enabling real-time monitoring of drowsiness-related stimuli. CNNs, on the other hand, may create delay as a result of their sophisticated computations, limiting real-time responsiveness
5. Portability: Because OpenCV and dlib are widely supported and have bindings for a variety of programming languages, they are easily transferable between systems and devices. Because of its portability, sleepiness detection may be easily integrated into current systems. While CNN models are successful, they may necessitate more work for deployment and optimisation on various systems.                                                                                                                                                                                       

It is vital to note that the option between utilising OpenCV, dlib, or CNNs for sleepiness detection is determined by the application's unique needs, available resources, and desired accuracy. CNNs shine when there is a lot of labelled data and you need to extract complicated features, but OpenCV and dlib are efficient and effective alternatives for many real-time sleepiness detection cases.

Conclusion

By monitoring the eyes and lips, the model can identify tiredness. To recognise essential characteristics on the face, shape prediction algorithms [16] are utilised. These algorithms\' inputs are face landmarks gathered by facial landmark detection. This module is concerned with the EAR function, which computes the distance ratio between horizontal and vertical eye landmarks. An eSpeak module (text to speech synthesiser) is also installed to provide suitable voice notifications when the driver becomes fatigued or yawns. The entire initiative is intended to reduce the number of accidents and to contribute to technology with the objective of preventing fatalities caused by road accidents. This paper\'s future work can be focused on the use of outer factors for measuring fatigue and drowsiness. Weather conditions, vehicle condition, sleeping time, and me-chanical data are examples of external influences. Driver sleepiness is one of the most serious risks to road safety, and it is especially severe for commercial motor vehicle operators. Twenty-four-hour services, variable environmental conditions, high annual mileage, and an increase in demanding work schedules all contribute to this major safety hazard. One crucial step towards resolving this problem is to continually monitor the driver\'s sleepiness [17] and provide information about their condition to the driver so that they may take appropriate action. Currently, no adjustments to the zoom or camera orientation can be made while the system is running. In the future, more work can be done to automate the zoom on the eyes after they are localized.

References

[1] Rahul Atul Bhope, “Computer Vision based drowsiness detection for motorized vehicles with Web Push Notifications”, IEEE 4th International Conference on Internet of Things, IEEE, Ghaziabad, India, 2019. [2] Jasper S. Wijnands, Jason Thompson, Kerry A. Nice, Gideon D. P, Aschwanden & Mark Stevenson, “Real-time monitoring of driver drowsiness on mobile platforms using 3D neural networks”, Neural Computing and Applications, 2019. [3] Chris Schwarz, John Gaspar, Thomas Miller & Reza Yousefian, “The detection of drowsiness using a driver monitoring system” , in Journal of Traffic Injury Prevention (Taylor and Francis Online), 2019. [4] Aditya Ranjan, Karan Vyas, Sujay Ghadge, Siddharth Patel, Suvarna [5] Sanjay Pawar, “Driver Drowsiness Detection System Using Computer Vision.”, in International Research Journal of Engineering and Technology(IRJET), 2020. [6] B.Mohana, C.M.Sheela Rani, “Drowsiness Detection Based on Eye Closure and Yawning Detection”, in International Research Journal of Engineering and Technology(IRJET), 2019. [7] Driver Alert Control (DAC). (2016, Feb 10). Retrieved from http://support.volvocars.com/uk/cars/Pages/owners-manual. aspx?mc=Y555&my=2015&sw=14w20&article=2e82f6fc0 d1139c2c0a801e800329d4e [8] Z. Mardi, S. N. Ashtiani, and M. Mikaili, “EEG-based drowsiness detection for safe driving using chaotic features and statistical tests,” Journal of Medical Signals and Sensors, vol. 1, pp. 130–137, 2011. [9] T. Danisman, I.M. Bilasco, C. Djeraba and N. Ihaddadene, “Drowsy driver detection system using eye blink patterns,” Universite Lille 1 & Telecom Lille 1, Marconi, France, 2010. [10] A Study of Heart Rate and Brain System Complexity and Their Interaction in Sleep- Deprived Subjects. Kokonozi A.K., Michail E.M., Chouvarda I.C., Mag-laveras N.M. Bologna, Italy. : Computers in Cardiology, 2008. [11] Effects of partial and total sleep deprivation on driving performance. Peters R.D., Wagner E., Alicandri E., Fox J.E., Thomas M.L., Thorne D.R., Sing H.C., Balwinski S.M. 1999. [12] Effect of driving duration and partial sleep deprivation on subsequent alertness and performance of car drivers. Otmani S, Pebayle T, Roge J, Muzet A. 2005. [13] TemplateMatching. [Online] Apr 21, 2014 [Cited: Sep 8, 2014.] Real time drowsy driver detection using haarcascade samples. Dr.Suryaprasad J, Sandesh D, Saraswathi V. [14] Integrated Approach for Nonintrusive Detection of Driver Drowsiness. Yu, Xun. Duluth : s.n., 2012. [15] Jay D. Fuletra., “A Survey on Driver’s Drowsiness Detection Techniques” International Journal on Recent and Innovation Trends in Computing and Commu-nication ISSN: 2321-8169 Volume: 1 Issue: 11 ,2013. [16] Rajneesh, “Real Time Drivers Drowsiness Detection and alert System by Measuring EAR,” International Journal of Computer Applications (0975 – 8887) Volume 181 – No. 25, November- 2018 .

Copyright

Copyright © 2023 Prof. Poonam Jadhavar, Parth Barahate, Samruddhi Chaudhari, Gaurang Keskar, Aditya Nene. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.