Driver Drowsiness Detection Using Deep Learning

Abstract

The main reason for accidents worldwide is the sleepiness experienced by individuals operating vehicles. Fatigue and drowsiness are prevalent among numerous drivers due to insufficient rest and tiredness. The most effective approach to preventing road accidents resulting from drowsiness is to provide advance warnings to the driver. There are plentiful techniques to detect drowsiness. The project, driver drowsiness detection using deep learning is implemented using a convolution neural network(CNN), an approach of deep learning. In this project first, we capture a person’s face with the help of a web camera after which with the help of the deep learning algorithm we predict if the person is drowsy or not. If the person is detected as drowsy it immediately rings an alarm to alert the person along with a warning voice message. The prime goal of this project is to alert drowsy people before the cause of accidents which helps us in preventing the cause of road due to drowsiness of the person.

Introduction

I. INTRODUCTION

Accidents are the major cause of death in which around 1.3 million people die every year. The majority of these accidents are caused because of distraction or the drowsiness of the driver. Every day and night, a large multitude of individuals embark on long-distance drives along the highway. Drowsiness can unexpectedly manifest in situations of stress and fatigue, often at inconvenient times. It can be triggered by various factors such as sleep disorders, certain medications, boredom situations, or even monotonous situations like extended periods of driving. In this way, drowsiness produces dangerous situations and increases the probability of an accident. In this context, it is important  to  use  new  technologies  to  design and build systems that will monitor drivers and measure their level of attention throughout the driving process. Driver drowsiness detection projects will be very helpful to overcome such deadly accidents. This project is built based on a deep- learning approach. In this project, a camera is used to record the user’s visual characteristics such as eyes closed, eyes open, and yawning. By employing face detection and CNN (Convolutional Neural Network) methods, we aim to identify signs of driver drowsiness. If the system detects drowsiness in an individual, it will activate an alarm accompanied by an alert message. So that the driver will get cautious and take preventive measures. Driver drowsiness detection contributes to the decrease in the number of deaths occurring in traffic accidents.

II. LITERATURE SURVEY

A. Driver fatigue detection system

The primary objective of the research is to identify when drivers become drowsy while their vehicle is in motion. This detection process involves continuously monitoring various signs of drowsiness, taking into account both physiological and physical indicators. Physiological aspects encompass factors like core body temperature and pulse rate, both of which typically decrease during the onset of drowsiness. These physiological markers are tracked using somatic sensors. In addition, physical cues such as yawning, drooping eyelids, closed eyes, and prolonged blink durations are considered. By combining these factors, the system detects drowsiness in the driver and alerts them through multiple stages, depending on the severity of the symptoms. Moreover, the system gradually adapts to the unique characteristics of the individual driver over time, minimizing the occurrence of false alarms.

B. Detecting Driver Drowsiness Based on Sensors

In recent times, driver fatigue has emerged as a significant cause of road accidents, resulting in severe physical injuries, fatalities, and substantial economic losses. This underscores the urgent need for a dependable system capable of detecting driver drowsiness and issuing timely alerts to prevent mishaps. Researchers have explored various measures to assess driver drowsiness, including vehicle-based, behavioral, and physio- logical measures. This paper provides a comprehensive review of these measures, focusing on the sensors employed, as well as discussing the advantages and limitations associated with each approach. Furthermore, the paper examines the different experimental methods employed to induce drowsiness.

The findings suggest that developing a hybrid drowsiness detection system, combining non-intrusive physiological measures with other approaches, would yield an accurate assessment of a driver’s drowsiness level. Such a system could potentially mitigate a significant number of accidents by promptly alerting drowsy drivers.

C. Driver drowsiness detection using ANN image processing

This paper presents the potential development of a drowsi- ness detection system for car drivers, utilizing three different methods: EEG and EOG signal processing, as well as driver image analysis. Previous studies have focused on the first two methods, while this paper introduces the possibility of detecting the drowsy or alert state of the driver based on analyzing images captured during driving and examining the condition of the driver’s eyes (opened, half-opened, or closed). To achieve this, two types of artificial neural networks were employed: a one hidden layer network and an autoencoder network

D. Driver alertness detection system based on the microcon- troller

Developing an embedded system capable of detecting and preventing driver drowsiness is a significant challenge in the field of road traffic accident prevention. To address this issue and enhance safety while driving, it is crucial to have an alert system that can identify a decline in driver concentration and promptly alert the driver. Studies have indicated that accidents often occur due to driver distraction. In this paper, the author proposes and reviews several detection systems designed to monitor driver concentration levels. The proposed solution is a portable Driver Alertness Detection System (DADS) that utilizes a pixelated coloration detection technique based on facial recognition. A portable camera is positioned on the front visor to capture facial expressions and eye movements. The effectiveness of DADS is evaluated with 26 participants, yielding a 100

E. Driver Drowsiness Detection Techniques

In recent years, driver drowsiness and fatigue have emerged as significant contributors to road accidents, particularly during long drives on highways. Preventing accidents and ensuring driver safety have become key objectives for smart systems. To achieve this, a robust driver detection system is necessary to alert drivers. This paper provides a comprehensive survey of the existing literature on various techniques employed to detect driver drowsiness. These techniques include physical- based methods that analyze features such as eye state (open or closed), blink rate, yawning, and head movements. Phys- iologically based methods utilize signals like EEG, ECG, PPG, Heart Rate Variability, EOG, and EMG to assess the level of driver drowsiness. Vehicular-based techniques involve monitoring and controlling the vehicle through measurements such as steering wheel movement (SWM) and the standard deviation of lane position (SDLP). Lastly, hybrid techniques combine multiple approaches to detect driver drowsiness. The paper aims to highlight the advantages, limitations, unresolved issues, and challenges associated with these methods.

III. METHODOLOGY

The methodology of this project works in this way where we take 3 features - left eye, right eye and face where it checks whether the eyes are closed or open and also checks whether the person is yawning or not. For this Python project, our approach involves utilizing OpenCV to capture images from a webcam and passing them into a Deep Learning model. The model’s purpose is to classify whether the person’s eyes are ’Open’ or ’Closed’. We constructed the model using Keras and implemented Convolutional Neural Networks (CNN).

CNNs are highly effective for image classification tasks, as they consist of input and output layers along with hidden layers, which can be multiple in number. These layers undergo a convolution operation using a filter that performs 2D matrix multiplication on both the layer and the filter. For the detection of these features, we have used the Haar Cascade Classifier. The Haar Cascade Classifier is a classifier that detects objects or faces irrespective of the scale. The Haar Cascade is taken in the form of XML files where 3 different files are used for the detection.

Where one file is for the left eye, one is for the right eye and one file is for the face. We also have a variable “Score” which is used to count the time the eyes are closed where ‘1’ point is equal to one second. So, when the score is greater than or equal to 10 which means that the eyes are closed for 10 seconds then the alarm beeps along with a voice message “Wake up, don’t sleep!”. The camera holds utmost significance in this project as it fulfills a crucial role in extracting the face and eyes, which is a fundamental process.

The dataset used is a directory which consists of multiple images of closed eyes, open eyes as well as images of yawning and not yawning. These images are been trained into the model which is then used for prediction.

V. DISCUSSIONS

From the results obtained, we can see that when the eyes are open the score just keeps on increasing without any ringing of alarm. Once the eyes are closed and the score is greater than 10 which means 10 seconds it immediately rings an alarm along with an alerting message, and the colour of the box changes to red indicating that the person is drowsy. The person gets detected as drowsy when his eyes are closed and the score is greater than 10. We can see that the model is working pretty much accurately, but is not totally ready for being used in a real car.

We still need to improve the model in such a way that we reduce its computational cost, as we found that the model sometimes might not work properly, so improvement in that aspect is required. Apart from that one more thing which we even found is that the model works accurately only when there is enough light in the surroundings to capture the video and detect the person. When tried in darker conditions we have found that the model is not able to capture the person in the video and detect whether he is drowsy or not. We were able to see that even when the person’s eyes were open the model was detecting that his eyes are closed and started giving the alert signals. Also in previous projects they have used different techniques such as detecting the drowsiness based using sensors where the sensors are measured using behavior, vehicle and physio- logical. We also have another project of drowsiness detection where they have built the project using ANN image processing. In that project signal processing of ECG and EOC of drivers during driving are the key components. Microcontroller based driving alertness detection system is one of the projects to detect the drowsiness of the driver, where they have used multiple detection systems and proposed a portable Driver Alertness Detection System (DADS). Comparing with the previous projects we have seen that the previous projects are taking 20 seconds of time to detect whether the person is drowsy or not, which according to us is a pretty large time to detect and alert the driver, so we reduced the time from 20 seconds to 10 seconds decreasing it by a total of 10 seconds. Which according to us is much more helpful and a much faster way of detecting and alerting the driver from sleeping and reduce the risk of an accident

We have also seen that in the previous projects there was only an alarm which beeps when the model has detected that the person is drowsy which again, we felt that it might not be sufficient, so we have added an audio “Wake up! Don’t sleep” which plays along with the beep until the driver comes out of his drowsiness. We found the model being more useful and efficient after adding this feature.

VI. ACKNOWLEDGEMENT

The resources and assistance provided by Malla Reddy University through the conduct of this study are acknowledged with sincere gratitude by the writers.

Conclusion

From this project, we have tried to lower the risk of road accidents especially while driving which is caused due to drowsiness. For this, we have trained the model in such a way that it keeps on learning with every record and keeps on improving itself. There were earlier projects which used different methods like sensors, and ECG, which were based on different approaches. In this project, we have used another method of capturing the input by using the cameras. Our goal in this project is to use deep learning techniques such as open CV and keras to help users reduce the risk of accidents. We have used the Haar Cascade algorithm which consists of different XML files which are used to scan the inputs such as the left eye, right eye, and the face. We have learned through the previous projects that the main difficulty for this project is the accuracy and also the lack of precision which is there for scanning the face. So, we have tried to improve this particular thing in our project by using machine learning and deep learning algorithms. Our main aim for the future is to further improve the accuracy of the project and also make it useful for various different problems in which this algorithm and model can be used.

References

[1] Adenin, Hasibah, Rahimi Zahari, and Tiong Hoo Lim. ”Microcontroller based driver alertness detection systems to detect drowsiness.” In Ninth International Conference on Graphic and Image Processing (ICGIP 2017), vol. 10615, pp. 197-201. SPIE, 2018. [2] Fuletra, Jay D., and Dulari Bosamiya. ”A survey on drivers drowsiness detection tech- niques.” International Journal on Recent and Innovation Trends in Computing and Communication 1, no. 11 (2013): 816-819. [3] Maga´n, Elena, M. Paz Sesmero, Juan Manuel Alonso- Weber, and Araceli Sanchis. ”Driver drowsiness detection by applying deep learning techniques to sequences of images.” Applied Sciences 12, no. 3 (2022): 1145 [4] Mittal, Ajay, Kanika Kumar, Sarina Dhamija, and Manvjeet Kaur. ”Head movement-based driver drowsiness detection: A review of state-of-art techniques.” In 2016 IEEE international conference on engineering and technology (ICETECH), pp. 903-908. IEEE, 2016 [5] Niloy, Amit Raha, Atiqul Islam Chowdhury, and Nusrat Sharmin. ”A brief review on different Driver’s drowsiness detection techniques.” International Journal of Im- age, Graphics and Signal Processing 10, no. 3 (2020): 41. [6] Phan, Anh-Cang, Ngoc-Hoang-Quyen Nguyen, Thanh-Ngoan Trieu, and Thuong- Cang Phan. ”An efficient approach for detecting driver drowsiness based on deep learning.” Applied Sciences 11, no. 18 (2021): 8441. [7] Ramzan, Muham- mad, Hikmat Ullah Khan, Shahid Mahmood Awan, Amina Ismail, Mahwish Ilyas, and Ahsan Mahmood. ”A survey on state-of-the-art drowsiness detection techniques.” IEEE Ac- cess 7 (2019): 61904- 61919 [8] Sahayadhas, Arun, Kenneth Sundaraj, and Murugappan Murugappan. ”Detecting driver drowsiness based on sensors: a review.” Sensors 12, no. 12 (2012): 16937-16953. [9] Saini, Vandna, and Rekha Saini. ”Driver drowsiness detection system and techniques: a review.” International Journal of Computer Science and Information Technologies 5, no. 3 (2014): 4245-4249. [10] Sikander, Gul- badan, and Shahzad Anwar. ”Driver fatigue detection systems: A review.” IEEE Transactions on Intelligent Transportation Systems 20, no. 6 (2018): 2339-2352.

Copyright

Copyright © 2023 Dr. G. Gifta Jerith, M. Shreyas Reddy, D. Varshit Krishna, S. Aarthi, Ch. Hanumantha Rao. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.