Driver Drowsiness Detection System

Abstract

Recently, in addition to autonomous vehicle technology research and development, machine learning methods have been used to predict a driver\'s condition and emotions in order to provide information that will improve road safety. A driver\'s condition can be estimated not only by basic characteristics such as gender, age, and driving experience, but also by a driver\'s facial expressions, bio-signals, and driving behaviours. Recent developments in video processing using machine learning have enabled images obtained from cameras to be analysed with high accuracy. Therefore, based on the relationship between facial features and a driver’s drowsy state, variables that reflect facial features have been established. In this paper, we proposed a method for extracting detailed features of the eyes, the mouth, and positions of the head using OpenCV and Dlib library in order to estimate a driver’s level of drowsiness.

Introduction

I. INTRODUCTION

One of the main reasons for untimely deaths today is road accidents. In the current times, we can see drastic changes in how humans manage their time. The natural sleep cycle of human beings has therefore been disturbed. Due to a lack of sleep and irregular sleep cycles, humans tend to feel drowsy at any time of the day. With these poor work-life timings, people can find it difficult to carry out the activities like driving which requires a healthy and properly functioning state of mind and body. Drowsiness is one of the major causes of road accidents in today’s time. According to the Central Road Research Institute (CRRI), tired drivers who drowse off while driving are responsible for about 40% of road accidents. Several misfortunes can be avoided if the driver is alerted in time. Most of the time the drivers would lose their alertness and meet with unfortunate accidents. This loss of the state of alertness is due to fatigue and drowsiness of the driver. This situation becomes very dangerous when the driver is alone. The ultimate reason for the loss of the state of alertness is accidental microsleeps. Drowsiness or fatigue is one of the main reasons of low road safety and some severe injuries, economy loss, and even deaths. Collectively, these situations increase the risk of road accidents. Using computer for automatic fatigue detection, several misfortunes can be avoided. The drowsiness detection systems continuously analyses the driver’s condition and warns before any unfortunate situation arises. Due to the accidents being caused due to the fatigue state of the drivers, several methods have been developed to detect the driver’s drowsiness state and warn accordingly. Each method has its advantages as well as disadvantages. There have been some great works in this field, but we can have some space for future improvements. Late insights, assess that yearly 1,200 deaths and 76,000 injuries can be credited to weariness related accidents. It can be seen there are around 2,400 road accidents consistently which is one death per every four hours. It has been figured around 20% of car crashes with driver fatalities are due to driver’s drowsiness. It was uncovered that driving execution quickly drop with expanded tiredness which result in making more than 20% of all vehicle accidents. Less attention heads the driver to being distracted and the likelihood of street accident goes high. Drowsiness related accidents have all the earmarks of being more serious, because of the higher speeds involved distraction and the driver being not able to take any avoiding activity, or even brake, before the accident. The improvement of innovations for recognizing or preventing tiredness of the driver is a significant test in the field of accident preventing systems. Because of the danger that that drowsiness presents on the road, strategies need to be created for checking its influences. Loss of the awareness because of the tiredness causes a few changes in the human's body and activities. These side effects and parameters empower us to effectively measure the drowsiness level. Because of the hazard that drowsiness presents on the road, methods need to be developed for counteracting its affects. Driver inattention might be the result of a lack of alertness when driving due to driver drowsiness and distraction. Driver distraction occurs when an object or event draws a person’s attention away from the driving task. Unlike driver distraction, driver  drowsiness involves no triggering event but, instead, is characterized by a progressive withdrawal of attention from the road and traffic. Additionally, we believe that drowsiness can negatively impact people in working and classroom environments as well. Although sleep deprivation and college go hand in hand, drowsiness in the workplace especially while working with heavy machinery may result in serious injuries similar to those that occur while driving drowsily. Our solution to this problem is to build a detection system that identifies key attributes of drowsiness and triggers an alert when someone is drowsy before it is too late

II. LITERATURE REVIEW

A survey done by National Highway Traffic Safety Administration estimated that there were 56,000 sleep related road crashes in the U.S.A in 1996. Another survey done in 2007 says that 18% of accidents involved fatigue as the main factor. In Britain up to 20% of serious road accidents were caused due to fatigue. Similarly, survey done by the Road and Traffic Authority states that in the year 2007, fatigue contributed to 20% of accidents caused on road. Accidents due to drowsy was prevented and controlled when the vehicle is out of control. The term used here for the identification that the driver is drowsy is by using eye blink of the driver.

These types of accidents occurred due to drowsy and driver could not be able to control the vehicle, when the driver wakes. The drowsiness was identified by the eye blink closure rate. If the driver is in drowsy state, then the system will give buzzer signal.

One of the suggested methods is to monitor the movement of the vehicle to detect drowsiness of the driver. However this method has limitations as the results are influenced by the type of vehicle and the condition of road. Another method is to process the electrocardiogram (ECG) signals of driver. This approach also has limitations as ECG probes shall always be connected to the driver‘s body. That would disturb the driver. Few researches tried to assess the fatigue factor by monitoring the eye blink rate of the driver. Successful detection of eye blink rate has been the interest of many researchers proposed methods based on combination of projection and the geometry feature of iris and pupil.

Some works also are based on "Support Vector Machine" (SVM) classifier. The SVM classifier is used to detect state of the eye. SVM classifier and Gabor filter has been used to extract eye characteristic. In the above methods, the authors used some conditions which make some difficulties in the eye state recognition. The system detects the fatigue symptoms of the driver which consists of an eye blink sensor for driver blink attainment and an adaptive speed controller designed using stepper motor for providing actual positioning of the throttle valve to adjust the speed of vehicle. Advanced technology offers some hope to avoid these up to some extent. This paper involves measure and control of accidents by using both alcohol sensor and IR sensor. 

It uses remotely located charge-coupled-device cameras with active infrared illuminators to acquire video images of the driver. Various visual signs that typically characterize the level of alertness of a person were extracted in real-time and systematically combined to infer the fatigue level of the driver. The visual cues employed characterize eyelid, gaze, head movements and facial expressions. A probabilistic model was developed to model human fatigue and to estimate fatigue based on the visual cues. The simultaneous use of visual cues and their systematic combination earns an accurate fatigue characterization. This system was validated under real-life fatigue conditions with the human subjects of different ethnic backgrounds with or without glasses; and beneath different illumination conditions. It was found to be reasonably reliable, and accurate in fatigue characterization.

The computer vision based method determine if a driver is holding a cell phone close of his/her ears using the Supervised Descent Method (SDM)  which it tracks some facial landmarks to extract a crop of regions of interest (ROI) (the driver‘s ear region). Features are extracted from the ROIs and the phone usage is detected using previously trained classifiers. The system can be processed in near real time. The approach of Yang send beeps of high frequency through the car sound equipment, network Bluetooth, and use software running on the phone for capturing and processing sound signals. The beeps are used to calculate the position where the cell phone is, and then we know when the driver (or another passenger in the car) is talking on it. The proposal achieved a classification accuracy of more than 90%. This approach works with hands-free usage, but it depends on the operating system and mobile phone brand, and the software has to be continually enabled by the driver.

Another proposal identifies the behaviours of a distracted driver associated with text messaging. The approach uses a cell phone programmed to record any typing done (pressing and releasing any key). An analysis can be performed to verify distractions through these records. Experiments were done with six participants used the cell phone as passenger and driver what distinct patterns of typing frequency were shown in each situation. The driver cannot reply the text messaging with an average frequency of 2 press keys by the second. This constraint detects the driver in 99% of cases, but it works with offline data processing.

The system could be applied to others equipment, i.e, the GPS.

Another system presents for detecting and parsing the movement of the driver‘s head. It uses the optical flow of the driver‘s image. The detected movements are compared with a predefined set of relevant movements of the automotive environment. The system observes the time spent which the driver is looking in the same direction. If the time spent is longer than a predefined interval of time, the distraction is observed. The system‘s accuracy in a real environment is 86%. The approach finds the driver distraction level for observing objects outside of the vehicle. A fusion of two computer vision systems is used.

The first one detects the driver‘s field of view (inside of the vehicle) and the other detects the movement (outside) by using the salience map (the outside movements what must attract the driver‘s attention).

Over the last decade, there have been various studies done related to drowsiness detection and drunk driving. Features using a driver‘s Visual characteristics, Physiological and Driving- behaviour based studies have been conducted each having their own advantages for drowsiness detection and by using sensors for drunk driving detection. A survey that provides a comprehensive insight into the well-established techniques for driver inattention monitoring and introduces the use of most recent and futuristic solutions exploiting mobile technologies such as smartphones and wearable devices. The studies were categorized into two groups: driver drowsiness and distraction. A comprehensive compilation, used features, classification methods, accuracy rates, system parameters, and environmental details, was represented. 

A similar approach was also taken for the methods used for the detection of driver distraction. A visual analysis of Eye State and Head Pose (HP) for continuous monitoring of alertness of a vehicle driver. The proposed scheme used visual features such as Eye Index (EI), Pupil Activity (PA), and HP to extract critical information on non-alertness of the driver . A system is designed and implemented an automatic system, using computer vision, which runs on a computationally limited embedded smart camera platform to detect yawning. Implementation of the Viola–Jones algorithm for face and mouth detections and, use of a back-projection theory for measuring both the rate and the amount of the changes in the mouth, to detect yawning along with the histogram of the Gray scale image.  

A warning alarm was also sounded if driver fatigue was believed to reach a defined threshold. And presented an efficient driver‘s drowsiness detection system, by using yawning detection. The consideration of eye detection and mouth detection was done, detecting the driver‘s face using YCbCr method. After that, eyes and mouth positions by using HAAR features. Lastly yawning detection performed by using mouth geometric features.

There has been  conducted the research in a relatively simpler way. They claim the sleep onset is the most critical consequence of fatigued driving, separate the issue of sleep onset from the global analysis of the physiological state of fatigue, and take eyes opening and closing as cues of sleep onset. They have used vision-based system to monitor the eyes conditions in order to detect fatigue in driving. Lee et al. have used two fixed cameras to capture the driver‘s sight line and the driving lane path for the purpose of driving pattern and status recognition. They calculate the correlation coefficients among them to monitor the driving status and patterns. These methods all need one or more cameras to be installed in the vehicle and just in front of the driver. It will cause certain potential safety hazard to the driver

III. LITERATURE SUMMARY

IV. METHODOLOGY

When people start driving, they tend to gaze at the left or right sides of the road after setting a destination. However, as time passes by, the frequency of blinking decreases as a driver becomes fatigued. A reduced rate of blinking may result in drivers closing their eyes unconsciously, and their heads may flop. These temporal behaviors can be observed continuously. Therefore, temporal changes of facial features may be analyzed to identify changes in a driver’s condition.

In this section results are obtained using software and hardware platforms to achieving the objective of driver drowsiness detection and alcoholic intoxication. Besides eye and head movements, another visual cue that can potentially capture one’s level of drowsiness is his/her eyes and faces detection analysis. Making a real time application with computer vision is very effective and efficient challenging task that needs processing powerful system. OpenCV is open source software, which is used for creating computer vision. OpenCV is available in C, C++, and Python and Java programming languages extension. In case of driver is in sleepy or finding fatigue, the message will be sent by using GSM and buzzer will be turned on till the GSM positive message from car owner.. Fig. 1 shows the basic block diagram of the proposed system. Haar Feature based Cascade Classifier technique, it is a machine learning based approach where a cascade function is trained from a lot of positive and negative images, and this positive image is used for detecting face region and eye region the update of region of interest ROI. Open CV is packed with a trainer as well as detector. The open CV is used for creating user defined object classifier. The object classifier that has been created is stored in.xml file extension classifier can be used in the later stages of programming. Also in this paper we use canny operator edge detection for recognize exact coordinate of eyes region Cellphone ring detector used here to detect cellphone usage during the driving and warn the driver to restrict driver not to use mobile while driving.

Conclusion

Purpose of our project is to help solving real life problem in very cost-efficient way. It alerts the truck driver, whenever the driver feels drowsy and closes his eyes for more than a second, As a result, it alerts the driver. As a result the accident ratio decreases. Hence, our project if commercially developed will help in saving the precious life of driver. The system which can differentiate normal eye blink and drowsiness which can prevent the driver from entering the state of sleepiness while driving. During the monitoring, the system can decide if the eyes are opened or closed. The driver drowsiness system will reduce accidents causing and provides safe life to the driver and vehicle safety. A system for driver safety and car security is presented only in the luxurious costly cars. Using drowsiness detecting system, driver safety can be implemented in normal cars also.

References

[1] Driver Drowsiness Detection Using Deep Learning, Ajinkya Ranker, Nilima Kulkarni, and Aniket Raut [2] Distracted Driver Detection with Deep Convolutional Neural Network O. G. Basubeit, D. N. T. How, Y. C. Hou, K. S. M. Sahari international Journal of Recent Technology and Engineering (IJRTE) ISSN: 2277-3878, Volume-8 Issue-4, November 2019 [3] Real-Time Driver State Monitoring Using a CNN Based Spatio-Temporal Approach, Neslihan Kose1 , Okan Kopuklu2 , Alexander Unnervik3 and Gerhard Rigoll4, 18 Jul 2019 [4] Real-Time Driver State Monitoring Using a CNN Based Spatio-Temporal Approach, Neslihan Kose1 , Okan Kopuklu2 , Alexander Unnervik3 and Gerhard Rigoll4, 18 Jul 2019 [5] Drowsiness Detection of a Driver using Conventional Computer Vision Application, Hitendra Garg, Feb 28-29, 2020 [6] Driver drowsiness detection with eyelid related parameters by Support Vector Machine Hu Shuyan *, Zheng Gangtie School of Astronautics, Beijing University of Aeronautics and Astronautics, 37 Xueyuan Road, Haidian District, Beijing 100083, China [7] Real-time Driver Drowsiness Detection for Android Application Using Deep Neural Networks Techniques, Rateb Jabbara* , Khalifa Al-Khalifa , Mohamed Kharbechea , Wael Alhaj Yaseen , Mohsen Jafarib , Shan Jiangb ,2018 [8] real-time Detection of Distracted Driving based on Deep Learning. Weihua Sheng,Duy Tran,Ha Do,he Bai,July 2018 [9] Detecting Driver Drowsiness in Real Time Through Deep Learning Based Object Detection Nabit A. Bajwa, Ahmad Muhammad Anwaar ,Anabia Sohail and Asifullah Khan 2019 [10] Real-Time Driver State Monitoring Using a CNN Based Spatio-Temporal Approach* Neslihan Kose1 , Okan Kopuklu2 , Alexander Unnervik3 and Gerhard Rigoll4, 18 Jul 2019 [11] Detection of Distracted Driver using Convolutional Neural Network Bhakti Baheti Suhas Gajre Sanjay Talbar,2018 [12] Distracted Driver Detection and Classification, Prof. Pramila M. Chawan,Shreyas Satardekar,Dharmin Shah,Rohit Badugu,Abhishek Pawar, 2018 [13] Distracted Driver Classification Using Deep Learning ? Munif Alotaibia,1 , Bandar Alotaibib,2020 [14] Received August 4, 2019, accepted August 18, 2019, date of publication August 21, 2019, date of current version September 5, 2019. Digital Object Identifier 10.1109/ACCESS.2019.2936663 Real-Time Driver-Drowsiness Detection System Using Facial Features WANGHUA DENG1 AND RUOXUE WU 1,2 [15] Vlad Tamas, Vistrian Maties. Real-Time Distracted Drivers Detection Using Deep Learning. American Journal of Artificial Intelligence. Vol. 3, No. 1, 2019, pp. 1-8. doi: 10.11648/j.ajai.20190301.11 Received: February 22, 2019; Accepted: April 8, 2019; Published: May 15, 2019

Copyright

Copyright © 2022 Manjunath S, Banashree P, Shreya M, Sneha Manjunath Hegde, Nischal H P. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.