Real Time Driver Drowsiness Detector Using Machine Learning

Abstract

Driver drowsiness has become one of the major reasons of deadly road accidents that lead to serious physical injuries, loss of lives. Continuously driving for long hours without rest causes fatigue and consequently fatal road accidents. Intense workload on the drivers tends to make them work overtime. These long working hours make them fatigued which in turn makes them feel drowsy while driving. The major victims of this overburden are the truck drivers and bus drivers who drive non-stop throughout day and night. This imposes a major threat to a human life and as well as to the life of the passengers. The aim of this is to develop a prototype drowsiness detection system (DDD). The focus will be placed on designing a system that will accurately monitor the open or closed state of the driver’s eyes in real-time. By monitoring the eyes, it is believed that the symptoms of driver fatigue can be detected early enough to avoid a car accident.

Introduction

I. INTRODUCTION

Driver drowsiness and fatigue are one of the most common reasons for accidents. The number of fatalities due to such accidents is increasing worldwide each year. This paper aims to lessen the number of accidents due to driver drowsiness and fatigue. This will in turn increase transportation safety. Driver drowsiness detection is a technology in vehicles that is useful in preventing accidents and saving the lives of drivers when they are getting drowsy. Drowsiness while driving tends to vehicle crashes and accidents. Many people die in car collisions every year due to fatigued driving that results from sleeping deprivation, intoxication, drug and alcohol abuse, exposure to heat or alcohol. Automobile manufacturers [8] such as Tesla, Mercedes-Benz, and others have various features for driving assistance such as warning of lane deviation, emergency braking systems, variable cruise control, and aid steering. These innovations have assisted drivers in avoiding the incidence of collisions. Samsung has investigated the attention level of a driver by reading facial characteristics and patterns .

Every year many people lose their lives because of major road accidents across the world and Drowsiness is the major issue that threatens the road safety and causes severe physical and sometime mental injuries to human life and to economic growth. The largest number of accidents in the world is due to drowsiness or fatigue. Drowsiness can be defined as an increasing awareness position along with a sleeping leaning to fall. This presence while driving usually leads to severe accidents as the driver is unable to action preventing such as turning off the road.

Intense workload on the drivers tends to make them work overtime. These long working hours make them fatigued which in turn makes them feel drowsy while driving. The major victims of this overburden are the truck drivers and bus drivers who drive non-stop throughout day and night. This imposes a major threat to his/her life and as well as to the life of the passengers.

Neglecting our duties towards safer travel has enabled hundreds of thousands of tragedies to get associated with this wonderful invention every year. It may seem like a trivial thing to most folks but following rules and regulations on the road is of utmost importance. While on road, an automobile wields the most power and in irresponsible hands, it can be destructive and sometimes, that carelessness can harm lives even of the people on the road.  One kind of carelessness is not admitting when we are too tired to drive.

A. Problem Statement

Drowsiness is one of the main issues that threatens the road safety and causes even injuries to human life and economical losses. The thought of mortality is increasingly haunted by approximately 20% fall road accidents worldwide. Teenagers, shift workers, commercial drivers and people with untreatedsleepdisordersorwithshort-termorchronicsleepdeprivation are at increased risk of drowsy driving and for having a fall-asleep crash. The aim is to implement a system that will prevent deadly accident sort owner drowsy driver just in time which would alert the driver before anything undesired happens.

B. Objectives

The primary objective of the proposed system is to overcome one or more problems of the existing systems.

1. To capture and pre-process the driver image.
2. To find eye detection fastly using non-intrusive method.
3. To find eye state fastly.
4. To wake up the driver to avoid accidents.
5. To protect the people and vehicle.
6. To provide a system that concentrates on a non-intrusive vision-based system.
7. To provide a driver drowsy detection system that will work well in day and night  irrespective of lighting condition.
8. To provide for increased efficiency by avoiding frame loss.
9. To provide a driver drowsy detection system that uses customized image processing algorithm to concentrate on the eyes detection.
10. To provide knowledge based method to authenticate the eyes movement using the image-processing algorithm to reduce the chance of misdetection or wrong alarm..

C. Proposed System

The measures that have been employed for drowsiness detection falls into three basic categories: physiological, behavioural and vehicle-based measures. The Haar cascade classifier proposed utilizes the ‘behavioural’ measure of the driver, that is, it works by interpreting the visual signs of the driver. The system deals with using information obtained for the binary version of the image to find the edges of the face, which narrows the area of where the eyes may exist. Once the face area is found, the eyes are found by computing the horizontal averages in the area.

Taking into account the knowledge that eye regions in the face present great intensity changes, the eyes are located by finding the significant intensity changes in the face. Once the eyes are located, measuring the distances between the intensity changes in the eye area determine whether the eyes are open or closed. A large distance corresponds to eye closure. If the eyes are found closed the system draws the conclusion that the driver is falling asleep and issues a warning signal. The system works under reasonable lighting conditions.

The aim of this is to develop a prototype drowsiness detection system. The focus will be placed on designing a system that will accurately monitor the open or closed state of the driver’s eyes in real-time. By monitoring the eyes, it is believed that the symptoms of driver fatigue can be detected early enough to avoid a car accident. There are many products out there that provide the measure of fatigue level in the drivers which are implemented in many vehicles. The driver drowsiness detection system provides the similar functionality but with better results and additional benefits.  Also, it reduces the number of accidents from sleep driving of vehicle and alerts the user on reaching a certain saturation point of the drowsiness measure

II. LITERATURE SURVEY

Real Time drowsiness detection using eye blink monitoring, Amna Rahman, MehreenSirsha, Aliya Khan - A literature survey summarizing some of the recent techniques proposedin this area is provided. To deal with this problem we propose an eye blink monitoring algorithm that uses eye feature points to determine the open or closed state of the eye and activate an alarm if the driver is drowsy.

Safe driving by detecting lane discipline and driver drowsiness, YashikaKatyall, SuhasAlur, Shipra Dwivedi - This paper will concentrate on avoiding the road accidents by concentrating mainly on drunk driving or drowsiness and lane discipline. The paper has two parts. Firstlylane detection using Hough Transform. Secondly, eye detection of driver for drowsiness detection. Thus, the main focus is on the fatigue of the driver and his maintenance of lane discipline.

Driver drowsiness detection using behavioral measures and machine learning techniques:  A review of state-of-art techniques, MkhuseliNgxande, Jules-Raymond Tapamo, Michael Burke – 2017. This paper presents a literature review of driver drowsiness detection based on behavioral measures using machine learning techniques. Faces contain information that can be used to interpret levels of drowsiness. There are many facial features that can be extracted from the face to infer the level of drowsiness. These include eye blinks, head movements and yawning.

Optimized driver safety through driver fatigue detection methods, Mr.  Omar Wathiq, Dr. Bhavna D. Ambudkar – In this paper, they have presented the detailed study on different approaches for driver safety using various driver drowsiness or distraction detection techniques.

After discussing the limitations of previous method, they designed the novel driver safety method with objective of improving the efficiency of driver distraction detection. The designed method is based on optimized face detector and feature extraction method with SVM classifier. The outcome of this paper is new efficient driver safety technique based on video image processing.

Machine learning and gradient statistics based real-time driver drowsiness detection, Cyun-Yi Lin, Paul Chang, Alan Wang, and Chih-Peng Fan – In this paper, the machine learning and gradient statisticsbased driver drowsiness detection is developed for the real-time application. The proposed system includes four parts, which are the face detection, the eye-glasses bridge detection, the eye detection, and the eye closure detection.

Driver drowsiness monitoring system using visual behavior and machine learning, Ashish Kumar, Rusha Patra – drowsy driving is one of the major causes of road accidents and death. Hence, detection of driver’s fatigue and its indication is an active research area. Most of the conventional methods are either vehicle based, or behavioral based or physiological based.  Few methods are intrusive and distract the driver, some require expensive sensors and data handling. Therefore, in this study, a lowcost, real-time driver’s drowsiness detection system is developed with acceptable accuracy.

Real-time drowsiness detection algorithm for driver state monitoring systems, Jang WoonBaek, Byung-Gil Han, Kwang-Ju Kim, Yun-Su Chung, Soo-In Lee – In this paper, they have proposed a novel drowsiness detection algorithm using a camera near the dashboard. The proposed algorithm detects the driver’s face in the image and estimates the landmarks in the face region.

A Survey on state-of-the-art drowsiness detection techniques, Muhammad Ramzan, Hikmatullah Khan, Shahid Mahmood Awan, Amina Ismail, Mahwish Ilyas, and Ahsan Mahmood – This paper tells drowsiness or fatigue is a major cause of road accidents and has significant implications for road safety. Several deadly accidents can be prevented if the drowsy drivers are warned in time. A variety of drowsiness detection methods exist that monitor the drivers drowsiness state while driving and alarm the drivers if they are not concentrating on driving. The relevant features can be extracted from facial expressions such as yawning, eye closure, and head movements for inferring the level of drowsiness.

A method of driver’s eyes closure and yawning detection for drowsiness analysis video image processing.by infrared camera, WisarootTipprasert, TheekapunCharoenpong, ChamapornChianrabutra – A challenge of research in area of the driver drowsiness detection is to detect the drowsiness in low light condition. In this paper, we proposed a method to detect driver’s eyes closure and yawning for drowsiness analysis by infrared camera.

Integration   of   ensemble   and   evolutionary   machine   learning   algorithms   for monitoringdriver behavior using physiological signals, AfsanehKoohestani, MoloudAbdar, Abbas Khosravi – The level of consciousness and the concentration of drivers while driving play a vital role for reducing the number of accidents. In recent decade, in-vehicle infotainment (IVI) [or in-car entertainment (ICE)] is one of the main reasons that lead to degradation of driver’s performance and losing awareness.

Driver drowsiness detection, KSatish, ALalitesh, KBhargavi, MSishir Prem and AnjaliT – All over the world drowsiness has been the significant cause of horrible accidents which is causing deaths  and  fatalities  injuries.  Day by day fatal injuries numbers are increasing globally. From the past many years, researchers have concluded drivers with a lack of sleep and more tiredness which causes drowsiness of the driver. This paper shows a new experimental model is designed for detecting drowsiness of driver is presented to reduce accidents caused by this problem which increases transport safety.

III. SYSTEM DESIGN

A. System Architecture

Below figure 1 showcases the various important blocks in the proposed system and their high-level interaction. It can be seen that the system consists of 5 distinct modules namely:

1. Video Acquisition: Video acquisition mainly involves obtaining the live video feed of the driver. Video acquisition is achieved, by making use of a camera.
2. Dividing into Frames: This module is used to take live video as its input and convert it into a series of frames/ images, which are then processed.
3. Face Detection: The face detection function takes one frame at a time from the frames provided by the frame grabber, and in each and every frame it tries to detect the face of the driver. This is achieved by making use of a set of pre-defined Haar cascade samples.
4. Eyes Detection: Once the face detection function has detected the face of the driver, the eyes detection function tries to detect the driver's eyes. This is achieved by making use of a set of pre- defined Haar cascade samples.
5. Drowsiness Detection: For detecting the face, since the camera is focused on the driver, we can avoid processing the Drowsiness detection System

After detecting the eyes of the driver, the drowsiness detection function detects if the driver is drowsy or not, by taking into consideration the state of the eyes, that is, open or closed and the blink rate.

As the proposed system makes use of OpenCV libraries, there is no necessary minimum resolution requirement on the camera. The schematic representation of the algorithm of the proposed system is depicted in Fig 3.  In the proposed algorithm, first video acquisition is achieved by making use of camera placed in front of the driver. The acquired video is then converted into a series of frames/images. The next step is to detect the driver's face, in each and every frame extracted from the video.

As indicated in Figure, we start with discussing face detection which has 2 important functions

a. Identifying the region of interest, and

b. Detection of face from the above region using Haarcascade.

To avoid processing the entire image, we mark the region of interest. By considering the region of interest it is possible to reduce the amount of processing required and also speeds up the processing, which is the primary goal of the proposed system.

Image at the corners thus reducing a significant amount of processing required. Once the region of interest is defined face has been detected, the region of interest is now the face, as the next step involves detecting eyes. To detect the eyes, instead of processing the entire face region, we mark a region of interest within the face region which further helps in achieving the primary goal of the proposed system. Next we make use of Haarcascade classifier constructed for eye detection and detect the eyes by processing only the region of interest. Once the eyes have been detected, the next step is to determine whether the eyes are in open/closed state.

Eye state is an important feature that can help in driver drowsiness. The methods that are based on eye state analysis uses PERCLOS (which is percentage of eyelid closure over time, reflecting slow eyelid closure) or droops rather than blinks. The classification is done as open eye or closed eye based on value of PERCLOSE state can also be detected by eye position orientation.

Eye blink detection method: The detected eye is equivalent to zero (closed eye) and non-zero values are indicated as partially or fully open eyes. The equation is used to calculate the average

If the value is more than the set threshold value, then system generates the alarm to alert the driver. The system uses the popular method of PERCLOS for drowsiness detection. The fatigue level of driver S is calculated as S=H/L, where H is the height and L is the length of drivers’ eye. Each frame of input video is categorized using the measured value of S. PERCLOS is measured as given in Equation:

B. Viola Jones Algorithm

First step of the algorithm is to extract the face region from the input video frame. For this purpose the Viola Jones algorithm has been used. Paul Viola and Michael Jones, in 2001, presented the first ever real time face detector. This algorithm classifies images on the basis of values of simple features. Its main benefits are speed and accuracy, as it achieves detection rates comparable to best systems and is much faster than the most of them. A distinctive quality of this algorithm is that it uses rectangular features instead of individual pixels. At first, sum of pixels is calculated within a rectangular box.

To compute these rectangle features rapidly and faster, Viola and Jones introduced a representation of the image called integral image. They demonstrated that using this integral image, four array references can be used to compute any rectangular sum.  The task of preparing the classifier to select the best feature from the all the features uses the characteristics chosen by Adaboost. Adaboost is a computational approach that finds a single rectangle feature using a weak learning algorithm.  The weak learner finds out the best threshold classification function for each feature, such that there is least misclassification of examples. Most of the area, in an image, is non-face region. Hence the better idea would be to discard a region if it is not a face region, and do not process it again. In this way, more time is left to look for a possible face region. Viola and Jones established the idea of “Cascade of Classifiers” to carry out this mechanism. In this method, the features are classified into various steps of classifier. Each feature is applied one at a time, instead of applying all 6000 on a single window. If a window does not pass the first phase of features it is rejected and all other features on it are also discarded. If it passes, the second stage is applied to the window and the method continues. The window that successfully passes through all these stages is selected as face area. Face detection - is a computer technology being used in a variety of applications that identifies human faces in digital images. Face detection also refers to the psychological process by which humans locate and attend to faces in a visual scene. Face-detection algorithms focus on the detection of frontal human faces. It is analogous to image detection in which the image of a person is matched bit by bit. Image matches with the image stores in database. Any facial feature changes in the database will invalidate the matching process.

GUI tkinter - Python offers multiple options for developing GUI (Graphical User Interface). Out of all the GUI methods, tkinter is the most commonly used method. It is a standard Python interface to the Tk GUI toolkit shipped with Python. Python with tkinter is the fastest and easiest way to create the GUI applications.

CNN - The convolution neural network is a representation of a particular type of feed-forward neural network which can be used for a variety of machine learning and predictive tasks. The CNN is responsible for taking in an image of an eye as the input and it classifies whether the eye is open or closed.

Image processing – Image processing is a method to perform some operations on an image, in order to get an enhanced image or to extract some useful information from it. It is a type of signal processing in which input is an image and output may be image or characteristics/features associated with that image. Nowadays, image processing is among rapidly growing technologies. It forms core research area within engineering and computer science disciplines too.

Image processing basically includes the following three steps:

• Importing the image via image acquisition tools;
• Analysing and manipulating the image;
• Output in which result can be altered image or report that is based on image analysis

IV. RESULTS AND SCREENSHOTS          

The figure 5 depicts the output screen of the proposed system. It is the home page in which we get login button. We use django framework for fron-end. Once we start the Flask server we get this page in our local host port.

Conclusion

In this paper, a low-cost, real time driver drowsiness monitoring system has been proposed based on visual behaviour and machine learning. Here, visual behaviour features like eye aspect ratio, mouth opening ratio and nose length ratio are computed from the streaming video, captured by a webcam. An adaptive thresholding technique has been developed to detect driver drowsiness in real time. The developed system works accurately with the generated synthetic data. Subsequently, the feature values are stored and machine learning algorithms have been used for classification. Bayesian classifier, Haar cascade classifier and Viola Jones Algorithm have been explored here.

References

[1] W. L. Ou, M. H. Shih, C. W. Chang, X. H. Yu, C. P. Fan, \"Intelligent Video-Based Drowsy Driver Detection System under Various Illuminations and Embedded Software Implementation\", 2015 international Conf. on Consumer Electronics - Taiwan, 2015. [2] [2] W. B. Horng, C. Y. Chen, Y. Chang, C. H. Fan, “Driver Fatigue Detection based on Eye Tracking and Dynamic Template Matching”, IEEE International Conference on Networking, Sensing and Control, Taipei, Taiwan, March 21-23, 2004. [3] [3] S. Singh, N. P. papanikolopoulos, “Monitoring Driver Fatigue using Facial Analysis Techniques”, IEEE Conference on Intelligent Transportation System, pp 314-318. [4] [4] B. Alshaqaqi, A. S. Baquhaizel, M. E. A. Ouis, M. Bouumehed, A. Ouamri, M. Keche, “Driver Drowsiness Detection System”, IEEE International Workshop on Systems, Signal Processing and their Applications, 2013..

Copyright

Copyright © 2022 Abdul Razak M S, Amulya H G, Gangothri R P, Sujan G, Aditi A M. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.