Driver Drowsiness Detection System Using Machine Learning

Abstract

Drowsy driving is one of the major causes of road accidents and death. Hence, detection of driver\'s fatigue and its indication is an active research area. Most of the conventional methods are either vehicle based, or behavioral based or physiological based. Few methods are intrusive and distract the driver, some require expensive sensors and data handling. Therefore, in my literature survey, a low cost, real time driver\'s drowsiness detection system is developed with acceptable accuracy. The proposed work mainly focus on a webcam records the video and driver\'s face is detected in each frame employing image processing techniques. Facial landmarks on the detected face are pointed and subsequently the eye aspect ratio is computed and depending on their values, drowsiness is detected based on developed adaptive thresholding. Machine learning algorithms have been implements as well in an offline manner

Introduction

I. INTRODUCTION

Drowsy driving is one of the major causes of deaths occurring in road accidents. The truck drivers who drive for continuous long hours (especially at night), bus drivers of long instance route or overnight buses are more susceptible to this problem. Driver drowsiness is an overcast nightmare to passengers in every country. Every year, a large number of injuries and deaths occur due to fatigue related road accidents. Hence, detection of driver’s fatigue and its indication is an active area of research due to its immense practical applicability. The basic drowsiness detection system has three blocks/modules; acquisition system, processing system and warning systemThe aim of this project is to develop a prototype drowsiness detection system. The focus will be placed on designing a system that will accurately monitor the open or closed state of the driver’s eyes in real-time.  By monitoring the eyes, it is believed that the symptoms of driver fatigue can be detected early enough to avoid a car accident. Detection of fatigue involves the observation of eye movements and blink patterns in a sequence of images of a face. Initially, we decided to go about detecting eye blink patterns using pyton. The procedure used was the geometric manipulation of intensity levels. The algorithm used was as follows. First we input the facial image using a webcam. Preprocessing was first performed by binarizing the image. The top and sides of the face were detected to narrow down the area where the eyes exist

II. RELATEDWORKS

Some efforts have been reported in the literature on the development of the not-intrusive monitoring drowsiness systems based on the vision. Malla et al. [1] develop a light-insensitive system. They used the Haar algorithm to detect objects [2] and face classifier implemented by [3] in OpenCV [4] libraries. Eye regions are derived from the facial region with anthropometric factors. Then, they detect the eyelid to measure the level of eye closure. Vitabile et al. [5] implement a system to detect symptoms of driver drowsiness based on an infrared camera. By exploiting the phenomenon of bright pupils, an algorithm for detecting and tracking the driver's eyes has been developed. When drowsiness is detected, the system warns the driver with an alarm message. Bhowmick et Kumar [6] use the Otsu thresholding [7] to extract face region. The localization of the eye is done by locating facial landmarks such as eyebrow and possible face center. Morphological operation and Kmeans is used for accurate eye segmentation. Then aset of shape features are calculated and trained using non-linear SVM to get the status of the eye. Hong et al. [8] define a system for detecting the eye states in real time to identify the driver drowsiness state. The face region is detected based on the optimized Jones and Viola method [2]. The eye area is obtained by an horizontal projection. Finally, a new complexity function with a dynamic threshold to identify the eye state. Tian et Qin [9] build a system that checks the driver eye states. Their system uses the Cb and Cr components of the YCbCr color space. This system locates the face with a vertical projection function, and the eyes with a horizontal projection function.

Once the eyes are located the system calculates the eyes states using a function of complexity. Under the light of what has been mentioned above, the identification of the driver drowsy state given by the PERCLOS is generally passed by the following stages: 1) Face detection, 2) Eyes Location, 3) Face and eyes tracking, 4) Identification of the eyes states, 5) Calculation of PERCLOS and identification of driver state.

A.Eyes Localization

Since the eyes are always in a defined area in the face (facial anthropometric properties), we limit our research in the area between the forehead and the mouth (Eye Region of Interest ‘eROI’) (Figure 4.a). We benefit from the symmetrical characteristic of the eyes to detect them in the face. First, we sweep vertically the eROI by a rectangular mask with an estimated height of height of the eye and a width equal to the width of the face, and we calculate the symmetry. The eye area corresponds to the position which has a high measurement of symmetry. Then, in this obtained region, we calculate the symmetry again in both left and right sides. The highest value corresponds to the center of the eye.

B. Face Dectction

The symmetry is one of the most important facial features. We modeled the symmetry in a digital image by a one-dimensional signal (accumulator vector) with a size equal the width of the image, which gives us the value corresponding to the position of the vertical axis of symmetry of objects in the image. The traditional principle to calculate the signal of symmetry is for each two white pixels which are on the same line we increment the value in the medium between these two pixels in the accumulator vector. (The algorithm is applied on an edge image, we called a white pixel: the pixel with value 1). We introduce improvements on the calculation algorithm of symmetry into an image to adapt it to the detection of face, by applying a set of rules to provide a better calculation of symmetry of the face. Instead of computing the symmetry between two white pixels in the image, it is calculated between two windows (Z1 and Z2)

C. Dataflow Diagrams

A use case diagram in the Unified Modeling Language is a type of behavioral diagram defined by and created from a Use-case analysis. Its purpose is to present a graphical overview of the functionality provided by a system in terms of actors, their goals, and any dependencies between those use cases. The main purpose of a use case diagram is to show what system functions are performed for which actor. Roles of the actors in the system can be depicted

 In software engineering, a class diagram in the Unified Modeling Language (UML) is a type of static structure diagram that describes the structure of a system by showing the system's classes, their attributes, operations (or methods), and the relationships among the classes. It explains which class contains information

D. Eyes States

The determination of the eye state is to classify the eye into two states: open or closed. We use the Hough transform for circles [10] (HTC) on the image of the eye to detect the iris. For that, we apply the HTC to the edge image of the eye to detect the circles with defined rays, and we take at the end the circle which has the highest value in the accumulator of Hough for all the rays. Then, we apply the logical 'AND' logic between edges image and complete circle obtained by the HTC by measuring the intersection level between them "S". Finally, the eye state “?????” is defined by testing the value “S” by a threshold.

E.  How The Algoritham Works

Our system starts with the initialization phase, which is face and eyes detection to extract both face and eyes regions and take them as templates to track them in the following frames. For each tracking we test if that tracking is good or bad? If the tracking is bad we return to the initialization step, else we pass to the following steps which are: eyes states identification and driver state.

III. EXPERIMENTAL RESULTS:

To validate our system (Figure 6), we test on several drivers in the car with real driving conditions. We use an IR camera with infrared lighting system operates automatically under the conditions of reduced luminosity and night even in total darkness.

???????

Conclusion

In this paper, a low cost, real time driver drowsiness monitoring system has been proposed based on visual behavior and machine learning. Here, visual behavior features like eye aspect ratio, mouth opening ratio and nose length ratio are computed from the streaming video, captured by a webcam. An adaptive thresholding technique has been developed to detect driver drowsiness in real time. The developed system works accurately with the generated synthetic data. Subsequently, the feature values are stored and machine learning algorithms have been used for classification. Bayesian classifier, FLDA and SVM have been explored here. It has been observed that FLDA and SVM outperform Bayesian classifier. The sensitivity of FLDA and SVM is 0.896 and 0.956 respectively whereas the specificity is 1 for both. As FLDA and SVM give better accuracy, work will be carried out to implement them in the developed system to do the classification (i.e., drowsiness detection) online. Also, the system will be implemented in hardware to make it portable for car system and pilot study on drivers will be carried out to validate the developed system.In this paper, we presented the conception and implementation of a system for detecting driver drowsiness based on vision that aims to warn the driver if he is in drowsy state. This system is able to determine the driver state under real day and night conditions using IR camera. Face and eyes detection are implemented based on symmetry. Hough Transform for Circles is used for the decision of the eyes states. The results are satisfactory with an opportunity for improvement in face detection using other techniques concerning the calculation of symmetry. Moreover, we will implement our algorithm on a DSP (Digital Signal Processor) to create an autonomous system working in real time.ber of frames multiplied by 100.

References

[1] Malla, P. Davidson, P. Bones, R. Green and R. Jones, \"Automated Video-based Measurement of Eye Closure for Detecting Behavioral Microsleep\", in 32nd Annual International Conference of the IEEE, Buenos Aires, Argentina, 2010. [2] P. Viola and M. Jones, \"Rapid Object Detection using a Boosted Cascade of Simple Features\", in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001. [3] R. Lienhart and J. Maydt, \"An Extended Set of Haar-like Features for Rapid Object Detection\", in Proceedings of the IEEE International Conference on Image Processing, 2002. [4] OpenCV. Open Source Computer Vision Library Reference Manual, 2001. [5] S. Vitabile, A. Paola and F. Sorbello, \"Bright Pupil Detection in an Embedded, Real-time Drowsiness Monitoring System\", in 24th IEEE International Conference on Advanced Information Networking and Applications, 2010. [6] B. Bhowmick and C. Kumar, \"Detection and Classification of Eye State in IR Camera for Driver Drowsiness Identification\", in Proceeding of the IEEE International Conference on Signal and Image Processing Applications, 2009. [7] N. Otsu, \"A Threshold Selection Method from Gray-Level Histograms\", IEEE Transactions on Systems,Man and Cybernatics, pp. 62-66, 1979. [8] T. Hong, H. Qin and Q. Sun, \"An Improved Real Time Eye State Identification System in Driver Drowsiness Detection\", in proceeding of the IEEE International Conference on Control and Automation, Guangzhou, CHINA, 2007. [9] Z. Tian et H. Qin, \"Real-time Driver\'s Eye State Detection\", in Proceedings of the IEEE International Conference on Vehicular Electronics and Safety, October 2005. [10] M. S. Nixon and A. S. Aguado, Feature Extraction and Image Processing, 2nd ed., Jordan Hill, Oxford OX2 8DP, UK, 2008. [11] W. L. Ou, M. H. Shih, C. W. Chang, X. H. Yu and C. P. Fan, \"Intelligent VideoBased Drowsy Driver Detection System under Various Illuminations and Embedded Software Implementation\", 2015 international Conf. on Consumer Electronics, 2015. [12] W. B. Horng, C. Y. Chen, Y. Chang and C. H. Fan, \"Driver Fatigue Detection based on Eye Tracking and Dynamic Template Matching\", IEEE International Conference on Networking Sensing and Control, March 21–23, 2004. [13] S. Singh and N. P. Papanikolopoulos, \"Monitoring Driver Fatigue using Facial Analysis Techniques\", IEEE Conference on Intelligent Transportation System, pp. 314- 318. [14] B. Alshaqaqi, A. S. Baquhaizel, M. E. A. Ouis, M. Bouumehed, A. Ouamri and M. Keche, \"Driver Drowsiness Detection System\", IEEE International Workshop on Systems Signal Processing and their Applications, 2013. [15] M. Karchani, A. Mazloumi, G. N. Saraji, A. Nahvi, K. S. Haghighi, B. M. Abadi, et al., \"The Steps of Proposed Drowsiness Detection System Design based on Image Processing in Simulator Driving\", International Research Journal of Applied and Basic Sciences, vol. 9, no. 6, pp. 878-887, 2015.

Copyright

Copyright © 2023 K. Tulasi Krishna Kumar, B. Keerthi, C. SP. Vishwak Sen, T. Karthikeya. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.