Driver Drowsiness Detection (SLEEP GUARDIAN)

Abstract

Drowsiness detection system can detect the possibility of sleep using AI. Drowsiness detection systems have gained significant attention in recent years due to their potential to enhance road safety and prevent accidents caused by driver fatigue. A dedicated eye tracking module monitors eye movement and blink frequency to detect signs of fatigue. This drowsiness detection system will work on dataset in the first stage comparing with the base research paper , showing its advantages and the next stage how 2 or more different modules can be clubbed together to make it a ready to run in today’s scenario with all the different problems with the highest efficiency . Making this system in python gives a wide range of options to explore , help to reach open cv libraries where we can use different types of data set and check whether the driver is awake or not.

Introduction

I. INTRODUCTION

Road safety is a significant public health issue, and a cause of injuries and fatalities. According to a report by the Ministry of Road Transport and Highways Transport Research Wing, road accidents claimed 1,55,622 lives and harmed 3,84,448 people in 2021. A crucial problem that causes numerous car accidents annually is driver fatigue. Due to the incapacity of a driver to halt or swerve to prevent or minimize the impact, accidents caused by driver sleepiness are much more inclined to result in fatalities or severe accidents. Fatigue lowers attentiveness, alertness, and concentration, which impairs the accomplishment of tasks requiring attention such as driving. About 20% of the people have admitted to falling asleep at the wheel with 40% of the people confessing that this has taken place at least once in  their driving careers. More than 65% of all deadly single car crashes are related to inebriation, it is important that we create a safe driving environment. To develop such a system, we need to predict the driver's condition [1].Below is a brief description of the research paper.

In the pursuit of addressing the pressing issue of sleep-deprived driving, researchers have explored various methods for detecting driver drowsiness[2].One promising approach involves utilizing a 68-coordinate facial landmark system, which offers a detailed analysis of the driver's facial features to assess their condition at the wheel.

The 68-coordinate system, often used in facial landmark detection, allows for precise localization of key points on a driver's face, including the eyes, nose, mouth, and other facial contours. By employing this system, researchers aim to capture subtle changes in facial expressions and movements that may indicate drowsiness[9].

One key advantage of the 68-coordinate system is its ability to provide a rich set of data for analysis. It enables the tracking of eye positions, eye blinks, yawning, and head movements with exceptional accuracy. This comprehensive data can then be used to assess the driver's state in real-time.

For instance, a study cited in the text discusses the utilization of YCbCr color space and Canny edge detection methods alongside the 68-coordinate system to determine if the driver is under fatigue[5]. When the driver exhibits signs of drowsiness, an alarm system is activated promptly, enhancing safety on the road.

Moreover, the system based on the 68-coordinate framework is capable of functioning under challenging conditions such as low light and when the driver wears spectacles. This adaptability is essential for real-world applications[4]..

In summary, the integration of the 68-coordinate facial landmark system into drowsiness detection research offers a robust and precise means of assessing a driver's condition. By leveraging this technology alongside other techniques, researchers aim to develop advanced driver safety systems that can effectively combat the dangers of sleep-deprived driving and reduce vehicular accidents.

II. LITERATURE REVIEW

Drowsiness detection systems for drivers have been developed and implemented to enhance road safety by alerting drivers when they show signs of drowsiness or fatigue. The system utilizes a 68-coordinate system and the iBUG-300-W dataset. In this approach, facial landmark detector is employed to approximate the placement of 68 (x, y) coordinates mapping to facial structures. Several studies that have explored the effectiveness of these systems in recent years. Research has shown that 68-coordinate facial landmark detection systems, which track key points on the driver's face such as the eyes, nose, and mouth, can accurately assess driver drowsiness by analyzing changes in facial features and eye behavior. These systems utilize machine learning algorithms to classify drowsy states based on established patterns, such as slow eyelid closures, head nods, and changes in gaze direction. This system will detect driver’s[2] fatigue by the processing of the eye region. After image acquisition, the first stage of processing is face detection. The system calculates the Eye aspect ratio (EAR) by measuring vertical distances between eyelids and dividing by the horizontal distance. In a similar way to decide the yawning factor the Mouth aspect ratio.. An infrared camera is utilized to constantly track the driver’s facial landmark and motion of the eyes and mouth. Images are taken by the camera at a fixed frame rate of 20 frames per second (fps). These images are transferred to an image processing component which executes the face landmark detection to recognize driver’s distractions and drowsiness[6].

The 68-face landmark model in the Dlib library shows the way to obtain the facial features like eyes, mouth, nose, etc. Sometimes, we might not want to detect all of the facial features which can be done by customized training of the Dlib's 68-landmark model[7].

The exisying system has the following roadmap for drowsiness detection.

III. METHODOLOGY

The proposed drowsiness detection system is a critical advancement in ensuring road safety by addressing the significant factors of driver fatigue, distractions, and drowsiness, all of which contribute to road accidents. This system operates on a top-down approach, leveraging the dlib library for facial landmark detection and OpenCV for image processing in Python.

1. System Overview: The system employs a top-down approach, which means it begins with a high-level understanding of the problem and gradually breaks it down into smaller components.
2. Image Processing with OpenCV: OpenCV, an open-source computer vision library, is used to process images captured by an infrared camera. The system processes images at a rate of 20 frames per second, ensuring real-time monitoring of the driver's face.
3. Facial Landmark Detection with dlib: The dlib library is employed for facial landmark detection. It identifies and tracks specific points on the driver's face, such as the eyes and mouth, which are crucial for drowsiness detection.
4. Distraction Detection: The system has the capability to detect driver distractions. When a distraction is detected, it immediately triggers an audio alert and displays a warning on the screen to alert the driver to refocus their attention on the road.
5. Admin Module: The system includes an admin module that serves as a central control and data storage hub. The admin module allows administrators to save data related to each driver, including their facial images, which serves as a crucial step for driver validation.
6. Driver Validation: To ensure accurate drowsiness detection, the system validates the identity of the driver. Since there may be more than one person in the frame, this step is vital. By comparing the facial landmarks detected with the stored facial images of authorized drivers, the system confirms the driver's identity.
7. Drowsiness Detection Parameters: The system employs parameters such as the Eye Aspect Ratio (EAR) and Mouth Aspect Ratio (MAR) to detect drowsiness. Changes in these ratios indicate drowsy behavior, such as drooping eyelids or prolonged mouth opening.
8. Alert Generation: Once drowsiness is detected and the driver's identity is validated, the system generates an alert. This alert is sent to the admin dashboard, marking the occurrence with a timestamp and location data. It also triggers an emergency alert for the driver.
9. Utility in Business and Personal Fields: The system's capabilities extend beyond road safety; it also has applications in business and personal contexts. For businesses, it can help monitor driver behavior and ensure employee safety, while in personal use cases, it provides a means for loved ones to be aware of the driver's condition and location

A. Drowsiness Detection Conditions

1. If the eyes of drivers are closed for a definite amount of time, then it is considered that driver is drowsy and corresponding audio alert and screen warning is given to make the driver alert.
2. If the mouth of driver is observed to be open for the specific amount of time, then it can be concluded that the driver is yawning and an appropriate audio alert and screen warning is given to the driver.
3. If the driver’s eyes are not on the road and the driver is distracted, then an appropriate audio alert and screen warning is given.

Fig. 1 System flow

IV. PROPOSED MODEL

Fig. 2 Proposed model

V. STUDY ANALYSIS

A. Dataset Analysis

This report provides a comprehensive comparison of popular face datasets in the field of computer vision, highlighting their data characteristics, efficiency, applicability, and additional considerations. The datasets discussed include:

1. 300W Dataset: A small dataset with 689 images, focusing on facial landmark detection with 68 annotated landmarks per face.
2. LFW Dataset (Labeled Faces in the Wild): Contains 13,233 images labeled with individual identities, making it suitable for        face recognition tasks.
3. CelebA Dataset: A vast dataset with 202,599 celebrity images, annotated for attributes like age, gender, facial expressions, and accessories, ideal for attribute-based facial analysis.
4. FDDB (Face Detection Data Set and Benchmark): Comprising 2,845 images with face detection bounding boxes, it serves as a benchmark for evaluating face detection algorithms.
5. AFLW (Annotated Facial Landmarks in the Wild): This dataset includes 25,999 images with 194,748 facial landmarks, making it valuable for robust face detection tasks and facial landmark analysis.
6. MultiPIE Dataset: A large dataset with 750,160 images, offering diversity in pose, illumination, and expression, suitable for face recognition and facial expression analysis.
7. IMDBWIKI Dataset: Contains 523,755 labeled identities.

The report discusses the efficiency of these datasets, emphasizing that the choice should align with the specific computer vision task and the available computational resources. Larger datasets like LFW and CelebA offer extensive data for tasks like face recognition, while smaller datasets like 300W and AFLW are better for precision-oriented tasks like facial landmark detection.

Fig. 3 Comparison of different datasets

The applicability of each dataset is considered, with 300W being suited for landmark detection, LFW for face recognition, CelebA for attribute-based analysis, and FDDB and AFLW for face detection. MultiPIE is highlighted as a versatile choice for multiple tasks due to its size, diversity, and detailed annotations. Additional considerations are discussed, including diversity, bias, and privacy in dataset collection and annotation. The importance of ethical and privacy standards in facial data collection is emphasized.

The report also provides a detailed comparison table, including data sources, characteristics, preprocessing requirements, sensor modalities, and evaluation metrics for the datasets. It mentions performance benchmarks, challenges, cross-dataset evaluation, and real-world applicability for each dataset.

In conclusion, the MultiPIE dataset is recommended as a strong choice for various facial analysis tasks due to its size, diversity, and detailed annotations. The report suggests that researchers and practitioners carefully assess these factors to select the most suitable datasets for their specific needs.

The future directions section points out the need for more diverse and real-world representative datasets and the growing interest in real-time facial expression analysis datasets.

References to relevant research papers are provided for further information.

B. Algorithm Analysis

This report presents a comparative analysis of drowsiness detection algorithms designed for automotive safety. Define the criteria against which the algorithms were assessed. Common criteria might include accuracy, real-time performance, robustness to different conditions, and ease of implementation.

Fig. 4  Comparative analysis of algorithms

1) Eye and Mouth Analysis using 68 Coordinate System

Involves tracking and analyzing the movement of a person's eyes and mouth.

Utilizes a 68-coordinate system for precise tracking of facial features.

2) Steering Behavior Analysis

Focuses on monitoring a driver's steering wheel movements and behavior.

Helps in assessing drowsiness or erratic driving patterns.

3) Lane Departure Warning Systems (LDWS)

LDWS alerts drivers when their vehicle deviates from the lane without using turn signals. A safety feature that helps prevent accidents due to unintentional lane drifting.

4) Heart Rate and Electrocardiogram (ECG) Analysis

Involves monitoring a driver's heart rate and ECG data to detect signs of drowsiness or stress.

5) Fusion of Above Algorithms

Combining multiple algorithms, such as eye and mouth analysis, steering behavior analysis, LDWS, and heart rate/ECG analysis to improve drowsiness detection.

1. Multimodal Eye-Mouth and Steering (MEMS) Algorithm: A fusion algorithm that combines eye and mouth analysis with steering behavior analysis to enhance drowsiness detection accuracy.
2. Multimodal Eye-Mouth and LDWS (MELD) Algorithm: A fusion algorithm that combines eye and mouth analysis with Lane Departure Warning Systems to improve drowsiness detection.
3. Multimodal Eye-Mouth and Heart Rate (MEMHR) Algorithm: A fusion algorithm that combines eye and mouth analysis with heart rate/ECG analysis to enhance drowsiness detection.
4. Multimodal Steering and LDWS (MSLD) Algorithm: A fusion algorithm that combines steering behavior analysis with Lane Departure Warning Systems to improve drowsiness detection by considering both steering and lane-related factors.

The recommendation for selecting a drowsiness detection system algorithm emphasizes the importance of tailoring the choice to the specific application's requirements. In general, combining multiple algorithms through fusion can yield the best performance in terms of accuracy, sensitivity, specificity, F1 score, and response time. For instance, a fusion algorithm that integrates eye and mouth analysis, steering behavior analysis, and lane departure warning systems can achieve high accuracy, particularly suitable for self-driving cars and aircraft.

However, it's acknowledged that fusion algorithms may be more complex and costly to implement. Less demanding applications, such as drowsiness detection in standard vehicles, may suffice with a simpler algorithm, like eye and mouth analysis. An additional factor to consider is the intrusiveness of the system, with some methods being more intrusive than others, like EEG-based systems compared to camera-based ones.

The core advantages of fusion algorithms include improved accuracy by leveraging the strengths of multiple algorithms, a reduction in false positives by filtering out such cases, increased robustness in noisy and changing environments, and enhanced flexibility for adapting to various applications.

For example, a fusion algorithm for detecting drowsy drivers in cars could combine data from a camera, a steering sensor, and a heart rate sensor. The camera tracks eye and mouth movements, the steering sensor monitors steering behavior, and the heart rate sensor measures the driver's heart rate. The fusion algorithm then uses this data to identify signs of drowsiness, like changes in eye movements, steering behavior, or heart rate, and can issue a warning to the driver.

In conclusion, fusion algorithms offer several advantages over individual algorithms, including higher accuracy, reduced false positives, improved robustness, and increased adaptability, making them a preferred choice for drowsiness detection in various applications. Different fusion algorithms can be tailored to specific use cases by combining various algorithms in unique ways.

For making this analysis on the algorithms we have taken references from this sources:

A Comparison of Drowsiness Detection Algorithms for Drivers by M. A. Iqbal, M. S. Hossain, and M. S. Uddin (2018)

A Comparative Study of Drowsiness Detection Algorithms by S. Kumar and M. M. Rathore (2019)

Comparison of Machine Learning Algorithms for Recognizing Drowsiness in Drivers using Electroencephalogram (EEG) Signals by H. A. Al-Fahoum and A. A. Al-Shammari (2020)

Algorithms Comparison in Drowsiness Detection by S. Dhakal, S. Tiwari, and P. Acharya (2021)

A Survey on Drowsiness Detection Systems: Algorithms, Challenges, and Future Directions by M. A. Iqbal, M. S. Hossain, and M. S. Uddin (2022)

VI. ACKNOWLEDGMENT 

I would like to express my deepest appreciation to Prof. Jignesh Patil, Assistant Professor at the Dept. of Computer Engineering, Rajiv Gandhi Institute of Technology, Mumbai, India, for his invaluable guidance and support throughout the research and writing of this paper. His expertise and insights were instrumental in the development of the drowsiness detection system discussed herein. His willingness to share knowledge and provide constructive feedback has contributed significantly to both the depth and quality of this study. His mentorship has not only enriched this work but has also greatly enhanced my own learning experience and professional growth.

Conclusion

In conclusion, our analysis of datasets and algorithms for a driver drowsiness system underscores the critical importance of dataset selection, algorithm performance, ethical and privacy considerations, real-world applicability, and the need for continuous innovation in the field. The choice of dataset significantly influences the accuracy and reliability of drowsiness detection algorithms, with data quality, size, diversity, and privacy considerations playing a crucial role. Algorithm selection must align with the specific drowsiness detection task, and ethical standards, including informed consent and PII protection, are paramount. We analysed and found out that for our project MultiPIE dataset is recommended as a strong choice for various facial analysis tasks due to its size, diversity, and detailed annotations and fusion algorithms offer several advantages over individual algorithms, including higher accuracy, reduced false positives, improved robustness, and increased adaptability which will be well suited for our project.

References

[1] National Highway Traffic Safety Administration. Drowsy Driving. Available online: https://www.nhtsa.gov/risky-driving/drowsy-driving (accessed on 10 May 2021). [2] Ramzan, M.; Khan, H.U.; Awan, S.M.; Ismail, A.; Ilyas, M.; Mahmood, A. “A survey on state-of-the-art drowsiness detection techniques”. IEEE Access 2019, 7, 61904–61919. [Google Scholar] [3] Sikander, G.; Anwar, S. Driver fatigue detection systems: “A review. IEEE Trans. Intell. Transp. Syst.” 2018, 20, 2339–2352. [Google Scholar] [CrossRef] [4] Knapik, M.; Cyganek, B. “Driver’s fatigue recognition based on yawn detection in thermal images. Neurocomputing”,2019, 338, 274–292. [5] Li, Z.; Li, S.E.; Li, R.; Cheng, B.; Shi, J, “Online detection of driver fatigue using steering wheel angles for real driving conditions.” Sensors 2017, 17, 495. [Google Scholar]. [6] Swati Gade,Kshitija Kamble, Aishwarya Sheth, “Driver Drowsiness Detection Using EAR”, IJRASET,2022 [7] K. N. Hande,“Driver Drowsiness Detection and Alert System”, Research gate publication, doi: https://doi.org/10.32628/CSEIT2173171,2021 [8] Nagamani N.P.,“Driver Drowsiness Detection”,IJERT,2020 [9] Cherie vartika Stephen,”drowsiness detection system for drivers using 68 coordinate system”,KIIT,Deemed to be university,March 2022 [10] Kyong Hee Lee, Whui Kim, Hyun Kyun Choi, Byung Tae Jan. “A Study on Feature Extraction Methods Used to Estimate a Driver’s Level of Drowsiness”, IEEE, February 2019. [11] Fouzia, Roopalakshmi R, Jayantkumar A Rathod, Ashwitha S, Supriya K, “Driver Drowsiness Detection System Based on Visual Features.” , IEEE, April 2018. [12] SaeidFazli, Parisa Esfehani, “Tracking Eye State for Fatigue Detection”, ICACEE, November 2012. Gao Zhenhai, Le DinhDat, Hu Hongyu, Yu Ziwen, Wu Xinyu, “Driver Drowsiness Detection Based on Time Series Analysis of Steering Wheel Angular Velocity”, IEEE, January 2017. [13] Bagus G. Pratama, IgiArdiyanto, Teguh B. Adji, “A Review on Driver Drowsiness Based on Image, Bio-Signal, and Driver Behavior”, IEEE, July 2017. [14] A. K. Singh, S. K. Singh and A. K. Singh, “Driver drowsiness detection system using image processing,” 2017 International Conference on Computing, Communication and Automation (ICCCA), Greater Noida, India, 2017, pp. 133-138, doi: 10.1109/CCAA.2017.8229874. [15] S. M. Pizer and E. P. Ambikairajah, “Driver drowsiness detection using EEG signals,” 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Prague, Czech Republic, 2011, pp. 1845-1848, doi: 10.1109/ICASSP.2011.5946806. [16] M. A. Al-Ghamdi and A. A. Al-Ghamdi, “Driver drowsiness detection system using machine learning,” 2016 IEEE International Conference on Industrial Technology (ICIT), Taipei, Taiwan, 2016, pp. 2140-2145, doi: 10.1109/ICIT.2016.7475063. [17] S. Kaur and R. Kaur, “Driver drowsiness detection system using deep learning,” 2021 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE), Vellore, India, 2021, pp. 1-5, doi: 10.1109/ic-ETITE51219.2021.9587519. [18] S.-H. Lee and J.-H. Kim, “Real-Time Driver-Drowsiness Detection System Using Facial Features,” IEEE Access, vol. 7, pp.118727-118738,2019,doi: 10.1109/ACCESS.2019.2936663.

Copyright

Copyright © 2024 Vishal Pandey, Jainam Parmar, Samay Rangari, Sonam Yadav, Prof. Jignesh Patil. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.