Driver Drowsiness Detection using Opencv and Raspberry Pi

Abstract

Driver drowsiness detection is a step to reduce the rate of road accidents that happen around the world. It integrates the technology of machine learning with computer vision for detection and alert generation. Year by year the degree of fatalities and road accidents has increased due to driver drowsiness and fatigue. The device is robust and cost-effective which can prevent road accidents in most cases. The device has a camera through which it monitors the driver’s eye continuously and processes the image for the sleep onset period. The images taken at random interval is passed through a Neural network that recognizes eye moments and classifies the sleep stage. If the driver is in the initial stage of falling asleep it can alert the driver to wake up and be attentive. If the driver is in a further stage of sleepiness the device will generate an alert to the respective emergency contacts that the driver is on the verge of sleep and they can able to alert the driver in any way. The emergency contacts can also get the location coordinates of the driver from the device\'s Gps tracking system. One’s inattentiveness driving or careless driving is not only a problem to himself but also to the others on the road, creating a devastating effect. So there is a level of prominence on every solution that can prevent road accidents. Not only the life of drivers but also the people who are dependent on them will have to face the consequences.

Introduction

I. INTRODUCTION

In the world, around 56% of major road incidents happen due to the inattentiveness of the driver. there are millions of people who use 4-wheelers daily around the globe. A survey by The Hindu media stated that around 40% of the accidents take place due to the driver feeling asleep as many will drive for long hours and it is also a source of livelihood for many rental drivers.

The most possible scenarios for the accidents are due to some shorter sleep time on before day, the co-passenger’s inactiveness, and dire tiring situation, especially during night times. The sensation to take rest and drowsiness will reduce the driver’s level of vigilance and is more prone to cause a road accident. A fatigued driver is a significant part of any road accident.

The device is a contactless and stable capturing device that can be mounted easily on the deck of the steering wheel. It is a portable device compactly designed to reduce the cost per unit and energy consumption. 

Driver drowsiness detection is a system made to alert the driver that he is at the onset of falling asleep, by using the eye moments of the driver. It also has the ability to share the current location and alert the emergency contact numbers so as to make the driver attentive or to wake them from falling asleep. The device can be powered up by the vehicle itself or we can use an external power source to keep it running. The device has a camera module integrated with raspberry pi. The camera continuously monitors the eye moments of the driver and detects the sleeping stage and onset of the sleep period to alert the driver. If the driver doesn’t respond even after the alert then an alert or notification will be sent to the concerned emergency contact to help the driver from falling asleep. The device contains an inbuilt gsm module to get the internet connectivity and can use the vehicle GPS, if it exists,  forgetting the vehicle coordinates to send to the emergency contacts. In the case of advanced solutions, we can integrate the hand steering with an electric pulsator or a vibrating device to give some jerks on the hand of the driver. Many advancements can be possible in this case so that the device can be upgraded according to the needs and practical research.

II. LITERATURE SURVEY

Drowsiness detection for a driver while at work is one of the most important aspects. There are various methods - Some of them are already present in the market and while the research process is still going on few other methods. Every method was built using a camera module that takes the real-time images of the object (the driver is the object ). All the methods used different ideas and technologies to detect driver drowsiness. Few used the cloud to receive the captured images from the camera module. The deep learning model which is capable of detecting closed eyes or semi-closed eyes was deployed on the cloud, from the cloud using services the alert would be sent to emergency contacts. But in these types of models there are a few disadvantages of connectivity, Considering the vehicle in a scenario where is less internet connectivity, the connection between the cloud and camera module would be lost and hence this creates a big flaw in the drowsiness detection system.

OpenCV is a library used in programming to support functionalities of computer vision. OpenCV has a modular type of structure which means it has many shared and static libraries. A few necessary modules it has are core functionality, Image processing, Video analysis, camera calibration and 3D reconstruction, object detection, and many more. Objection detection, Image processing are two important modules to be used in drowsiness detection.

Raspberry Pi is a card-sized mini-computer that provides a high-speed processor, and several ports to attach memory. Raspberry Pi is an awesome device while working with a camera module. Images really require high computing resources to get processed and that makes raspberry pi a strong competitor for our application. Raspberry pi is also great while working with the internet of things.

A GSM sensor used GSM ( Global System for Mobile communication ) mobile communication technology to provide a wireless data link to a network. This method is also the default method as a mode of mobile communication.

III. IMPLEMENTATION

Our system uses a microprocessor, which has a camera attached to it. The camera is used for continuous revival of the photo frames of the person driving the car. Using this real-time video, we will use OpenCV software in the raspberry pi to determine whether the person is drowsy or not.

A. Raspberry pi with Camera

We have used raspberry pi as the microprocessor and a camera of resolution 1080p. Connect the raspberry pi and the camera, locate the port for the camera module in the raspberry pi, and pull the plastic port clip's edges gently up. Insert the camera module's flat cable (Note: Make sure the cable is pointing in the right direction. The jack connector and USB ports are on the blue side of the cable), and replace the plastic clip in its original location. Now, the camera and the rpi are connected.

B. OpenCV in Raspberry pi

After connecting the hardware components, we have to configure the software and write a program for it. We are going to use OpenCV and dlib library for recognizing the “Facial Landmarks”. Facial landmark can give us the position’s of eyes, nose, mouth, etc.

As you can see in the above picture there are 6points located on the eye, if there is some distance between the points then the driver is awake, if the distance is negligible or close to zero, the driver is sleepy. If the distance between the points is more than

C. Buzzer and Message

So, what after detecting the driver is sleepy? We will ring a buzzer to wake him up and an automated message will be sent to one of the emergency contacts of the driver. So that they will contact the person and will make sure that he will not doze off again.

We can send the message using the GSM module, we have to enter the contacts of the person manually. The GSM module sends the messages using the “AT” commands, meaning attention.

GSM modules have the following features:

1. Read, write and delete SMS messages.
2. Send SMS messages.
3. Monitor the signal strength.
4. Monitor the charging status and charge level of the battery.
5. Read, write and search phone book entries.

IV. OUTPUTS AND RESULTS

This part provides our experimentation results, the flowwork will be as shown in the FIg.

A. Eye Detection

As said before, we use OpenCV software, dlib package to detect the facial landmarks and using those landmarks we can detect the eye and the blink rate of it.

So, using these facial landmarks we will take the points of the eye. As said before, we will measure the distance between them. If the distance between those points is very less, then the driver is sleepy or the eyes is closed. The fig are as shown below.

B. Algorithm

1. Import the required libraries and the facial landmarks data.
2. Live feed the frames of the driver using cv2 library.
3. Mark the facial landmarks of the eye on the frames and calculate distance between them.
4. If the distance between them is below 0.2, the driver is sleepy.

C. Buzzer and GSM module

After detecting the driver is sleepy, we have to wake him up. We can use the buzzer attached to the rpi or we can use any of the system sound present in the rpi. Along with that, we will message to the emergency contacts, so that they will contact the driver and wake him up.

*SHould keep the picture of mobile receiving the message*

Conclusion

Hence, This paper outlines a preventive step to reduce road accidents. Drowsiness and driver fatigue has been most prominent cases in the majority of road accidents. As there is a rapid increase in technology and research, there is every possibility for an advanced and optimized upgrade to the device. Integration of technologies can be better helpful for newer innovative solutions. Thus our device integrates the capabilities of Neural networks with Image reception and was successful in providing alerts with the help of IoT.

References

REFERENCES [1] A. Malla, P. Davidson, P. Bones, R. Green and R. Jones, \"Automated Video-based Measurement of Eye Closure for Detecting Behavioral Microsleep\", in 32nd Annual International Conference of the IEEE, Buenos Aires, Argentina, 2010. [2] P. Viola and M. Jones, \"Rapid Object Detection using a Boosted Cascade of Simple Features\", in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001. [3] R. Lienhart and J. Maydt, \"An Extended Set of Haar-like Features for Rapid Object Detection\", in Proceedings of the IEEE International Conference on Image Processing, 2002. [4] OpenCV. Open Source Computer Vision Library Reference Manual, 2001. [5] S. Vitabile, A. Paola and F. Sorbello, \"Bright Pupil Detection in an Embedded, Real-time Drowsiness Monitoring System\", in 24th IEEE International Conference on Advanced Information Networking and Applications, 2010. [6] B. Bhowmick and C. Kumar, \"Detection and Classification of Eye State in IR Camera for Driver Drowsiness Identification\", in Proceeding of the IEEE International Conference on Signal and Image Processing Applications, 2009. [7] N. Otsu, \"A Threshold Selection Method from Gray-Level Histograms\", IEEE Transactions on Systems,Man and Cybernatics, pp. 62-66, 1979. [8] N. Wang and D.-Y. Yeung, \"Learning a deep compact image representation for visual tracking\", Proceedings of the 26th International Conference on Neural Information Processing Systems., vol. 1, pp. 809-817, 2013. [9] Varsha E Dahiphale, Satyanarayana R, A Real-Time Computer Vision System for Continuous Face Detection and Tracking, IJCA, Volume 12 Number 18, July 2015. [10] SaeidFazli, Parisa Esfehani, Tracking Eye State for Fatigue Detection, ICACEE, November 2012. Gao Zhenhai, Le DinhDat, Hu Hongyu, Yu Ziwen, Wu Xinyu, Driver Drowsiness Detection Based on Time Series Analysis of Steering Wheel Angular Velocity, IEEE, January 2017. [11] Dwipjoy Sarkar, Atanu C, Real Time Embedded System Application for Driver Drowsiness and Alcoholic Intoxication Detection, IJETT, Volume 10 Number 9, April 2014. [12] Kyong Hee Lee, Whui Kim, Hyun Kyun Choi, Byung Tae Jan. A Study on Feature Extraction Methods Used to Estimate a Drivers Level of Drowsiness, IEEE, February 2019. [13] Tianyi Hong, Huabiao Qin, Drivers Drowsiness Detection in Embedded System., IEEE, December 2007.

Copyright

Copyright © 2022 Dr. K. Venkat Reddy, Dr. T. Ramaswamy, Sai Harshith Kumar, Channa Prajeeth, Choudhary Vineet. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.