Driving Alert System based on Facial Expression Recognition

Abstract

Inattention is one of the major factors and causes of many road accidents and unforeseen crashes. As a result, it is vital to develop an automatic warning system for drivers, which will send timely warning signals to them. Based on the driver\'s facial expressions, this issue involves determining the driver\'s mental state. In applications such as driver warning systems, automated facial emotion recognition has become a recent development in the image processing world. There are existing methods for recognizing facial emotions regardless of whether the signal is noisy or imperfect data, but ultimately it lacks accuracy. It is also useless and not getting proper output in dealing with spontaneous feeling, and recognition. The proposed approach develops a driver warning system that extracts the facial expressions based on a novel efficient Local Octal Pattern (LOP) and effectively recognizes the facial expressions based on Deep Neural Networks, Convolutional Neural Networks (CNN). The LOP feature map provides as an input to CNN and guides in the selection of CNN learning data thereby improving and further enhancing the understanding and learning of CNN. It also has an ability to recognize both natural and spontaneous feeling, considered input as a image as well as video. The experimental results considering YawDD dataset indicates that the proposed system has been efficiently evaluated by considering the with metrics such as Precision, Recall and F-Score and thereby it is observed and inferred that the proposed system obtained a high recall rate of 96.09% in comparison with the other state-of-the-art methods

Introduction

I. INTRODUCTION

The high degree of mental complication of a person driving the vehicle along with the consumption of alcohol and over-speeding is the most important cause of occurrence of majority of traffic accidents and unforeseen crashes. According to statistics from related departments, it is observed that driving fatigue condition accounts for the largest proportion of road traffic accidents. Many drivers who are in the fatigue state may experience sleepiness, short loss of consciousness and that they usually don’t perceive that they are within the tired and drowsy state. The driver driving for duration greater than eight hours a day, excessive physical activity, lack of sleep, etc may be the cause of fatigue in drivers. There arises weakness in driver’s mental stability and he will not be able to handle traffic problems in the road accurately. Driving fatigue is authenticated and confirmed by one of two aspects of consideration: First, before the fatigue behavior is found, the driver driving the vehicle cumulatively within 24 hours is being calculated. If it is more than eight hours, it is automatically considered as fatigue though

The driver doesn't feel tired. Second, the fatigue of driving, even though it is not equivalent to the usual eight hours of driving daily, is confirmed by an examination of the driving fatigue. And if the driver's observation, which has been confirmed as a result of a lack of sleep, driving fatigue and sleepiness, is confirmed by the following reasons, such as a lack of physical hard work, he cannot accurately process traffic-related issues. The use of a single method of testing, which is used only for social control, is not comfortable, as it is not possible to rely on a completely different technical method of testing. This is due to limited human material resources and driver unconsciousness during fatigue driving

The key contributions of this paper include:

1. The design of a Local Octal Pattern-Convolutional Neural Network (LOP-CNN) method for an efficient Driver warning System using Deep Learning based Facial Expression Recognition.
2. CNN based feature extraction minimizes the semantic gap and improves the overall efficiency by using Facial Expression Recognition

Objective

a. Reduce the Rode Accident.

b. Compared to existing system traditional attendance marking system, this system reduces the workload of people.

c. The objective of face recognition is, from the incoming image, to find a series of data of the same face in a set of training images in a database. The great difficulty is ensuring that this process is carried out in real-time, something that is not available to all biometric face recognition software providers. .

d. Here faces will be recognized using face recognition algorithms.

e. The processed image will then be compared against the existing stored record and then attendance is marked in the database accordingly

II. METHODOLOGY

A. Face Detection

For the images or videos captured by the camera, the Viola-Jones Face Detector method is used to detect and locate the faces in the image or video. By using this method, the training becomes slow, but the detection is fast. This algorithm makes use of Har basis feature filters and does not use multiplications. Detection is usually going to happen in the detection window. A minimum and maximum frame size is optimally chosen, and for each of the recognized sizes a sliding step size is eventually chosen, after which the detection frame is moved accurately across the corresponding face image.

B. Face Feature Extraction

 In the feature extraction phase, for each of the detected facial components, 7X5 grids are extracted, in which each grid is represented as a square patch [25]. Using this technique, the same as projected in [25], there will be a total of 175 grids from five significant elements of the face, including two mouth corners, nose tip and two eyes, on the aligned face image. A face image patch is rooted out and extracted from each grid and the LOP feature descriptor [26] is accordingly computed and evaluated as local features. The CNN is then trained and map with the LOP feature vector, which serves as an input to the CNN.

1. LOP – based Face feature Recognition.

After face alignment, the feature extraction in the face is obtained with LOP. The LOP algorithm successfully encodes the information between the center pixel and the reference pixel, purely based on the direction of the center pixel. In the LOP, encoding information is calculated for all directions (horizontal, vertical and diagonal) of pixel values using first-order derivatives for each center pixel in the given face image. From the pixels, 8-bit octal pattern is obtained for each center pixel in the given Image . The octal patterns are divided into eight parts from the calculate binary Pattern

2. CNN-based Facial Expression Recognition

One of the best known optimal representative means of network structures in the field of deep learning is the Convolution neural network It has unanimously gained prominence in the area of facial image recognition and classification. The resulting data are efficiently processed by Convents in the bundled form of multiple set of arrays. The CNN is capable of taking the raw information of an image as an input there by passing the feature extraction and reconstruction procedures as part of the standard learning algorithms.

III. LITERATURE REVIEW

A Theoretical Concept Behind Study

The development of a has been made some efforts The system that will detect the driver's drowsiness Based on certain factors. The factors, such as the movement of the head, heart rate, grip and quality of the steering wheel The movement of the feet on the path marked by the path Meticulously recorded to detect drowsiness. The eyes of the eye Driver was concentrated to examine the drowsiness using The system to detect the drowsy drivers. Based on Drowsiness parameters for detection of drowsiness Techniques are classified into two types, namely the invasive and the intrusive. Methods of detection and non-interactive methods. The Distinction is based on the condition that the instrument is Whether the driver is connected to the driver For facial recognition, especially for appearance-based methods Geometric methods based on the principles of the geometric system and neural networks based on methods. The classic way to look at the face recognition faces geometry. The human face is represented by the human face And characterized by an optimal set of facial landmark points The approach based on the geometrical structure of the approach. The location and the location The angle and angle of the facial structure determines the shape of the facial structure. The distance between the bones of the face. To classify the input Face, the vectors of features that would adequately represent the face are The classifier will receive the fed-in data from the classifier. In order to accurately locate the the face must be marked, the face must be scanned by a facial point detector. It's a difficult task to find out. The characteristics have been unanimously approved The values of the relative pixel gray values were derived from the values of the Face image for the purpose of the appearance method. The pixel The intensity of the light is determined by the intensity of the light that is being emitted. the image

B. Face Detection

For the images or videos captured by the camera, the Viola-Jones Face Detector method is used to detect and locate the faces in the image or video. By using this method, the training becomes slow, but the detection is fast. This algorithm makes use of Har basis feature filters and does not use multiplications. Detection is usually going to happen in the detection window. A minimum and maximum frame size is optimally chosen, and for each of the recognized sizes a sliding step size is eventually chosen, after which the detection frame is moved accurately across the corresponding face image 

Conclusion

1) An automatic driver warning system is proposed and developed to send timely warning signals to the drivers based on the recognized facial expressions to avoid road accidents and unforeseen crashes. 2) The driver warning system proposed extracts the facial expressions based on an efficient texture Local Octal Pattern (LOP) and recognizes the facial expressions based on Deep Neural Networks, i.e., CNN. 3) We applied SVM classification to the trained images to detect the difference between face images and trained Image and improve the categorized face images. 4) It has an ability to recognize both natural and spontaneous emotions from both image and video inputs. 5) The proposed system can be further extended to include different maneuvers to ultimately make the driving system capable of dealing with all types of vehicles, under varying driving environments and scenarios.

Copyright

Copyright © 2023 Prof. Dr.Mahendra Gaikwad, Sunil Balpande, Shubham Bhoyar, Shreyash Parse , Saurabh Sawarkar, Sanket Rudrakar, Tanmay Walke. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.