Facial Recognition with Face Regions by Machine Learning: Review

Abstract

This paper proposes a comprehensive study of facial recognition method supported a unique facial decomposition. First, seven regions of interest (ROI), representing the most parts of a face (left eye, right eye, between eyebrows, left eyebrow, right eyebrow, mouth and nose), are extracted by using facial landmarks detected by the IntraFace algorithm. Then, proposes a study of completely different native descriptors, such as LTP, Dynamic LTP, LBP and CLBP are used to extract features.

Introduction

I. INTRODUCTION

Face is one amongst the foremost necessary means of human communication. It plays a central role altogether social interactions. Facial expressions are non-verbal clues to emotions. Indeed, some facial muscles are specifically related to sure emotional states and permit, consistent with ekman [8] the expression of primary emotions (Sadness, Anger, Fear, Joy, Disgust and Surprise). These external signals specific express emotional state of a personal, and thus his intentions. In fact, 7% of the communication depends on verbal interaction, 38% represent tone and sound of voice, fifty fifth are articulated around gestures and expressions of the face consistent with Meharabian [18].

Automatic recognition of facial expressions is an interesting drawback that finds its interest in many fields such as eLearning and affectional computing [20], [15], [21]. When planning associate automatic face expression recognition system, three issues are considered: face detection, facial feature extraction, and classification of expressions. First, face acquisition could be a process stage to automatically find the face region within the input pictures. the next step is to extract and represent facial changes caused by facial expressions. Finally, the classification task permits inferring the facial expressions. According to the prevailing kinds of feature extraction, the facial expression recognition process are often normally divided into appearance-based and geometric-based strategies

II.  LITERATURE SURVEY

A latest local texture pattern, the characteristic descriptor constructed with the CLBP codes, and a compound local binary pattern have been presented for facial expression recognition. The proposed method of Faisal Ahmed [1] utilizes an encoding strategy that combines the magnitude information of the difference between two gray values with the original LBP pattern and thus provides increased robustness in many situations where LBP fails to generate consistent codes. Experimental results show that, the CLBP operator provides an effective and efficient approach for facial characteristic representation with high discriminative ability, which outperforms several existing characteristic representation

Facial expression may be a robust medium to precise one’s feelings and emotions. accurate detection of face expression can convey plenty of information a few person’s mood. The major barriers to accurate recognition are the presence of noise,illumination variation and occlusion. [2] proposes an effective appearance-based facial feature descriptor made with a brand new texture pattern, specifically the median ternary pattern (MTP) for face expression recognition. The proposed MTP operator encodes the texture data of a local neighborhood by thresholding against a local median gray-scale value and quantizing the intensity values of the neighborhood around every pixel into three totally different levels. The MTP codes generated for an image or image patch is then used because the feature illustration of the face expression image. The effectiveness of the proposed method has been evaluated victimisation pictures from the Cohn-Kanade (CK) Expression info. Classification was done using Support Vector Machine (SVM). Experimental results show raised accuracy in recognition rates once using the proposed approach in comparison to alternative common gray-scale based strategies.

In [3] a comprehensive study of however the HOG descriptor might be effectively exploited for facial features recognition purposes has been carried out. so as to perform investigation, a pipeline, usually utilized in FER analysis and equipped with a HOG descriptor, has been used as a baseline. Then, the classification performance, on a well known FER dataset, has been analyzed as a function of the HOG parameters setting.

Results highlighted a configuration of HOG parameters ready to match the particular aspects of facial expressions that enables a high classification performance capable to beat the performances of the leading state of the art approaches. so as to prove that the achieved HOG configuration is general, i.e. it doesn't depend upon the input file, the pipeline was tested on three further datasets. As a final step, experiments on video sequences gave proof that the proposed approach is additionally appropriate to be utilized in real-world application contexts.

The authors in [5] defined facial components from which they extracted HOG feature descriptors. Almost all the studies stated to used Support Vector Machine (SVM) as a classifier for facial expression recognition. The objective of their work is to recognize automatically the six basic emotions as well as the neutral state by applying appearance-feature methods like LBP and its variants to new specific face-regions.

Automatically detection facial expressions has become a vital research area. It plays a major role in security, human-computer interaction, and health care. Yet, earlier work focuses on posed facial expressions. In [7] authors proposed a spontaneous face expression recognition technique based on effective feature extraction and facial expression recognition for micro Expression analysis. In feature extraction, they used a histogram of orienting gradients (HOG) descriptor to extract facial expression|face expression} features. Expression recognition is performed by using a Support vector machine (SVM) classifier to acknowledge six emotions (happiness, anger, disgust, fear, sadness, and surprise). Experiments show promising results of the proposed technique with a recognition accuracy of ninety fifth on static pictures while eightieth on videos.

In [9], authors extensively investigate local features based facial expression recognition with face registration errors, which has never been addressed before. Their contributions are three fold. Firstly, they propose and experimentally study the Histogram of Oriented Gradients (HOG) descriptors for facial representation. Secondly, they present facial representations based on Local Binary Patterns (LBP) and Local Ternary Patterns (LTP) extracted from overlapping local regions. Thirdly, they quantitatively study the impact of face registration errors on facial expression recognition using different facial representations. Overall LBP with overlapping gives the best performance (92.9% recognition rate on the Cohn-Kanade database), while maintaining a compact feature vector and best robustness against face registration errors.

One of interpersonal human communication may be a face expression that is used to manifest emotions. If computers might recognize these emotional inputs, the interaction between humans and computers will be more natural. Currently, the video game is one among the applications that use face expression detection to extend natural interaction. Kinect may be a motion sensor for a game controller which will track the detected face using the Active appearance Model (AAM). AAM is that the technique that adjusts form|the form} and texture model during a new face once there's a variation of shape and texture compared to the training result. The aim of [10] is to detect facial {features|face expression} by observing the modification of key features in AAM using fuzzy logic. fuzzy logic is used to determine the present emotions based on previous knowledge derived from Facial Action writing (FACS). The experimental results on user generated dataset show that: (i) FACS doesn't give any information concerning the degree of muscle activation, therefore the degree of the key features need to infer from a specific dataset, (ii) detection accuracy of depending on the type of emotion itself.

Extraction of discriminative options from salient facial patches plays an important role in effective facial expression recognition. The correct detection of facial landmarks improves the localization of the salient patches on face pictures. [11] proposes using framework for expression recognition by victimisation appearance features of designated facial patches. a number of distinguished facial patches, depending on the position of facial landmarks, are extracted which are active throughout emotion evocation. These active patches are more processed to get the salient patches that contain discriminative features for the classification of every combine of expressions, thereby choosing totally different facial patches as salient for various combine of expression classes. The one-against-one classification methodology is adopted using these features. additionally, an automatic learning-free facial landmark detection technique has been proposed, that achieves similar performances like that of different state-of-art landmark detection methods, yet requires significantly less execution time. The proposed methodology is found to perform well systematically in several resolutions, hence, providing an answer for expression recognition in low resolution pictures. Experiments on CK+ and JAFFE facial expression databases show the effectiveness of the proposed system.

METHODOLOGY

The methods that are used to recognize the facial Expressions and neutral state. It consists of three main steps.

1. Extract the face regions named as region of interest (ROI).
2. Then do the experimentation with LBP, CLBP, LTP and Dynamic LTP to extract features in each ROI.
3. To recognize facial expression use SVM classifier to fed the feature vector.

A. Face Regions Extraction

To achieve the ROI extraction step, we start by detecting facial landmarks using IF framework. The IF algorithm can detect landmarks around the regions of eyebrows, eyes, nose, and mouth  using the Supervised Descent Method (SDM) [29]. SDM which is a supervised method thatis used to learns the optimize non-linear least squares (NLS) problems. It is a non-parametric shape model which allows to generalize better to untrained situations. During training, SDM learns a sequence of generic descent directions. In testing, SDM uses the learned descent directions to minimize the NLS objective function.

B. Local Binary Pattern

Ojala et al. [22] was introduced LBP operator. Although it is simplest to operate. In texture classification very good performances are demonstrated by LBP. The method analyzes the textures by encoding each pixel of the image into a string of binary values. This is performed by thresholding the differences between its value and the 3x3 neighborhood pixel values.

C. Compound Local Binary Pattern

There are some limitations with LBP descriptor. To overcome that issue, the CLBP was proposed by Ahmed et al. [1]. CLBP that combines the sign and the magnitude of the differences between the center and the neighbor gray values

D. Dynamic LTP

LBP operator is robust to monotonic illumination variation. However, it is sensitive to noise. To deal with this problem, Tan and Triggs [25] extended the original LBP formulation to a variant with three value codes, called LTP. LTP can take three values according to distance of the values of the neighboring pixels to the value of the central pixel.

E. Support Vector Machine

The input of the classifier is the concatenation of all the histograms calculated from each block of each ROI. Block decomposition of ROI allows extracting more local information. For the classification task, we employ linear Support Vector Machines.

IV. ACKNOWLEDGMENT

It is our proud privilege to release the feelings of our gratitude to every person who helped us directly or indirectly to conduct this research work. we express our heart full indebtness and owe a deep sense of gratitude to our professors for their sincere guidance and inspiration for completing this paper.

Conclusion

This paper, proposed Comprehensive study of a new facial decomposition for basic emotion states recognition. Based on landmarks of facial expression detected by IntraFace algorithm, seven regions of interest (ROI), corresponding to the main components of face, are first extracted to represent face image. A preprocessing stage is then applied on these ROIs for resizing and partitioning them into blocks, before performing feature extraction to build face feature descriptor. Finally, a multiclass SVM classifier is utilized to infer emotion state.

References

[1] F. Ahmed, H. Bari, and E. Hossain. Person-independent facial expression recognition based on compound local binary pattern (clbp). Int. [2] F. Bashar, A. Khan, F. Ahmed, and M. H. Kabir. Robust facial expression recognition based on median ternary pattern (mtp). In Electrical Information and Communication Technology (EICT), 2013 International Conference on, pages 1–5. IEEE, 2014. [3] P. Carcagn`?, M. Coco, M. Leo, and C. Distante. Facial expression recognition and histograms of oriented gradients: a comprehensive study. SpringerPlus, 4(1):1, 2015. [4] C.-C. Chang and C.-J. Lin. Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2(3):27, 2011.. [5] J. Chen, Z. Chen, Z. Chi, and H. Fu. Facial expression recognition based on facial components detection and hog features. In International Workshops on Electrical and Computer Engineering Subfields, pages 884–888, 2014. [6] J. de Andrade Fernandes, L. N. Matos, and M. G. dos Santos Aragao. ˜ Geometrical approaches for facial expression recognition using support vector machines. In Graphics, Patterns and Images (SIBGRAPI), 2016 29th SIBGRAPI Conference on, pages 347–354. IEEE, 2016. [7] M. M. Donia, A. A. Youssif, and A. Hashad. Spontaneous facial expression recognition based on histogram of oriented gradients descriptor. Computer and Information Science, 7(3):31, 2014. [8] P. Ekman. An argument for basic emotions. Cognition & emotion, 6(3-4):169–200, 1992. [9] T. Gritti, C. Shan, V. Jeanne, and R. Braspenning. Local features based facial expression recognition with face registration errors. In Automatic Face & Gesture Recognition, 2008. FG’08. 8th IEEE International Conference on, pages 1–8. IEEE, 2008. “PDCA12-70 data sheet,” Opto Speed SA, Mezzovico, Switzerland. [10] A. A. Gunawan et al. Face expression detection on kinect using active appearance model and fuzzy logic. Procedia Computer Science, 59:268–274, 2015. [11] S. Happy and A. Routray. Automatic facial expression recognition using features of salient facial patches. IEEE transactions on Affective Computing, 6(1):1–12, 2015. [12] M. Ibrahim, M. Alam Efat, S. Kayesh, S. M. Khaled, M. Shoyaib, and M. Abdullah-Al-Wadud. Dynamic local ternary pattern for face recognition and verification. In Proceedings of the International Conference on Computer Engineering and Applications, Tenerife, Spain,volume 1012, 2014.

Copyright

Copyright © 2022 Vijaya Ravindra Wankhade, Prof. Krutika K Chhajed. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.