Facial Expression Based Restaurant Scoring System

Abstract

Automated becoming unmanned restaurants have become more and more common in recent years. Customers\' opinions cannot be directly seen in order to learn about their encounters with the idea of a restaurant because staff members are not there. This research provides a rating system based on already trained convolutional neural network (CNN) algorithms for face expression recognition. It consists of an AI server that has been trained, a web server, and an Android mobile application. Ratings are intended for both the cuisine and the setting. At the moment, the scoring system offers three expressions: neutral, disappointed, and satisfied.

Introduction

I. INTRODUCTION

The Internet of Things (IoT), artificial intelligence (AI), and the fields of information and communication technology (ICT) have all developed quickly in recent years, leading to an increase in the number of applications based on these advancements. The popularity of automated as well as unmanned restaurants is still rising as a result of his trend. Particularly throughout Japan and Taiwan, the amount of automated or unmanned restaurants operating successfully is expanding. Two such eateries are illustrated as examples in Fig. 1. An Android mobile application for patrons and a web application to restaurant employees make up the online smart restaurant management system that Saeed et al. proposed. The customer can order dishes and pay for their bill using an interactive menu, among other features, provided by the mobile application. The management of unmanned restaurants finds it challenging to gauge how patrons will react to the concept and cuisine because there is no staff on hand. Furthermore, accurate targeted marketing cannot be achieved by estimating the ages and gender distribution of its clientele. Because they only include a portion of the opinions of users, rating services like Google and TripAdvisor only partially address this issue.

II. RELATED WORK

1. The Expression Of The Emotions In Man And Animals

A contentious novel for many of its contemporaries, The Expression was an original work. It was the last instillment in a series that began with On the Origin on Species and controversially reached its apex the year before its conclusion with the Descent of Man. The first, published in 1859, introduced Darwin's notion of fall in modification via natural selection at plants and animals. It postulated that within a population, variations that arise at random and offer a breeding as well survival advantage tend to be preserved, eventually leading to divergence.

2. Recognizing Action Units For Facial Expression Analysis

The majority of computerized expression analysis algorithms aim to identify a limited range of typical expressions, including fear, surprise, rage, and enjoyment. However, these prototypic expressions are not very common. A person's intents and feelings are typically conveyed through subtle changes to one or a few distinct facial characteristics. In this work, we create an automatic face analysis (AFA) system to analyte the facial emotions in a roughly frontal-view face image sequence, taking into account both transient (deepening of facial furrows) and permanent (brows, eyes, and mouth) facial features. Unlike a few prototypic expressions, the AFA system recognizes fine-grained variations in facial expression into action units (AU) of the Facial Action Coding System (FACS). For tracking and modelling the different facial features, such as lips, eyes, brows, cheeks, and furrows, multistate face and facial component models are suggested. Extensive parametric descriptions of the facial features are extracted during tracking. By using these characteristics as inputs, it is possible to identify a set of action units—neutral expression, six upper face AU, and ten lower face AU—whether they appear separately or in combination. The average recognition rates for the upper face AU and lower face AU, respectively, are 96.4 percent and 95.6 percent, respectively, if neutral expressions are eliminated from the analysis.

Using separate image databases that have been gathered and FACS-coded for ground truth by several research teams, the generalizability of the method has been evaluated.

3. Constants across cultures in the face and emotion.

looked at the possibility that there are universal facial expressions for different emotions. The fact that members of literate cultures linked similar emotion concepts to similar facial behavior’s in recent studies does not prove that at least some emotional facial expressions are universal. This is because the cultures under comparison were all exposed to similar mass media presentations of facial expression, which may have helped the citizens of each culture to recognize the distinctive facial expressions of other cultures. In New Guinea, data were collected by telling a story to 342 Ss, showing them a set of 3 faces, and asking them to choose the face that best reflected the emotion appropriate to the story. This was done to demonstrate that members of a preliterate culture with little exposure to literate cultures would associate the same emotion concepts with the same facial behaviors as do members of Western and Eastern literate cultures. Members of the Fore linguistic-cultural group, Ss belonged to an isolated Neolithic material culture until a period of twelve years ago. Findings offer proof that the theory is correct.

4. Strong Evidence For Universals In Facial Expressions: A Reply To Russell’s Mistaken Critique

In addition to misrepresenting what universality is and how it is demonstrated in earlier research, J. A. Russell (1994) neglects to take into account or disclose data that contradict his claims. In response to Russell's main query on the prevalence of forced-choice formats in earlier research, new statistics are presented that conclusively address the issue. This article further demonstrates the futility of his several other criticisms on other elements of the layout of the studies of literate cultures. Russell presents an incorrect criticism of preliterate civilizations since he withholds information about the methods and conclusions of those who conducted the research on preliterate subjects. Taking care of all of Russell's complaints, my research reveals that the data from both literate and preliterate civilizations is overwhelming in support of universals in facial expressions.

III. METHODOLOGY

1. Face Detection: Face detection or localization is an important step for image classification since only the principal component of face such as nose, eyes, mouth are needed for classification. Face detection algorithms can be broadly classified into feature, knowledge, template and appearance-based methods. Our proposed system uses Viola Jones object detection algorithm for face localization which comes under feature-based classification. Viola Jones object detection algorithm uses Haar feature based cascade classifiers. The Haar Cascade classifier is extremely important element of the face detection. The presence of the features in any of the input image is determined by the Haar features.
2. Facial Expression Recognition classification: After learning the deep features, the final step of FER (Facial Expression Recognition) is to classify the given face into one of the basic emotion categories. Unlike the traditional methods, where the feature extraction step and the feature classification step are independent, deep networks can perform FER in an end-toned way. Specifically, a loss layer is added to the end of the network to regulate the back-propagation error; then, the prediction probability of each sample can be directly output by the network. In CNN, SoftMax loss is the most common used function that minimizes the Cross entropy between the estimated class probabilities and the ground truth distribution.
3.  Convolutional neural network (CNN): CNN has been extensively used in diverse computer vision applications, including FER. At the beginning of the 21st century, several studies in the FER literature found that the CNN is robust to face location changes and scale variations and behaves better than the multilayer perceptron (MLP) in the case of previously unseen face pose variations, employed the CNN to address the problems of subject independence as well as translation, rotation, and scale invariance in the recognition of facial expressions.

Conclusion

In this research, a facial expression recognition-based restaurant assessment system is proposed. When a direct request is made to the consumer at the conclusion of the visit, a greater variety of customer opinions can be obtained than with independent scoring platforms. However, because just two scores are asked for, this is simply a rough understanding. The integration of contemporary technology, facial expression detection, in a light-hearted manner for the scoring system piques customers\' interest in providing a rating. Subsequently, the system might be integrated with current text-based rating systems, such as Google Rating, to combine the benefits of both. An additional advancement might result in a technology that allows patrons to rate the restaurant without using their hands. This requires making sure that the face expression recognition accuracy is high enough. Adding a speech recognition function to the image-based rating system is also a good idea. The user can orally share his thoughts and feelings or offer suggestions for enhancements, just like Google ratings do.

References

[1] Ekman P, Friesen WV. Constants across cultures in the face and emotion.” Journal of personality and social psychology. 1971; 17(2):124-129. [2] Tian YI, Kanade T, Cohn JF. Recognizing action units for facial expression analysis, IEEE Transactions on pattern analysis and machine intelligence. 2001; 23(2):97-115. [3] Darwin C, Prodger P. The expression of the emotions in man and animals. Oxford University Press, USA, 1998. [4] Liu, X., Li, H., & Hu, F. (2019). A hybrid approach for improving customer feedback analysis in the restaurant industry. Industrial Management & Data Systems. [5] Matsumoto D. More evidence for the universality of a contempt expression,” Motivation and Emotion, vol. 1992; 16(4):363-368. [6] Bartlett, M. S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., & Movellan, J. (2006). Recognizing facial expression: machine learning and application to spontaneous behavior. In Computer Vision and Pattern Recognition Workshop (CVPRW\'06), vol. 2, pp. 76-76. IEEE. [7] Ekman, P., & Friesen, W. V. (1978). Facial Action Coding System. Consulting Psychologists Press. [8] Buhalis, D., & O\'Connor, P. (2005). Information technology in tourism: Implications for the design of online hotel marketplaces. Information Technology & Tourism, 7(2), 73-91. Kwun, D. J. W., & Lee, J. S. (2018). A review of restaurant satisfaction studies for the past decade. Journal of Hospitality Marketing & Management, 27(7), 776-805. [9] Lerner, J. S., Li, Y., Valdesolo, P., & Kassam, K. S. (2015). Emotion and decision making. Annual review of psychology, 66, 799-823. [10] Solnais, C., Andreu, L., & Joyeau, A. (2013). The role of emotions in explaining consumer satisfaction and future behavioural intentions: The case of an experiential service. Journal of Retailing and Consumer Services, 20(6), 579-586. [11] Jack RE. Garrod OG, Yu H, Caldara R, Shins PG. Facial expressions of emotion are not culturally universal, Proceedings of the National Academy of Sciences. 2012; 109(19):7241–7244. [12] Ekman P. Strong evidence for universals in facial expressions: a reply to Russell’s mistaken critique, psychological bulletin. 1994; 115(2)268–287.

Copyright

Copyright © 2024 D. Sumanth, G. Sai Chandar, B. Sri Sai Dheeraj, D. Madhavi. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.