Digital Content Recommendation System through Facial Emotion Recognition

Abstract

The use of facial emotion recognition technology in movie and music recommendation systems has become increasingly popular. In this paper, an all-encompassing evaluation of the present advancements in this ?eld is provided, and the possible advantages and di culties of utilizing this technology are deliberated. The ?rst part of the paper presents a brief overview of the di‰erent facial emotion recognition algorithms and how they work. This is followed by a review of the existing research on the use of these algorithms in movie and music recommendation systems. The second part of the paper discusses the potential bene?ts of using facial emotion recognition in movie and music recommendation systems. These include the ability to make more personalized recommendations based on the user\'s emotional state, as well as the potential for improving the overall user experience. The third part of the paper examines the challenges and limitations of using facial emotion recognition in movie and music recommendation systems. These include issues related to the accuracy and reliability of the algorithms, as well as concerns about privacy and ethical considerations.

Introduction

I. INTRODUCTION

This paper proposes a movie recommendation system through an emotion recognition system that would use arti?cial intelligence to analyze a user's facial expressions and emotions in order to recommend movies that align with their current mood. For example, if a user is feeling happy and excited, the system might recommend a comedy or action ?lm. If the user is feeling sad or down, the system might recommend a drama or romance ?lm. The system could also take into account the user's past movie preferences and ratings in order to make more personalized recommendations. Overall, this system would provide users with a convenient and personalized way to discover new movies that they are likely to enjoy.

The proposed system involves the application of BERT which is used for genre processing, along with hybrid deep learning models, which is used for sentimental analysis on the user reviews. It also uses a Facial Emotion recognition model which will be used with the movie and music classi?er to give the best suggestions.

Motivation

The motivation behind building a movie and music recommendation system for the youngsters according to their mood is to provide personalized recommendations that cater to their individual preferences and moods. This will enable them to discover new content and enhance their entertainment experience. It can also help in promoting the consumption of diverse content, thus fostering cultural awareness and appreciation among the young audience. Additionally, such a recommendation system can aid in boosting the engagement and retention of users on the platform, ultimately leading to increased revenue for the company.

II. RELATED WORKS

The paper [1], The article explores recommendation systems, which are intelligent systems designed to deliver users with customized information. While collaborative and content-based ?ltering methods are commonly used, they have shortcomings, like the requirement for previous user behavior and preferences. To overcome these limitations, the paper suggests a hybrid recommendation system that merges collaborative ?ltering, content-based ?ltering, along with sentiment analysis of movie tweets. The analysis helps to gain better insights into current trends and audience feedback on the movie. The tweets are gathered from microblogging services. The paper reports positive ?ndings from trials using open databases.

The paper [2], discusses how social media is being increasingly used in various application sectors, including recommender systems, to address issues related to collaborative ?ltering techniques and produce more reliable recommendations. In particular, the paper proposes a customer collaborative ?ltering approach that utilizes word embedding method tags to identify and classify individuals, and combines user similarities based on ratings and tag embedding to generate recommendations.

The proposed methodology outperformed the most popular collaborative ?ltering techniques in experiments using two well-known datasets to test its reliability for rating predictions and top-N suggestions.

The paper discusses how social media is being increasingly used in various application sectors, including recommender systems, to address issues related to collaborative ?ltering techniques and produce more reliable recommendations. In particular, the paper proposes a customer collaborative ?ltering approach that utilizes word embedding method tags to identify and classify individuals, and combines user similarities based on ratings and tag embedding to generate recommendations. The proposed methodology outperformed the most popular collaborative ?ltering techniques in experiments using two well-known datasets to test its reliability for rating predictions and top-N suggestions.

The paper [3], discusses the growing use of sentiment analysis by web services to provide content recommendations that take into account the emotions conveyed in users' social media posts. Current methods for sentiment analysis only categorize the sentiment of a sentence or reviews as neutral, positive, or negative, without taking into consideration variations in sentiment that may be attributed to the user's pro?le. To address this issue, this article introduces a improved Sentiment Metric (eSM), which combines a lexicon-based sentiment metric which also has a correction factor based on the user's pro?le. The eSM is used in a music recommendation system that will suggest songs to the user according to the user's sentiment intensity, extracted from social media posts. Subjective tests are conducted to determine the correction factor. The system achieved a better performance than a randomly selected song suggestion in a user evaluation, and it was perceived to have minimal impact on the mobile’s network, latency and energy consumption.

The paper [4], discusses how humans have created musical genres as a way to categorize di‰erent types of music based on shared traits such as instruments, rhythm, and harmony. Currently, musical genre annotations are done manually, but automatic genre classi?cation systems have the potential to supplement or replace human classi?cation. The paper investigates the automatic classi?cation of audio signals into a hierarchy of musical genres using three feature sets to describe pitch, rhythm, and timbre and suggests a framework for developing and evaluating features for content-based analysis of music signals. The e cacy of these features is assessed using statistical pattern recognition classi?ers trained on real-world audio datasets, achieving a 61% accuracy rate for ten musical genres, which is comparable to human classi?cation.

The paper [5], discusses that in recent months, Web2.0 has seen a surge in the use of social tags.User-de?ned tags have a high amount of semantic meaning for music.With the help of Last.fm's use-de?ned tags, we examine musical artists and classify their genres in this study.The calculation of artist similarity is based on tag co-occurrence.The genre of music has been classi?ed using the k-nearest neighbor algorithm (k-NN).The results of experiments exemplify that tags are useful in describing artistic similarities, and the proposed approach performs better than earlier web-based approaches for classifying artists by genre, with an average accuracy of 95% as opposed to 89.5% for Schedl et al. and 81.2% for Knees et al.

The paper [6], discusses the e‰ectiveness of recommender systems (RS) as information ?ltering tools to mitigate information overload for web users in the times of big data. One of the most e‰ective recommendation approaches, Collaborative ?ltering, has gained widespread attention from businesses and organizations. CF relies on past rating data from a number of patrons to make recommendations for the active user, without considering the information resource's substance. However, the large dimensionality and data sparsity of big data have posed challenges for conventional CF-based recommendation techniques, leading to the emergence of hybrid CF-based algorithms. These algorithms build a training model for the RS that further enhances recommendation accuracy and user satisfaction by introducing context information, such as time and trust relationships among peers.

This paper provides a review and summary of classic CF-based approaches and techniques utilized in RS, and the most recent hybrid memory-based and model-based CF recommendation algorithms, along with an analysis of contemporary hybrid CF-based recommendation approaches and techniques. We also introduce the latesthybrid Collabarative Filtering-based recommendation methods that take social networks into account in order to handle data sparsity and high dimensionality. Our proposed approach o‰ers a fresh viewpoint to improve RS e ciency, which can serve as a valuable resource for future researchers. Finally, we discuss potential future developments and their impact on RS.

The paper [7], discusses how automatic emotion detection which is based on facial expressions has become an important area to research in various ?elds, such as human-machine, health, and safety interactions. Researchers are keenly interested in developing methods to accurately identify and extract emotional characteristics from facial expressions to improve computer predictions. Deep learning techniques have shown signi?cant success, and various architectures are being used to enhance performance. The objective of this paper is to analyze recent studies on automatic facial emotion recognition using deep learning. The authors compare the proposed techniques with the actual results, highlighting the contributions, architectures, and databases used in these studies.

The paper [8], discusses the ongoing research on emotion recognition through facial feature detection in the ?eld of human-computer interaction (HCI).

Through their facial expressions and body language, people are able to convey a broad variety of emotions and feelings. The project will use Convolution Neural Network (CNN) and OpenCV to automatically detect human emotions based on facial identi?cation, and to recognize live emotions from human facial gestures.

Emotion detection requires integration of data from various patterns, and closing the gap between human and computer understanding of emotions is the ultimate goal. The paper focuses on the recognition of neutral, happy, and sad emotions by analyzing a person's frontal facial expressions.

The paper [9], discusses that the increasing use of social media platforms has made it more common for businesses to collect and analyze customer feedback to better understand their opinions. Therefore, an accurate and systematic method is necessary to predict ideas that may have a positive or negative impact on the market. This type of research is important for both producers and consumers. This paper focuses on analyzing movie reviews to determine the factors that in?uenced their reception and to develop an action plan for improving customer satisfaction. A robust sentiment analysis model is needed to categorize movie reviews, which are critical to the success or failure of a ?lm. This study employed various methods, such as Random Forest, Logistic Regression, Linear SVM, Stochastic Gradient, and Multinomial NB, to evaluate movie reviews.

The paper [10], discusses how sentiment analysis can automatically categorize the sentiment of a subject, such as optimistic, pessimistic, or unbiased, about a particular subject, news item, product, or movie. Machine learning is an e‰ective arti?cial intelligence (AI) method for achieving precise sentiment analysis, which is in high demand. User sentiment analysis on social networks like Facebook and Twitter is a powerful tool for discovering user opinions, and it has numerous applications in the same industry. However, there are many challenges in the domain of natural language processing which have hindered the accuracy and e‰ectiveness of sentiment analysis.

This study presents a state-of-the-art soft voting ensemble (SVE) method for sentiment analysis in movie reviews. The method employs ?ve well-known machine learning (ML) classi?ers. The proposed ensemble technique outperformed all other classi?ers, achieving an overall accuracy, precision, recall, and f1-score of 89.9%, 90.0%, 90.0%, and 90.0%.

The paper [11], discusses that Sentimental analysis is the act of comprehending and categorizing how people feel about a speci?c good or service. Every departmental ?eld is involved in gathering client input online or through social media symbols about a speci?c product or service. Positive, negative, or neutral feelings about the product or service are re?ections of human opinion. It is known as opinion mining, or the gathering of conceptions about human-related things. Sentimental analysis is a fascinating study of how human perceptions, attitudes, and emotions are expressed and composed in various languages using various multimedia data. It was used in a movie review application to track and examine social media activity related to the ?lm. Data gathering, data extraction, data classi?cation, and data interpretation are all involved. Through text analysis, the resource provider receives feedback and opinions regarding their resource. The proposed study work examines a solution in favor of sentiment categorization employing a ?ne-grained Naive Bayes Classi?er. Python-based software that applies machine learning and sentiment analysis to movie reviews is developed.

IV. LIMITATIONS

1. Limited Data Availability: The availability of data on the preferences and moods of young individuals may be limited, making it di cult to accurately build a recommendation system based on their moods.
2. Diculty In Accurately Detecting Mood: It may be challenging to accurately detect the mood of an individual, as it can vary greatly from person to person and can also change frequently.
3. Diculty In Categorizing Movies And Music According To Mood: There may be diculty in categorizing movies and music according to mood, as di‰erent individuals may perceive the same movie or music di‰erently based on their individual moods and preferences.
4. Personal Preferences:   The   personal preferences of young individuals may vary greatly, making it di cult to build a recommendation system that can cater to the unique preferences of each individual.
5. Limited Choice:  The  number of movies and music available for recommendation may be limited, making it di cult to provide a wide range of choices to young individuals.

Conclusion

This paper introduces a novel method to improve the accuracy of streaming-service recommender systems by incorporating sentiment analysis and genre embedding. The suggested strategy makes use of user-based joint ?ltering techniques, hybrid deep learning models, and genre embedding, and is evaluated using music and movie datasets. Our experiments demonstrate that the use of sentiment analysis and item genres in conjunction with rating data improves the accuracy and personalization of recommendations, This is because it considers the similarity between users based on their interests and the context of the reviews and genres they assign. In order to build on our work, we intend to investigate how our methodology might be applied in other ?elds and integrate aspect sentiment analysis to analyse user sentiments in-depth by correlating them with particular features or themes. We aim to leverage graph convolutional networks to predict the sentiment polarity of sentences towards speci?c aspects and to model the interaction between aspect terms and semantic and syntactic information to further enhance the performance of the recommender system.

References

[1] Kumar, S.; De, K.; Roy, P.P. Movie recommendation system using sentiment analysis from microblogging data. IEEE Trans. Comput. Soc. Syst. 2020, 7, 915–923. [CrossRef] [2] Sánchez-Moreno, D.; Moreno-García, M.N.; Mobasher, B.; Sonboli, N.; Burke, R. Using Social Tag Embedding in a Collaborative Filtering Approach for Recommender Systems. In Proceedings of the 2020 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, Melbourne, Australia, 14–17 December 2020. [3] Rosa, R.L.; Rodriguez, D.Z.; Bressan, G. Music recommendation system based on user’s sentiments extracted from social networks. IEEE Trans. Consum. Electron. 2015, 61, 359–367. [CrossRef] [4] Tzanetakis, G.; Cook, P. Musical genre classi?cation of audio signals. IEEE Trans. Speech 2002, 10, 293–302. [CrossRef] [5] Hong, J.; Deng, H.; Yan, Q. Tag-based artist similarity and genre classi?cation. In Proceedings of the 2008 IEEE Symposium on Knowledge Acquisition and Modeling Workshop, Wuhan, China, 21–22 December 2008; pp. 628–631. [6] Chen, R.; Hua, Q.; Chang, Y.-S.; Wang, B.; Zhang, L.; Kong, X. A survey of collaborative ?ltering-based recommender systems: From traditional methods to hybrid methods based on social networks. IEEE Access 2018, 6, 64301–64320. [CrossRef] [7] Improved Sentimental Analysis to the Movie Reviews using Naive Bayes Classi?er, Gowri S;Surendran R;Divya Bharathi M;Jabez, 2022 International Conference on Electronics and Renewable Systems (ICEARS) [8] Sentimental Analysis of Movie Reviews using Soft Voting Ensemble-based Machine Learning Ali Athar;Sikandar Ali;Muhammad Mohsan Sheeraz;Subrata Bhattachariee;Hee-Cheol Kim 2021 Eighth International Conference on Social Network Analysis, Management and Security (SNAMS) [9] Sentimental Analysis of Movie Reviews Using Machine Learning Algorithms ; Suresh Kumar;Kamlesh Sharma;Dhruv Veragi;Ankit Juyal 2022 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COM-IT-CON) [10] Emotion Detection with Facial Feature Recognition Using CNN & OpenCV Sarwesh Giri;Gurchetan Singh;Babul Kumar;Mehakpreet Singh;Deepanker Vashisht;Sonu Sharma;Prince Jain 2022 2nd International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE)

Copyright

Copyright © 2023 Arnav Dubey, Bhargey Shingala, Jay R Panara, Kush Desai, Assistant Prof. Sahana M P. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.