Emotion Based Music Recommendation System

Abstract

Music has a powerful impact on people and is extensively utilised for relaxation, mood control, stress relief, and illness prevention, as well as to sustain mental and physical work. Music has been nowadays acting as an stress-buster for every human. However opting out the song on the mood is so essential. It would be so skeptical for a person to decide the song to listen from heavy existing choices. Human emotions have become increasingly important in recent years. Human sentiments underpin emotion, which can be expressed or not. Emotion represents the individual behavior of humans, which can take several forms. The extraction of emotion states the individual state of behavior of humans. In this presentation hitting those right tracks for desired person is Fisher’s Face algorithm (LDA) briefly. In our proposed system a mood based music player is created which performs real time mood detection and suggests songs based on the detected mood

Introduction

I. INTRODUCTION

Now-a-days; recommendation systems have immense popularity and help people to select appropriate music. Mostly, presentation corners on facial based recommendation systems.

Sensors would usually collect face and voice patterns of a user and can track heart rate, oxygen saturation, steps travelled and many others. Even though, sensors and computing devices create extra inputs but they will increase the accuracy of generated music suggestions.

II. LITERATURE SURVEY/REVIEW

Multidimensional reduction is the process of taking primary data and reducing it to many different classes for sorting or organisation. A user's webcam image is used to capture and extract their emotions. By following the fundamental data, the dimensional reduction procedure improves the acquired image. These data are transformed into binary picture format, and face is detected using fisher’s face algorithm.

A. Fisher’s face Algorithm

We particularly use this because it maximises the separation between classes during the training process. This image processing system is used to reduce the face space dimensions using the principal component analysis (PCA) method and then it applies fishers linear discriminant (FDL) or the LDA method to obtain the feature of the image characteristics. While we utilise the minimal euclidean technique for matching faces and Fisher Face for picture identification, this algorithm aids in classifying the expressions that suggest the user's emotions.

B. Comparative analysis

The table below compares the accuracy of several existing systems and tools to our used system :

III. PROPOSED SYSTEM

The goal of the Facial Expression Depending Music Player is to scan and understand the data, then create a playlist based on the given characteristics. To construct an emotion-based music player, our suggested system focuses on identifying human emotions. It describes the methodologies used by existing music players to detect emotions, the way our music player adopts to detect human emotions, and how it is best to use our system for emotion detection. Additionally, a brief explanation of our systems' operation, playlist creation, and emotion classification is given.

The project's results are divided into two phases:

1. Using Python, build an application that can identify a user's emotion  from their expression.
2. Add Python code to the web service, and the music will be played based on the user's expression.

A. Mood Detection module

On Kaggle, a dataset of songs labelled according to mood was discovered for Hindi and English.

In order to save, retrieve, and query this song's data in response to user requests, research was done on the best cloud storage platforms.

There were alternatives like AWS, Google Cloud, etc., but these were rejected since they were expensive and offered very little storage for free. After that, research was done on open-source streaming services like Restream.io, Ampache, etc., but once more, these systems were web-based, used for live YouTube broadcasting, and were only intended for private use. After much investigation (and due to time restrictions), Firebase was decided to be a backend server. It only requires one click to integrate with an Android app, and its free plan offers 5GB of storage.

B. Integration

The trained MobileNet model was saved as an.h5 file, which was then converted to a .tflite file using TensorFlow Lite Converter in order to integrate these two modules in an Android application. It accepts a TensorFlow model as input and produces an output with the.tflite extension for a TensorFlow Lite model. The size of the tflite file is predicted to be between 20 and 25 Megabytes because the MobileNet model is utilised (MB) which size was desired. The.tflite file and labels.txt file were placed in an assets folder that was made in Android Studio.

The model's class labels are contained in the labels.txt file. The correct procedures for loading the model, starting the interpreter, and getting the results were all developed.

A Firebase project was made, and MP3 files were added to the storage area. These songs are included in the real-time database area according to language and mood. After that, Android Studio was connected to the Firebase database. The tflite model methods were connected with the songs on Firebase, and an appropriate user interface was made for the android application. The programme was then examined for defects and, if any, fixed.

IV. METHODOLOGY

A. Face Capturing

The main aim of this process is to take pictures, thus we are utilising a routine devices, such as camera and webcams but anyhow other physiological equipment may be used.

For the capturing sake we will use computer vision libraries.

In initial phase it starts to access the camera and capture about 10 images for further processing .

We will use particular algorithms to pick out a authentic image amongst all captured ones.Thus, We require a large number of positive and negative pictures that would have faces or no faces such that the classifier may be trained.

B. Face Detection

The basic purpose of the face detection approach is to recognise the face in the frame while removing external noises and other elements in images.

By using Principal Component Analysis ;The data received will be decomposed into multiple scales. Subsequently smoothing the picture and subsampling it by reducing its resolution, the procedure must be repeated several times to achieve a flawless outcome.

Linear Discriminant is the last step in the process which isused to shorten the computing time required by the classification process, resulting in a more accurate detection.

C. Emotion Classification

When the face is correctly recognised, a box appears and overlays the picture to extract the face and for further analysis.

On progress, the images that are extracted previously will be processed using the function. The code will extract the facial spatial locations from the face picture using a boosting approach and pixel intensity values that are indexed at each place.

Finally, using Boosting algorithm where, it compares the input data to the stored data in order to forecast the class that contains the emotion.

D. Music Recommendation

Initially, every song has an emotion attributed to it. When an emotion is transmitted, it is allocated to a song, and the emotions are sorted and assigned to each song.

After all the churning processes the emotion will be deduced from input picture then based on emotion the song will be played.

Here in this project we are mostly using PCA and LDA algorithms.

The classifier's effectiveness is high 90-95 % such that even if the face varies owing to external factors, the system can still recognise the face and emotion and can run song on it.

V. RESULT AND ANALYSIS

At present,the project is still on run to meet the ends of the race where we built the model to detect the face and classify the mood of the user as positive(happy) and negative(sad) and recommend the song based on that emotion classified.

Here; during training and testing the model it displayed about 99% of accuracy because we only initially classified the emotion only in into two i.e;happy or sad as furnished in the below images.

In further work we are going to work on molding the model to classify the all human emotions and accordingly make music to play the all human emotions and accordingly make music to play automatically by emotion.

Conclusion

In this project we have proposed a model that trades-off accuracy in order to reduce the computation weight. This project\'s music suggestion algorithm is based on the user\'s real-time emotions as seen in their photographs. A better interface between the user and the music system is the goal of this project\'s design. due to the fact that music can alter a person\'s mood and, for others, serve as a stress relief. The creation of an emotion-based music recommendation system has a broad future, according to recent developments. In order to be able to recognise facial expressions and identify emotions, the current system has a face-based recognition system.

References

[1] EMOTION BASED MUSIC RECOMMENDATION SYSTEM ; H. Immanuel James1, J. James Anto Arnold2, J. Maria Masilla Ruban3, M. Tamilarasan4, R. Saranya5.(2019-20) [2] Mood based Music Recommendation System ; Ankita Mahadik, Prof. Vijaya Bharathi Jagan, Shambhavi Milgir, Vaishali Kavathekar(.2020). [3] Emotional Detection and Music Recommendation System based on User Facial Expression ;S Metilda Florence1 and M Uma2.(2019) [4] Melomaniac- Emotion Based Music Recommendation System ; Ishwar More1 ,Vedant Shirpurkar2 ,Yudhish Gautam3 , Natasha Singh.(2019) [5] FACE EMOTION BASED MUSIC DETECTION SYSTEM;Prof. Subhash Rathod1 , Akshay Talekar2 , Swapnil Patil3 , Prajakta Gaikwad4, Ruchita Gunjal5.(2020) [6] EMOTION BASED MUSIC RECOMMENDATION SYSTEM ; Dr.Sunil Bhutada,CH.sadhvika,Gutta.Abigna,P.Srinivasreddy.(2020-21)

Copyright

Copyright © 2023 Kavin Kumar, Nanda Kishore Reddy, Hemaa Chandhan, Nalinraj , Dr. M. John Basha. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.