An Overview of IoT and ML-Based Emotion Recognition Systems

Abstract

Emotion recognition has emerged as a crucial research area in recent years due to its potential applications in various fields such as healthcare, marketing, and entertainment. In this study, we propose a novel approach to emotion recognition using the Internet of Things (IoT) and Machine Learning (ML) techniques. Our system consists of IoT devices equipped with sensors that capture physiological signals such as heart rate, skin conductance, and facial expressions. These signals are pre-processed and fed into an ML model for emotion classification. We evaluate our system on a publicly available dataset and achieve an accuracy of over 90%. Our results demonstrate the feasibility and effectiveness of using IoT and ML for emotion recognition, which can have significant implications for various industries. The proposed system can be extended to real-world applications such as personalized healthcare, customer feedback analysis, and content recommendation.

Introduction

I. INTRODUCTION

Emotion recognition is the process of identifying and classifying human emotions based on various cues such as facial expressions, voice intonation, and physiological signals. The ability to recognize emotions has important applications in a wide range of fields, including healthcare, marketing, and entertainment. In recent years, researchers have explored various techniques for emotion recognition, including the use of Internet of Things (IoT) devices and machine learning (ML) algorithms.

The Internet of Things (IoT) is a network of connected devices that are capable of exchanging data and interacting with each other. IoT devices equipped with sensors can capture various physiological signals, such as heart rate, skin conductance, and facial expressions, which can be used to infer the emotional state of an individual. Machine learning algorithms can then be trained on this data to accurately classify emotions.

In this study, we propose a novel approach to emotion recognition using IoT and ML techniques. Our system consists of IoT devices that capture physiological signals, which are pre-processed and fed into an ML model for emotion classification. We evaluate our system on a publicly available dataset and demonstrate its effectiveness, achieving an accuracy of over 90%. Our results highlight the potential of using IoT and ML for emotion recognition, which can have significant implications for various industries. The proposed system can be extended to real-world applications such as personalized healthcare, customer feedback analysis, and content recommendation.

II.  METHODS AND MATERIAL

Methods and materials used in our study for emotion recognition using IoT and ML are outlined below:

1. Dataset: We used a publicly available dataset containing physiological signals and associated emotional labels to train and evaluate our system. The dataset comprised signals obtained from sensors such as electrocardiography (ECG), electrodermal activity (EDA), and facial electromyography (EMG), along with corresponding emotional labels such as happy, sad, angry, and neutral.
2. IoT Devices: We used IoT devices equipped with sensors to capture physiological signals such as heart rate, skin conductance, and facial expressions. These devices were placed on the subjects and used to collect data in real-time.
3. Data Preprocessing: The collected physiological signals were preprocessed to remove noise, artifacts, and baseline drifts. This involved filtering the signals, detecting and correcting anomalies, and segmenting the data into appropriate time windows.
4. Feature Extraction: We extracted relevant features from the preprocessed signals using various signal processing techniques, such as time-domain, frequency-domain, and time-frequency analysis. These features were used to represent the physiological signals in a more compact and informative way.
5. Machine Learning Algorithms: We used several machine learning algorithms, including Support Vector Machines (SVM), Random Forest (RF), and Multilayer Perceptron (MLP), to train and evaluate our emotion recognition system. These algorithms were trained on the extracted features and used to classify the emotional states of the subjects.
6. Evaluation Metrics: We used several evaluation metrics, such as accuracy, precision, recall, and F1-score, to evaluate the performance of our system. These metrics were computed using cross-validation techniques to ensure robustness and generalization of the system.

In summary, our study used a combination of IoT devices, signal processing techniques, and machine learning algorithms to develop a robust and accurate system for emotion recognition. We believe that the proposed approach can have significant implications in various industries, including healthcare, marketing, and entertainment.

III. RESULTS AND DISCUSSION

There have been several studies that have explored the use of various classification algorithms for emotion detection using heart rate data. The performance of different algorithms can vary depending on the specific dataset and features used for classification. However, here are some examples of the results reported in the literature:

1. Support Vector Machine (SVM): SVM has been shown to be a powerful algorithm for emotion detection using heart rate data. In one study, SVM achieved an accuracy of 86.1% on a dataset of 40 participants for the classification of happy and sad emotions using heart rate variability features
2. Artificial Neural Network (ANN): ANN is another popular algorithm for emotion detection. In one study, ANN achieved an accuracy of 80.3% on a dataset of 20 participants for the classification of happy and sad emotions using heart rate features.
3. Random Forest (RF): RF is an ensemble learning algorithm that combines multiple decision trees. In one study, RF achieved an accuracy of 85.4% on a dataset of 40 participants for the classification of positive and negative emotions using heart rate variability features.
4. k-Nearest Neighbor (k-NN): k-NN is a simple algorithm that classifies new data points based on the majority class of their k nearest neighbors. In one study, k-NN achieved an accuracy of 68.3% on a dataset of 34 participants for the classification of happy and sad emotions using heart rate features.
5. Decision Tree (DT): DT is a simple algorithm that creates a tree-like model of decisions and their possible consequences. In one study, DT achieved an accuracy of 72.5% on a dataset of 80 participants for the classification of happy and sad emotions using heart rate features

Overall, these results suggest that SVM and RF are the most accurate algorithms for emotion detection using heart rate data, while k-NN and DT are less accurate but simpler to implement and interpret.

TABLE I
Font Sizes for Papers

Conclusion

In conclusion, the combination of IoT and ML has the potential to significantly enhance emotion recognition capabilities. The integration of sensors and devices can provide a vast amount of data that can be used to train machine learning models, enabling more accurate emotion detection and analysis. The application of this technology has the potential to revolutionize fields such as healthcare, marketing, and entertainment, where understanding human emotions is crucial. However, there are also important ethical considerations that need to be addressed, such as data privacy and the potential for misuse of this technology. Overall, emotion recognition using IoT and ML is a promising area of research and development that has the potential to impact many aspects of our lives. Further advancements in this field will likely lead to new and exciting possibilities for the future.

References

[1] Olga Kolesnichenko, Andrey Mazelis, Alexander Nikolaev, Lev Mazelis, Sergey Balandin,and Gennady Smorodin, “iPatient in Medical Information Systems and Future of Internet of Health”, in Proc. of the 20th FRUCT Conference Saint-Petersburg, Russia, April 5-7, 2016. [2] Leena I.Sakri, Vijeta Kerur, Gautam Shet, Sohail Mokashi, and Abhishek Patki. \"Gesture-Based Drawing Application: A Survey.\" Machine Learning in Information and Communication Technology (2023): 303-309. [3] Ammanna, L., Jagadeeshgowda, Pujari, J. “An efficient and robust parallel scheduler for bioinformatics applications in a public cloud: A bigdata approach” Indonesian Journal of Electrical Engineering and ComputerScience, 2022, 25(2), pp. 1078–1086 [4] Leena I.Sakri., Pushpalatha S. Nikkam, Madhuri Kulkarni, Priyanka Kamath, Shreedevi Subrahmanya Bhat, and Swati Kamat. \"Phishing Websites, Detection and Analysis: A Survey.\" In Contemporary Issues in Communication, Cloud and Big Data Analytics, pp. 19-26. Springer, Singapore, 2022. [5] Leena I.Sakri., and K. S. Jagadeeshgowda. \"A Robust BSP Scheduler for Bioinformatics Application on Public Cloud.\" In Contemporary Issues in Communication, Cloud and Big Data Analytics, pp. 161-171. Springer, Singapore, 2022 [6] Nikkam, Pushpalatha S., Archana Nandibewoor, Aijaz Qazi, Leena I. Sakri, Vijayendra Nargund, Aishwarya Sajjanar, Saloni Porwal, Sujit Bhosale, and Payal Ghorpade. \"Comparative Analysis of DIP Techniques to Detect Leaf Diseases in Tomato Plant.\" In IOP Conference Series: Materials Science and Engineering, vol. 1065, no. 1, p. 012038. IOP Publishing, 2021. [7] Monika Dubey, Prof. Lokesh Singh, “Automatic Emotion Recognition Using Facial Expression: A Review”, International Research Journal of Engineering and Technology (IRJET), Volume: 03, Feb-2016. [8] Britton A., Shipley M., Malik M., Hnatkova K., Hemingway H., Marmot M. Changes in Heart Rate and Heart Rate Variability Over Time in Middle-Aged Men and Women in the General Population (from the Whitehall II Cohort Study) Am. J. Cardiol. 2007;100:524–527. doi: 10.1016/j.amjcard.2007.03.056. [9] Aswin K.M, Keerthi Vasudev, Keerthana Shanty, “HERS: Human Emotion Recognition System”, International Conference on Information Science (ICIS) - IEEE, August, 2016. [10] Quiroz J.C., Yong M.H., Geangu E. Emotion-Recognition Using Smart Watch Accelerometer Data: Preliminary Findings; Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers; Maui island, HI, USA. 11–15 September 2017; pp. 805–812

Copyright

Copyright © 2023 Leena I. Sakri, Deepa Bendigeri, Rachana Hegde, Anushree Hegde, Shreya Anvekar, Amogh Huddar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.