Human Activity Recognition Using Tensorflow

Abstract

Human Activity Recognition (HAR) has become a key focus in healthcare and machine learning, aiming to improve personal well-being and lifestyle management through sensor data. As individuals lead increasingly busy lives, continuous monitoring of their activities can provide valuable insights for health management. Despite advancements, identifying patterns in human activity remains a challenging task, especially with diverse sensor data sources. This paper explores various technical approaches for human activity recognition, focusing on the use of TensorFlow and Long Short-Term Memory (LSTM) networks, which are effective in modelling time-series data from devices like smartphones and wearables. The paper highlights the potential of these models for real-time activity classification

Introduction

I. INTRODUCTION

Human Activity Recognition (HAR) is an emerging area of research focused on identifying and classifying human activities based on sensor data from devices like smartphones and wearables. It plays a crucial role in various applications, such as health monitoring, fitness tracking, and smart environments. The recognition of activities, such as walking, running, and sitting, relies on data from sensors such as accelerometers and gyroscopes, which capture motion patterns over time. The primary objective of this study is to evaluate the effectiveness of various deep learning algorithms, particularly using TensorFlow and Long Short-Term Memory (LSTM) networks, for human activity classification. The results demonstrate that TensorFlow-based models, specifically LSTM, can effectively process time-series sensor data to accurately classify a range of human activities. These findings highlight the potential of deep learning for real-time activity recognition

II. LITERATURE SURVEY

The research paper titled "Human Activity Recognition using Deep Learning Model" by Antara Chakrabarti et al. [1]. 2022, explores human activity classification using a deep learning model, likely a CNN, RNN, or a hybrid CNN-LSTM. The methodology involves preprocessing steps like normalization and segmentation to prepare sensor data for training. Key contributions include the development of an effective deep learning architecture, detailed performance analysis, and exploration of mobile optimization to enhance practical applications. While the study demonstrates significant advantages such as high classification accuracy, automated feature extraction, and real-time implementation potential, it acknowledges limitations like high computational costs, dependency on large datasets, and increased battery consumption on mobile devices.

The research paper titled "LSTM Networks for Mobile Human Activity Recognition" by Yuwen Chen et al. [2]. 2016, explores the application of Long Short-Term Memory (LSTM) networks for classifying human activities using mobile sensor data. The study leverages the ability of LSTM networks to capture temporal dependencies within sequential data, making them well-suited for human activity recognition (HAR). Key contributions include demonstrating the effectiveness of LSTM networks in improving classification accuracy through robust temporal data analysis and providing insights into optimizing deep learning models for mobile environments. The study emphasizes advantages such as high accuracy in recognizing sequential activities, strong temporal modeling capabilities, and potential for real-time implementation on mobile devices. However, the research also identifies limitations, including the computational intensity of continuous mobile use, which can result in battery consumption challenges, and the model's reliance on large, high-quality datasets for effective training.

The research paper titled "Modeling Human Activity Recognition using Keras and TensorFlow: Deep Learning Approach" by R.S. Kamath et al. [3]. 2018, explores the use of deep learning techniques for human activity recognition (HAR) based on smartphone sensor data. The methodology involves leveraging the Keras and TensorFlow frameworks, likely employing Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs) to classify activities by capturing both spatial and temporal features from the data.

Key contributions include demonstrating the practical application of Keras and TensorFlow in HAR, achieving effective activity classification, and showcasing the potential for real-time mobile implementation. The study highlights advantages such as high classification accuracy, reduced need for manual feature engineering through automatic feature extraction, and the adaptability of Keras and TensorFlow for efficient model development. However, it also acknowledges limitations, including the computational intensity of deep learning models, which can affect battery efficiency and overall performance on mobile devices. Additionally, the model's success depends heavily on the quality and quantity of training data.

The research paper titled "Human Activity Recognition" by Ms. Shikha, et al. [4]. 2020, explores the classification of human activities using sensor data from mobile or wearable devices. The study likely employs machine learning or deep learning models, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), or Long Short-Term Memory (LSTM) networks, to capture spatial and temporal patterns in the data. The methodology involves preprocessing steps like normalization and segmentation to prepare the sensor data for effective training. Key contributions of the paper include presenting a practical model for human activity recognition, with an emphasis on improving classification accuracy and exploring applications in health monitoring and fitness. The study highlights advantages such as automatic feature extraction, reducing the reliance on manual feature engineering, and potential support for real-time activity recognition on mobile platforms. However, it also identifies limitations, including the need for significant computational power and large datasets, which may hinder real-time performance on mobile devices without further optimization.

The research paper titled "Human Activity Recognition Using CNN and LSTM Methods" by Ayesha Sarwat, et al. [5]. 2022, presents a hybrid methodology combining Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks for classifying human activities using sensor data. In this approach, CNNs are employed for feature extraction, while LSTMs capture temporal dependencies, enabling improved recognition of sequential activities. Key contributions of the paper include the introduction of this CNN-LSTM hybrid model, which enhances both accuracy and efficiency in human activity recognition, particularly for complex activities requiring an understanding of spatial and temporal patterns. The study highlights advantages such as achieving high recognition accuracy by leveraging both spatial and temporal features, making it especially effective for complex sequential activities. However, the research also identifies limitations, including the high computational cost, which poses challenges for real-time applications on mobile devices. Additionally, the model's dependence on large datasets for effective training and its potential inefficiency in resource-constrained environments are noted as key challenges.

The research paper titled "Human Activity Recognition" by Pooja M V et al. [6]. 2022, explores the classification of human activities using data from sensors such as accelerometers and gyroscopes. The study likely employs machine learning or deep learning techniques, incorporating preprocessing steps like data normalization and segmentation to enhance classification accuracy. Key contributions of the paper include proposing a novel approach for recognizing human activities, aiming to improve classification accuracy and exploring applications in health monitoring and wearable technology. The study highlights advantages such as enabling automatic activity detection, which can be integrated into real-time mobile applications for use in fitness tracking, healthcare, and smart home systems. However, the paper also acknowledges limitations, including the need for a large and diverse dataset for effective performance. Additionally, deploying the model on mobile devices may face challenges due to computational demands and battery constraints.

The research paper titled "LSTM-CNN Architecture for Human Activity Recognition" by Kun Xia, et al. [7]. 2020, proposes a hybrid architecture that combines Long Short-Term Memory (LSTM) networks and Convolutional Neural Networks (CNNs) for human activity recognition (HAR). Conducted at the University of Shanghai for Science and Technology, the study leverages CNNs to extract spatial features and LSTMs to capture temporal dependencies in sensor data, enabling more accurate classification of human activities by effectively handling both spatial and temporal patterns. Key contributions of the paper include introducing this LSTM-CNN hybrid model, which enhances the recognition of complex sequential activities and improves overall classification accuracy. The study highlights advantages such as achieving high accuracy and enabling applications in real-time scenarios, including health monitoring and smart home systems. However, the research also notes limitations, including the high computational cost associated with real-time applications and the model's reliance on large datasets for effective training. These factors can pose challenges in resource-constrained environments, such as mobile devices.

The research paper titled "Deep Learning Based Human Activity Recognition (HAR) Using Wearable Sensor Data" by Saurabh Gupta,[8].2021, explores the use of deep learning techniques to classify human activities using data from wearable sensors. Conducted at the Department of Computer Science, Liverpool John Moores University, the study processes sensor data, such as accelerometer and gyroscope readings, to identify various human activities.

The methodology includes preprocessing steps to clean and normalize the data, followed by the application of deep learning models like Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs) for activity classification. Key contributions of the paper include demonstrating the effectiveness of deep learning methods for human activity recognition, with a focus on wearable technology applications such as health monitoring, fitness tracking, and real-time activity recognition. The study highlights advantages such as achieving high classification accuracy and enabling automatic feature extraction from raw sensor data, reducing the need for manual feature engineering. However, it also identifies limitations, including the significant computational resources required to train deep learning models and the reliance on large, diverse datasets for optimal performance.

The research paper titled "Advancing Human Action Recognition: A Hybrid Approach Using Attention-Based LSTM and 3D CNN" by El Mehdi Saoudi, et al. [9]. 2023, presents a hybrid model that integrates attention-based Long Short-Term Memory (LSTM) networks and 3D Convolutional Neural Networks (CNNs) for human action recognition (HAR). Conducted at the University of Hassan II Casablanca, Faculty of Science Ain Chock, Morocco, the study aims to improve the accuracy and efficiency of human action recognition by leveraging the strengths of both models. The attention-based LSTM captures temporal dependencies in sequential data, while the 3D CNN extracts spatial features from video data, enabling the model to recognize complex human actions in dynamic environments. Key contributions of the paper include the fusion of LSTM and 3D CNNs to create a robust solution for HAR. The hybrid approach offers several advantages, such as enhanced accuracy in recognizing human actions, the ability to process both spatial and temporal data, and the potential for real-time applications in fields like surveillance, healthcare, and sports analysis. However, the research also identifies challenges, including high computational demands and the need for large datasets for effective training, which could limit the model's performance in resource-constrained environments.

The research paper titled "Human Activity Analysis Using Machine Learning Classification Techniques" by Zameer Gulzar, et al. [10]. 2019, focuses on applying machine learning classification techniques for human activity analysis using sensor data. The study likely utilizes a variety of machine learning algorithms, such as Decision Trees, Support Vector Machines (SVM), and k-Nearest Neighbors (k-NN), to classify human activities based on data from wearable sensors like accelerometers and gyroscopes. The paper emphasizes the use of data preprocessing and feature extraction techniques, such as normalization and segmentation, to enhance classification accuracy. A key contribution of the study is its exploration of different machine learning algorithms for activity recognition, offering valuable insights into their effectiveness and applicability for real-time activity classification in domains such as healthcare, fitness, and smart homes. The proposed approach offers advantages like simplicity, lower computational cost compared to deep learning models, and the ability to deploy on mobile devices for real-time applications. However, the study also identifies limitations, such as the reliance on high-quality sensor data and the challenge of recognizing complex activities with basic machine learning techniques.

III. COMPARISON AMONG MODELS

We compared the work based on metrics, algorithms used and the accuracy on several datasets used by the authors. The work is summarized as shown in Table 1.

Table-1: Comparison Among Models

Conclusion

This human activity recognition project demonstrates the successful use of LSTM models to classify a variety of physical activities, including walking, standing, jogging, sitting, biking, and going up or down stairs. By leveraging time-series data from multiple sensors, the model effectively distinguishes between dynamic activities (e.g., jogging) and static activities (e.g., sitting, standing) based on unique signal patterns. The project highlights the potential for real-time activity tracking, with applications in health monitoring, fitness assessment, and behavioral analytics.

References

[1] Antara Chakrabarti and Siddhanta Kumar Singh, \"Human Activity Recognition using Deep Learning Model,\" Mody University International Journal of Computing and Engineering Research, Vol. 6, Issue 1, pp. 27-30, 2022. [2] Yuwen Chen, Kunhua Zhong, Ju Zhang, Qilong Sun, and Xueliang Zhao, \"LSTM Networks for Mobile Human Activity Recognition,\" International Conference on Artificial Intelligence: Technologies and Applications (ICAITA), January 2016. DOI: 10.2991/icaita-16.2016.13. [3] R.S. Kamath and R.K. Kamat, \"Modelling Human Activity Recognition using Keras and TensorFlow: Deep Learning Approach,\" International Journal of Modern Electronics and Communication Engineering (IJMECE), Vol. 6, Issue 5, pp. 27-30, September 2018. [4] Ms. Shikha, Rohan Kumar, Shivam Aggarwal, and Shrey Jain, \"Human Activity Recognition,\" International Journal of Innovative Technology and Exploring Engineering (IJITEE), ISSN: 2278-3075 (Online), Vol. 9, Issue 7, May 2020. [5] Ayesha Sarwat, Dr. K. Vaidehi, and M. Sowmya, \"Human Activity Recognition Using CNN and LSTM Methods,\" Journal for Basic Sciences, Vol. 22, Issue 11, pp. 367-381, 2022. DOI: 10.37896/JBSV22.11/1541. [6] Pooja M V and Dr. Ravindra P Rajput, \"Human Activity Recognition,\" International Research Journal of Engineering and Technology (IRJET), e-ISSN: 2395-0056, Vol. 9, Issue 7, July 2022. [7] Kun Xia, Jianguang Huang, and Hanyu Wang, \"LSTM-CNN Architecture for Human Activity Recognition,\" IEEE Access, Vol. 8, pp. 56855-56866, 20 March 2020. DOI: 10.1109/ACCESS.2020.2982225. [8] Saurabh Gupta, \"Deep Learning Based Human Activity Recognition (HAR) using Wearable Sensor Data,\" International Journal of Information Management Data Insights, Vol. 1, Issue 2, November 2021, Article 100046. Elsevier. DOI: 10.1016/j.jjimei.2020.100017. [9] El Mehdi Saoudi, Jaafar Jaafari, and Said Jai Andaloussi, \"Advancing Human Action Recognition: A Hybrid Approach Using Attention-Based LSTM and 3D CNN,\" Scientific African, Vol. 21, September 2023, Article e01796. DOI: 10.1016/j.sciaf.2023.e01796. [10] Zameer Gulzar, A. Anny Leema, and I. Malaserene, \"Human Activity Analysis using Machine Learning Classification Techniques,\" International Journal of Innovative Technology and Exploring Engineering (IJITEE), ISSN: 2278-3075, Vol. 9, Issue 2, December 2019.

Copyright

Copyright © 2024 N Nisarga, Sankhya N Nayak, Mala G B, Manasa S, Pranathi T. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.