Human Activity Recognition System Using Smartphone Data Sensors with Python and Machine Learning

Abstract

This project depicts recognition Human activity Using data generated from user Smartphones Machine Learning repository to recognize six human activities. These activities are standing, sitting, laying, walking, upstair and walking, ddownstairs. Data is collected from embedded accelerometer, gyroscope and other sensor .Data is randomly divided into 7:3 ratios to From training and testing data set respectively. Activity Classification done using Machine Learning models Namely Random Forest. support vector machine, Neural Network and k-Nearest Neighbor. We have compared accuracy and performance of these model using confusion matrix and random simulation. Human Activity recognition(HAR) is classifying activity of person using responsive sensor that are affected from human movement. Both users and capabilities of smartphone With them. These facts makes HAR more important and Popular. This work focuses on recognition of Human activity using smartphone sensor different machine learning clssification approaches. Data retrieved from smartphones accelerometer and gyroscope sensor are classified On order to recognize human activity. Results of the approaches used compared in terms of efficiency and precision.

Introduction

I. INTRODUCTION

This project uses low-cost and commercially available smartphone as sensor to identify human activity. These growing popularity and computational power of smartphone makes it an ideal candidate for non-intrusive body -attached sensors. According to the statistic of us mobile subscribers around 44% of mobile subscribers in 2011 own smartphone and 96% of these smartphone have built-in inertial sensor such as accelerometer or gyroscope.

Research has shown that gyroscope can help activity recognition even though its contribution alone is not as good as accelerometer. Because gyroscope is not so easily accessed in cellphones as accelerometer our system only uses readings from 3-dimensional accelerometer.

Unlike many other works before we relaxed the constraints of attaching sensor to fixed body position with device orientation. In our design, the phone can be placed at any position around waist such as jacket pocket and pants pocket with arbitrary orientation. These are the most common positions where people carry mobile phone.

Training process is always required when a new activity is added to the system. Parameter of the same algorithm may need to be trained and adjusted when the algorithm runs on different devices due to the variance of sensors.

A. Motivation

Human Recognition system have various approaches, such as vision based and sensor based, which farther categorized into wearable, object-tagged, dense sensing etc. Before  moving further there are also exist some design issues in HAR system such as selection of different types of sensors, data collection related set of rules, recognition performances ,how much energy is  consumed, processing capacity and flexibity kepping all these parameters in mind it is Important to design an efficient and lightweight Human activity recognition model. A network for mobile human activity recognition has been proposed using long-short term memory approach for human activity recognition using triaxial accelerometer data.

II. LITERATURE SURVEY

The HAR model is a challenging and growing field of research in ML that employs sensor and computer vision system(jobanputra et al,2019).Over the past years, HAR models mainly relied on sensor -based systems, but the rapid evolution of computer vision(CV) and DL developed sophisticated algorithm that can provide higher accuracy. It is fact that CV-based algorithm still face challenges due to the complexity caused by the variability of the background in everyday environments more specifically, there are limitations due to the variable illumination condition of each scene during the day.

Pirttikangas et al.(2006),in their pioneer research tested a model that used several multilayer perceptron and k-nearest neighboure (K-NN) algorithm to recognize activity to achieve an overall accuracy of 90.61%.Casale et al.(2011) used a wearable device and applied a random forest classification algorithm to model five distinct activities(walking, climbing stair, talking) with a person staying standing and working on the computer .This research achieved an overall accuracy of  90% . a few year later, Ahmed and Loutfi in (2013) performed an HAR system using case-based reasoning support vector machine (SVMS) and neural network (NN) to achieve an overall accuracy of 0.86 , 0.62 and 0.59 respectively for there specific activity (breathing, walking, running or sitting) wang et al in (2013)  introduced a wearable system using passive tags which achieved an accuracy an of 93.6%.

Li at (2016) achieved an accuracy of 96% and f-score is equal to 0.74 for 10 medical activity’s.

In fact Li et al introduced  an activity recognition system  for complex an dynamic medical settings that use passive radio-frequency identification technology (RFID). Ryoo et al. (2018) proposed a backscattering activity recognition network of passive RF tags capable of recognition daily human activity with an average error of 6%.

III. TECHNOLOGY USED

1. Python: Python is an interpreted, high-level, general- purpose programming language. Python's design philosophy emphasizes code readability with its notable use of significant whitespace. Its language constructs and object-oriented approach aim to help programmers write clear, logical code for small and large-scale projects. Python is dynamically typed and supports multiple programming paradigms, including procedural, object- oriented, and functional programming.
2. JUPYTER Lab: Project JUPYTER is a nonprofit organization created to develop open-source software, open-standards, and services for interactive computing across dozens of programming languages.
3. Google Colaboratory: The IDE used for this project is Google Colaboratory which is the best of the times to deal with deep learning projects. Phase 1 was explained above as from where the dataset is downloaded. In this sequence to start with the project open a new notebook in Google Colaboratory first import all the necessary libraries.
4. Pandas: Pandas for loading dataset.
5. Numpy: Numpy for performing neumerical computation.
6. Matplotlib: Matplotlib for plotting.
7. Tensorflow: Tensorflow for creating different neural networks.

IV. IMPLEMENTATION

A. Data Collection

Machine Learning is the new big thing in the world of computer science. The motivation behind this project is to implement the machine learning algorithm  in real-world data set that their accuracy can be studied and effective conclusion can bedrawn. In this task we develop AI model for “Human Activity Rcognition using smartphone data and machine laerning” from UCI(university of california lrvine) online storehouse. This informational index has been gathered from chronicles of 30 human subject caught by means of cell phone empowered with installed intertial sensor. Many AI course utlize this information for Educating puropses. This is a multi-arrangement issue. The information collection has 10299 lines and 561 segments there are thirty volunteers of age group 18-50 year and examination done in that. Every individual performed physical activity like Walking, Walking_upstair, Walking_doenstair Sitting , Standing , Laying etc.

B. Design Approach

Science the data is a multivariate classification problem we have used supervised and unsupervised learning algorithm. Orange tool has been used for implementation and  neural network is implemented in python with drop outs which in turn gave better result. The data set was fed into the modules for random forest, knn, neural network, Logistic regression stochastic gradient descent and naive bayses. Their precision and recall values were calculated and a confusion matrix for each model was made.

Neural  network was implemented using python where keras over tensorflow was used. The basic concept is same as the neural network given in orange however this  approach differs by providing drop outs in the random Forest wprks for the both classification and or regression tasks. Arbitray forests from a lots of choice tree. Each tree is created from a bootstrap test from the preparation information. Determine what number of choice trees will be incorporated into woodland (Number of tree in the timberland) and what number of characteristics will be self -assertively drawn for thought at every hub.

C. Machine Learning

Machine learning is the scientific study of algorithms and statistical models that computer systems use in order to perform a specific task effectively without using explicit instructions, relying on patterns and inference instead. It is seen as a subset of artificial intelligence. Machine learning algorithms build a mathematical model based on sample data, known as "training data", in order to make predictions or decisions without being explicitly told image processing is the use of computer algorithms to perform image processing on digital images.

D. Proposed System

The overall architecture diagram of the method proposed in this paper is shown in Figure , which contains three parts. The first part is the preprocessing and transformation of the original data, which combines the original data such as acceleration and gyroscope into an image-like two-dimensional array. The second part is to input the composite image into a three-layer CNN network that can automatically extract the motion features from the activity image and abstract the features, then map them into the feature map. The third part is to input the feature vector into the LSTM model, establish a relationship between time and action sequence, and finally introduce the full connection layer to achieve the fusion of multiple features. In addition, Batch Normalization (BN) is introduced , in which BN can normalize the data in each layer and finally send it to the Softmax layer for action classification.

V. OTHER SPECIFICATION

A. Advantages

1. Monitor health analyze the activity of a person from the information collected different devices .
2. Discover activity patterns which are variable that determine which activity is doing a person.

Detect the human daily life activity.

B. Nonfunctional Requirements

1. Limitations: With 80% accuracy, it is obvious that there are limitations to the system. The most significant limitation is that it will not work with people who have very blur the image and fast movement of person.

VI. FUTURE WORK

Human activity recognition can benefit various applications in fields like smart home monitoring healthcare  services , security surveillance, children etc . In future we can update this application by using object activity recognition in which activities performed by object can also be tracked and analyzed. Application of integrated large datasets can be done to identify the activity taking place as slower rate of time. Even very subtle or minute variations should be recognition by the system. The data of actor performing the anomalous activity can be stored and identification of actor can be done if not caught in first place. Activity that are recurring manner should be stored to save time and space during recognition process. Implementation of such model can also be done in government authority section. Much more developments for improvisation in accuracy and dealing with issue related to optical identify and background clutter of image can be done.

VII. ACKNOWLEDGMENT

Firstly, we would like to express our sincere gratitude to our project supervisor Prof. Swati Gade continuous support of this project and his patient guidance, enthusiastic encouragement and useful critiques, motivation, and immense knowledge. We would like to thank the rest of our thesis committee, Prof. Swati Gade for their guidance helped, insightful comments and encouragement, also for the hard question which incited me to widen my research from various perspectives. Our sincere thanks also go to all the staff members of the department for the valuable information provided by them in their respective fields.

We are grateful for their cooperation during the period of our project. Finally, we would like to thank our family for supporting us spiritually during working on the project and our friends who gave us moral support and encouragement throughout the project study

Conclusion

Prediction of six basic human activities such as walking, joging, standing, sitting, moving upstairs, moving downstairs by using accelerometer data and CNN model. A Human Recognition System has various approaches, such as vision-based and sensor-based, which further categorized into wearables, object-tagged, dense sensing, etc. Before moving further, there also exist some design issues in HAR systems, such as selection of different types of sensors, data collection related set of rules, recognition performance, how much energy is consumed, processing capacity, and flexibility . Keeping all these parameters in mind, it is important to design an efficient and lightweight human activity recognition model. A network for mobile human activity recognition has been proposed using long-short term memory approach for human activity recognition using triaxial accelerometers data.

References

[1] Volume 2020 |Article ID 2132138 | https://doi.org/10.1155/2020/2132138 Huaijun Wang, Jing Zhao, Junhuai Li, Ling Tian, Pengjia Tu, Ting Cao, Yang An, Kan Wang, Shancang Li, \"Wearable Sensor-Based Human Activity Recognition Using Hybrid Deep Learning Techniques\", Security and Communication Networks, vol. 2020, Article ID 2132138, 12 pages, 2020. https://doi.org/10.1155/2020/2132138 [2] Agarwal, P.; Alam, M. A lightweight deep learning model for human activity recognition on edge devices. Procedia Comput. Sci. 2020, 167, 2364–2373. [3] Shugang Zhang, Zhiqiang Wei, Jie Nie, Lei Huang, Shuang Wang, Zhen Li, \"A Review on Human Activity Recognition Using Vision-Based Method\", Journal of Healthcare Engineering, vol. 2017, Article ID 3090343, 31 pages, 2017. https://doi.org/10.1155/2017/3090343 [4] A. Murad and J. Y. Pyun, “Deep recurrent neural networks for human activity recognition,” Sensors (Switzerland), vol. 17, p. 11, 2017.View at: Publisher Site | Google Scholar.

Copyright

Copyright © 2022 Prof. Swati Gade, Shubham Lokare, Tushar Gadekar, Karan Khoje. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.