Review on Human Activity Recognition for Military Restricted Areas

Abstract

Human Activity Recognition(HAR) is an active field of research and scientific development in which various models have been proposed using different methods for identification and categorization of activities using Machine Learning. HAR has reached a remarkable milestone in the area of computer vision. Except for applications in human-computer interactions, surveillance systems and robotics, lately it has extended its applicability in the fields of healthcare, multimedia retrieval, social networking, and education as well. It aims to bring latest technologies together to develop complex assistive system with adaptive capability and learning behaviour. HAR interprets human motion using computer and machine vision technologies to identify and detect simple and complex actions in real-world. This paper presents research made for surveillance of restricted military areas. Our scope is to develop a live monitoring system for tracking the illegal activities done in the restricted area for border security, which is an issue of concern since decades. In this we have introduced a deep learning model that learns to classify human actions without having prior knowledge. The features of image or video set are extracted and detected for detected for classifying whether the activity is illegal or not. Many harmful actions can be avoided or at least have their negative effects reduced as a result of the adoption of this concept. Finally, the activity recognition rate showed a good performance as a result of these findings.

Introduction

I. INTRODUCTION

The recognition and analysis of daily human activities is an attractive area for the researchers due to its effectiveness and wide applications in various domains. Human activity recognition (HAR) aims to provide information on physical activity and to detect simple or complex actions in real world . Human activity recognition is an active research area in last decades due to its applicability in various fields and increasing need for home automation and convenient service for elderly ones. Human activity recognition has been revolutionized in various domains such as healthcare, sports, entertainment etc. It involves behaviour and environment monitoring, activity modelling, data processing and pattern recognition. We use IoT technologies to monitor in real time or to get sensitive data to be analysed for security purposes. Artificial intelligence is one of them which is used to detect a human activity and is trendy nowadays.

Artificial intelligence aims towards building machines that are capable to think like humans. It refers to systems or machines that mimic human intelligence to perform tasks and can continuously improve themselves based on the information they collect. An Artificial Neural Network (ANN) in the field of Artificial intelligence attempts to mimic way nerve cells work in the human brain. It makes understanding of computers easy to make decisions similar to a human-brain. The artificial neural network is trained to behave like interconnected brain cells by programming computers. It utilizes computer software programs that analyse the audio and images from videos in order to recognize humans, vehicles, objects, attributes, and events. It has been proven a very useful technology to detect illegal occurrences to avoid threats and maintain security.

Recognition of human activity is an ability to look over the movements/gestures of the human body and to determine human action/activity. If the majority of routine human tasks can be identified by activity recognition systems, they can either be made simpler or mechanised. The main objective of HAR framework is to observe and analyse human behaviours efficiently, understand the human actions and then to retrieve and process the relative data. Abnormal activity recognition is another area of research for public safety. In public sectors CCTV cameras are used to monitor the crowd activities and track the motion of suspicious activities. HAR has attracted a great amount of attraction in past decades due to its great value in wide range of real-world applications. 

In this study we have proposed a HAR system that collects data and uses ANN for classification in military restricted areas. For compound security, a military system must be able to identify specific human behaviours. To secure military installations or other critical infrastructure, early identification of human activity that suggests a potential danger is required. The variety of human actions makes their automated recognition a real issue.

There are activities that are performed by person such as running, fighting, picking up an item, exchanging items, digging, entering restricted area etc. The area of human action recognition is related to other ways of research that analyse human actions from images and video. We focus on actions and do not explicitly consider context such as the environment, interactions between persons or objects. Moreover, we consider only full-body movements. The system must be able to represent each of these components in order to be able to recognise a wide range of human actions. To identify the focus of attention, the person is distinguished from the rest of the scene. Based on the position of key points system can generate alerts in real time when potential assault is in progress.

II. LITERATURE SURVEY

In this paper they have used two approaches for human activity recognition Conditional Random Field (CRF) and Long Short Term Memory (LSTM) layers. Accelerometers sensors placed on multiple body locations for example ankle, chest and hand for capturing the body motion and later this information use for predicting the output. Another way they used was machine learning to capture the data and then perform comparison. [1]

 In this paper they have used various hardware devices for human activity recognition devices such as fitness trackers, watches, wearable devices and for feature extraction they have used two techniques called using ensemble empirical mode decomposition (EEMD). Then time and frequency domain features are extracted by applying Hilbert-Huang Transform. [2]

In this, wearable devices and smartphone are widely used for detecting human activity recognition. In this paper they have used CNN-LSTM network which tackles all the activities which are captured by digital devices they have used. [3]

This paper focuses on both human and robotic activities so that they can collaborate with each other. Human robot collaboration (HRC) is achieved by combining high-level robot control techniques and human action recognition (HAR) into a single control system. It uses artificial neural network classifier for activity recognition. [4]

In the modern world, millions of people pass away every year because they lack knowledge about their health. If the healthcare system focused more on disease prevention through routine health status assessments and the treatment of diseases in their early stages, rising expenditures may be lowered. Due to this circumstance, numerous researchers are working to create technologies that will use IoT to enhance human life quality. According to the research, IoT technologies can be used to monitor a patient's condition in real time or to collect sensitive data that will then be analysed for a medical diagnosis. These technologies can also help gather information to assess health and provide information to properly trained personnel in diagnosing patients. [5]

Activity detection uses sensor data to recognise and detect both straightforward and intricate actions in natural environments. It is a difficult endeavour since sometimes the sensor data that is produced is unclear regarding the activity that is occurring. Due of this, actions are difficult to interpret. Sometimes the data that is obtained can also be noisy. Human error or network technology flaw that prevents the network from providing accurate sensor readings can both lead to noise in the data. Such real-world situations demand ways to learn from data, to extract knowledge, and to aid in decision-making because they are full with uncertainty. In addition, using inverse probability, one can forecast and infer unknowns. [6]

In this Paper, Accuracy and computational cost are typically used to evaluate Human Activity Recognition (HAR). Previous works have attempted to extract hand-crafted features from the accelerometer and gyroscope signals, and have utilised various traditional machine learning algorithms. [7]

In every aspect of modern life, including education, health, security, agriculture, business, and notably health assessments, technology plays a vital role. Technology-related fields are currently the most researchable topics. This study also examines activity detection, which can be used to track human activity in a variety of contexts, including the health sector to track patient activity and smart cities to leverage the Internet of Things to track domestic activity. In addition to being helpful for crowd anomaly detection and object tracking, activity detection can also be used for security considerations. [8]

Due to the complexity and wide variety of human activities, the topic of human activity recognition (HAR) has recently attracted a lot of attention. In great parts developments in sensing technologies and wireless sensor networks, sensor-based methods to HAR have become particularly popular in pervasive computing. HAR is a crucial element in a wide number of application fields, including as connected health, pervasive computing, security systems, human computer interface (HCI), and ambient assisted living (AAL) in smart home environments. Other prominent interest domains include human/object detection and recognition based on object analysis and processing, for example, tracking and detection, computer engineering, physical sciences, health-related difficulties, natural sciences, and industrial academic disciplines. [9]

In this Paper, they have designed an IMU based activity recognition system that accurately and robustly identifies among 6 different activities. They achieved the state of the art accuracy of  97% by generating a compact 15-D discriminative space, via exploring feature extraction and reduction and discriminative analysis of the resulting space. [10]

III. PROPOSED-SYSTEM ARCHITECTURE

Human activity recognition system helps to detect the human movements in the surroundings. The main aim of this model is to identify and detect the movements happening in the restricted military areas to ensure security

Human activity recognition system helps to detect the human movements in the surroundings. The main aim of this model is to identify and detect the movements happening in the restricted military areas to ensure security.

We are using a very useful deep learning technology for detection and identification of human movements and to track their activities. Long Short-term Memory also known as LSTM is used for video capturing, prediction and classification of images. Due to their ability to understand long-term connections between data time steps, LSTMs are frequently used to learn, analyse, and classify sequential data.

We will be using a webcam which will help to continuously monitor the area and track any activities carried out by humans. After capturing the signal from surrounding the system will process with denoising techniques. This denoising techniques will help to remove the noise from a noisy image and restore the original image. Data filtering will help in the process of image detection, face recognition and whatever task involved in computer vision and improve the edge of an image. Normalization techniques will help to change the range of pixels and reducing the scale from 0-255 to 0-1 range and will convert the image into a collection of pixels which is represented by labelled image by segmentation process. Various image pre-processing techniques like normalization, resizing, segmentation is applied on images. Later, feature extraction techniques will be useful in classifying and recognition of images. It will classify into time and frequency domain, and obtain most relevant information from the original image and represent information in low dimensionality space. The extracted features will be later used for detection of the activities carried out by the humans. If the tracked motion or activity is found illegal it will send an alert or alarm to make aware of the illegal activity carried out in a restricted area. This will ensure the security of the military bases.

IV. ALGORITHM

We are using CNN and LSTM algorithm for implementing this system. A deep convolutional neural network (CNN) is used to automate feature learning in a systematic manner from raw data in order to recognise human activities. A study proposed a deep learning model called Long Short-Term Memory (LSTM) to classify human behaviours and learn without requiring prior knowledge.

Live Capturing will start whenever admin starts the video Surveillance.

1. Taking input from live video capturing.
2. Detecting human using Mediapipe holistic model.
3. Segmentation and feature extraction using CNN algorithm.
4. Detection of activities using LSTM algorithm.
5. Classifying legal and illegal activities using a model trained on dataset.

Conclusion

The description of methodologies and methods used in each process stage, including data collection and filtering, data segmentation, feature extraction, dimension reduction, and classification, served to introduce the idea of HAR processes. This paper provides a real time solution for surveillance of restricted military areas. We propose to implement the given solution with the help of deep learning techniques and a comparative study of different methodologies. Deep neural network substantially increases the output by automatically learning features from raw data, making motion tracking a promising application. In addition, we demonstrate that our method is accurate and versatile. It can recognize the human actions and detect the illegal or abnormal activities accordingly assuring security of the areas. Although the proposed method can achieve better results than other methods for target detection, the accuracy of activity recognition still needs to be improved. Other possible problems, such as the identification of complex actions, could not be considered in this research. Solving the identification and accurate detection of these complex actions will become the next main step for research.

References

[1] J. Okai, S. Paraschiakos, M. Beekman, A. Knobbe and C. R. de Sá, \"Building robust models for Human Activity Recognition from raw accelerometers data using Gated Recurrent Units and Long Short Term Memory Neural Networks,\" 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2019, pp. [2] E. W. Sinuraya, A. Rizal, Y. A. A. Soetrisno and Denis, \"Performance Improvement of Human Activity Recognition based on Ensemble Empirical Mode Decomposition (EEMD),\" 2018 5th International Conference on Information Technology, Computer, and Electrical Engineering (ICITACEE), 2018, pp. 2486-2491. [3] S. Mekruksavanich and A. Jitpattanakul, \"Smartwatch-based Human Activity Recognition Using Hybrid LSTM Network,\" 2020 IEEE SENSORS, 2020, pp. 1-4. [4] T. Uzunovic, E. Golubovic, Z. Tucakovic, Y. Acikmese and A. Sabanovic, \"Task-Based Control and Human Activity Recognition for Human-Robot Collaboration,\" IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society, 2018, pp. 5110-5115. [5] S. Oniga and J. Süt?, \"Human activity recognition using neural networks,\" Proceedings of the 2014 15th International Carpathian Control Conference (ICCC), 2014, pp. 403-406. [6] Singh, D. et al. (2017). “Human Activity Recognition Using Recurrent Neural Networks.” In: Holzinger, A., Kieseberg, P., Tjoa, A., Weippl, E. (eds) Machine Learning and Knowledge Extraction. CD-MAKE 2017. Lecture Notes in Computer Science(), vol 10410. Springer, Cham. [7] R. R. Koli and T. I. Bagban, \"Human Action Recognition Using Deep Neural Networks,\" 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), 2020, pp. 376-380. [8] Geravesh, Shahab & Rupapara, Vaibhav. (2022) “Artificial neural networks for human activity recognition using sensor based dataset”. Multimedia Tools and Applications. 10.1007/s11042-022-13716-z. [9] Irvine, Naomi, Chris D. Nugent, Shuai Zhang, Hui Wang and Wing W. Y. Ng. “Neural Network Ensembles for Sensor-Based Human Activity Recognition Within Smart Environments.” Sensors (Basel, Switzerland) 20 (2020): n. pag. [10] O. Dehzangi and V. Sahu, \"IMU-Based Robust Human Activity Recognition using Feature Analysis, Extraction, and Reduction,\" 2018 24th International Conference on Pattern Recognition (ICPR), 2018, pp. 1402-1407, doi: 10.1109/ICPR.2018.8546311.

Copyright

Copyright © 2022 Prof. Sonali Patil, Siddhi Shelke, Shivani Joldapke, Vikrant Jumle, Sakshi Chikhale. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.