Unusual Crowd Activity Detection in Video Using CNN, LSTM and OpenCV

Abstract

Suspicious behaviour is dangerous in public areas that may cause heavy causalities. There are various systems developed on the basis of video frame acquisition where motion or pedestrian detection occur but those systems are not intelligent enough to identify the unusual activities even at real time. It is required to recognized scamper situation at real time from video surveillance for quick and immediate management before any casualties. Proposed system focuses on recognizing suspicious activities and target to achieve a technique which is able to detect suspicious activity automatically using computer vision. Here system uses OpenCV library for classifying different kind of actions at real time. The motion influence map has been used to represent the motion analysis that frequently changes the position from one place to another. System uses pixel level presentation for making it easy to understand or identify the actual situation.

Introduction

I. INTRODUCTION

With the increasing importance of security, a great number of surveillance cameras have been installed in private and public places. However, the plethora of video sequences available is overwhelming the human resources monitoring them. To this end, there has been significant interest in a smart surveillance system that can automatically detect unusual or abnormal activities. Over the last decades, many researchers in computer vision and pattern recognition have devoted their efforts toward human action and human-human interaction recognition in video sequences. Recently, abnormal or unusual activity detection in crowded scenes has gained more interest from researchers. Unlike human action or interaction recognition, conventional methods are not applicable to the detection and/or tracking of human subjects in a crowded scene owing to the presence of occlusions, small objects sizes, and other factors. For unusual activity detection in a crowded scene, texture information such as a spatio-temporal gradient, mixture of dynamic textures, and spatio-temporal frequency has been considered an efficient means of detection. In the meantime, other groups have used optical flows that directly characterize motion features in a sequence , e.g., a motion heat map, clustered motion patterns, spatial saliency of the motion feature, crowd prediction using a force field model, optical flow fields, particle trajectory, a social force model, and a local motion histogram. Although motion flow based approaches have shown their efficacy in previous works, we believe it is still important to consider the information on the size of the objects and their interactions. For example, in Fig. 1b, where riding a bicycle is considered an unusual activity, the size of the object and its effect to the nearby pedestrians’ moving directions are important information along with the movement speed. To the best of our knowledge, none of the previous methods has explicitly considered this information, the use of which can be helpful in enhancing the performance. However, as stated above, owing to the inapplicability of human segmentation and tracking in a crowded scene, an alternative approach is needed. In this paper, we propose a novel method to represent the motion characteristics of moving objects by considering their motion flows, sizes, and interactions, simultaneously. Specifically, we define a “motion influence map” that efficiently depicts the underlying motion patterns in a crowded scene. There are various researches have been done in the field of human activity detection but fewer researches found for unusual human activities detection at real time using camera. Computer vision is a challenging approach that can acquire real time human activity.  The objective of the system is to recognize unusual human activity from crowd using motion influence map and OpenCV for prior appraisal against crime in public places using camera. It will help to build or install a system that can work 24x7 for real time surveillance and decrease the crowd based criminal activities and saves us from fatal results.

II. LITERATURE SURVEY

Suspicious exercises on open zones and individual security are in genuine threat. In open territories, a huge number of video reconnaissance frameworks are utilized, for example, streets, detainment facilities, blessed locales, air terminals and grocery stores. Video reconnaissance cameras are not astute enough to perceive irregular exercises even at ongoing. It is important to screen the recognition of suspicious exercises and to check the legitimacy of reconnaissance video. It is required to perceived hurry circumstance at constant from video observation for fast and quick administration.

Zakia Hammal et al. [4] proposed a framework which is Based on Conventional Neural Network that trains for human facial acknowledgment. Framework can be prepared with various outward appearances and track exercises w.r.t. to sentenced articulations. The CNNbased AU location uncovered a comparable change in discoveries concerning newborn child quality between assignments. The exactness rate for acknowledgment right activity or articulation runs between 79 to 93 %.

He Xu et al. [5] proposed a framework which depends on RFID which is a physical sensor. The RFID framework can be isolated into the accompanying three segments: Reader, Tag and Back-end PC framework. Can convey through the peruser and label recieving wires.

The means of crafted by RFID framework are as per the following:

1. The perusers send radio recurrence flag in the encompassing condition, and check whether there is any tag;
2. When the tag in the peruser's reception apparatus perusing range is enacted by its very own recieving wire to speak with the peruser and send its chip electronic code or other information;
3. RFID Reader gets an electronic item code (EPC) or information sign of the tag by reception apparatus; Then the information is decoded and prepared, and it will be sent to the back-end PC framework.

Varsha Shrirang Nanaware et al. [6] made an overview over different executed framework over activity acknowledgment. Various specialists have chipped away at location procedures of different human pursue and activity acknowledgment in an undeniable time moving video, exhaustive writing overview of the ongoing works done by various writers is being given during this energizing and application disapproved of handy examination field. Truth be told, the study/audit paper is finished by U.S. as this can be the spot to start for our investigation deal with "location systems of different human pursue and activity acknowledgment in an undeniable time moving video observation".

Jiahao Li et al. [7] proposed a framework which depends on pyramid vitality map as highlight descriptor for a grouping of casings, it can spare and present the activity history that spatially contrasts and the activities perceived. It depends on bidirectional neural system which can back track the concealed layers and present the most pertinent outcomes. It is likewise powerful for single objective or skeleton however mistakes for different targets.

Nour El Din Elmadany et al. [8] proposed a framework which depends on Biset Globality Locality Preserving Canonical Correlation Analysis, which means to get familiar with the normal component subspace between two sets. The subsequent strategy is Multiset Globality Locality Preserving Canonical Correlation Analysis, which expects to manage at least three sets. It make arrangements of skeletons as informational indexes. The exactness for right acknowledgment rate is 90.1%.

Soumalya Sen et al. [9] proposed a framework which depends on picture parsing procedure. Picture parsing relates various kinds of activities which are performed by human that can be perceived in grouping of casings. Activity arranges as – strolling, running, applauding, running, cycling, surfing, and so forth. It depends on frontal area and foundation connection through which framework improves the closer view item and stores these edges for future correlation. Picture parsing brings together picture division, object discovery or acknowledgment.

III. EXISTING SYSTEM

The Existing framework which is Based on Conventional Neural Network that trains for human facial acknowledgment. Framework can be prepared with various outward appearances and track exercises w.r.t. to sentenced articulations. The CNNbased AU location uncovered a comparable change in discoveries concerning newborn child quality between assignments.

IV. PROPOSED SYSTEM

We describe a method for representing motion characteristics for the detection and localization of unusual activities within a crowded scene. Here, we should note that, we considered two types of unusual activities: local and global. The activities such as the unique appearance of non-human objects or the fast movement of a person when most of the other pedestrians are walking slowly considered as local. Global unusual activities occur across the frame, for example, when every pedestrian within a scene starts to run suddenly to escape from the scene.

Proposed work is able to recognize human activity in crowd and analyze whether the action is usual or unusual. System purely debates with crowd based activities that ensure situations. System uses OpenCV library along with python IDE that deals with best precision. System proposes motion influence map that comprises for correct recognition rate. Recognizing unusual activity from crowd is difficult task especially for sensor networks; computer vision is an effective approach that can acquire real time human activities and later analyzes for uncommon frames.

Also Enhanced the Work with Improvising the Detection for Violence Using CNN-LSTM Model

Conclusion

The systems which have been proposed till now are intended to recognize simple human action such as walking, running and many more but not suitable for crowded area. System which has been proposed is able to recognize unusual human action from crowd and action accordingly using motion influence map and OpenCV. The precision rate is bit higher than other and less researches have been made over this concept. Proposed system is able to work for Prior Appraisal against Crime. The accuracy is 96.42 % which is good enough for recognizing unusual activity in complex backgrounds. The proposed system is capable enough to efficiently recognize the unusual human activity from crowd by using OpenCV and Motion Influence Map, which enhances the accuracy and proficiency of the system up to a great extent. The Unusual Crowd Activity Detection can be implemented in various public places for prior and crime notification that enhances the casualty management. But accuracy is often important which requires enhancing for developing an ideal system that can be implemented practically

References

[1] D. Lee, H. Suk, S. Park and S. Lee, \"Motion Influence Map for Unusual Human Activity Detection and Localization in Crowded Scenes,\" in IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 10, pp. 1612-1623, Oct. 2015. [2] Donghui Song, \"https://www.youtube.com/watch?v=y7Jo_9Z9z0U”, Published on Jun 13, 2018. [3] D. Lee, H. Suk, S. Park and S. Lee, \"Motion Influence Map for Unusual Human Activity Detection and Localization in Crowded Scenes,\" in IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 10, pp. 1612-1623, Oct. 2015. [4] Zakia Hammal1, Wen-Sheng Chu1, Jeffrey F. Cohn1,2, Carrie Heike3, and Matthew L. Speltz4, “Automatic Action Unit Detection in Infants Using Convolutional Neural Network”, 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), IEEE. [5] He Xu1,2, Chengcheng Yuan1, Peng Li1,2, Yizhuo Wang3, “Design and Implementation of Action Recognition System Based on RFID Sensor”, 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD 2017), IEEE. [6] Mrs. Varsha Shrirang Nanaware, Dr. Mohan Harihar Nerkar and Dr. C.M. Patil, “A Review of the Detection Methodologies of Multiple Human Tracking & Action Recognition in a Real Time Video Surveillance”, IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI2017). [7] Jiahao Li†, Hejun Wu? and Xinrui Zhou‡, “PeMapNet: Action Recognition from Depth Videos Using Pyramid Energy Maps on Neural Networks”, 2017 International Conference on Tools with Artificial Intelligence, IEEE. [8] Nour El Din Elmadany, Yifeng He and Ling Guan, “Information Fusion for Human Action Recognition via Biset/Multiset Globality Locality Preserving Canonical Correlation Analysis”, IEEE TRANSACTIONS ON IMAGE PROCESSING, 2018. [9] Soumalya Sen, Moloy Dhar and Susrut Banerjee, “Implementation of Human Action Recognition using Image Parsing Techniques”, 2018 Emerging Trends in Electronic Devices and Computational Techniques (EDCT), IEEE.

Copyright

Copyright © 2023 Thaneshwara M, Janardhan Singh. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.