Unveiling Anomalies in Surveillance Videos

Abstract

The implementation of Convolutional Neural Network (CNN) algorithm for the detection of anomalies like fire and potholes in images or videos. Leveraging deep learning technique, the model aims to accurately identify fire, potholes, etc. within various contexts. The CNN architecture is trained on a dataset comprising diverse images to enhance its ability to generalize. This proposed project not only addresses safety concerns by promoting helmet usage and proper maintenance of roads but also contributes to the broader field of computer vision and object detection. This implementation involves training the CNN on annotated datasets, fine-tuning the model for optimal performance, and integrating it into a practical application for efficient anomaly identification. With a focus on real-time detection, the system utilizes image processing techniques and a trained CNN model to analyze visual data from cameras or video feeds. The system\'s versatility allows integration into surveillance systems, industrial sites, or any environment where fire detection is crucial for safety. By automating the detection process, the project contributes to minimizing human error and ensuring consistent monitoring. The proposed solution holds the potential to significantly impact safety protocols, particularly in industries where protective headgear is paramount. Potholes pose a significant threat to both drivers and pedestrians, leading to accidents and infrastructure damage. The proposed solution leverages CNN\'s ability to effectively process visual data, making it well-suited for image recognition tasks. Our approach involves training the CNN on a diverse dataset of road images to enable it to accurately identify potholes. The model will learn distinctive features and patterns associated with potholes, allowing for robust detection under various environmental conditions. Real-time implementation on embedded systems or cameras along roadways will enable instantaneous identification and alerting.

Introduction

I. INTRODUCTION

In response to the escalating need for heightened security measures, an Anomaly Recognition System has been proposed to address the challenge of manually monitoring the vast volume of surveillance videos. Leveraging advanced Deep Learning models, specifically Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN), this system aims to automate the detection and classification of offensive or disruptive activities in real-time. Videos are categorized into two main groups: Threat (encompassing anomalous activities such as Abuse, Road Accidents, and Assault) and Safe (representing normal activities). The foundational component, CNN, plays a crucial role in extracting intricate feature maps from surveillance recordings, thereby simplifying the input complexity. To further enhance the system's adaptability, transfer learning is employed, initially utilizing a pre-trained model like ImageNet and subsequently retraining it with new weights assigned to specific classes. This approach optimizes the model's performance in detecting various anomalous activities.

The output from CNN is then fed into an RNN, providing an additional forecasting capability. RNN's ability to predict the next item in a sequence adds a nuanced understanding of the captured actions or movements in surveillance recordings. This dynamic integration of CNN and RNN facilitates real-time surveillance on CCTV cameras across diverse organizations. The ultimate goal is to streamline the surveillance process, automating the detection and categorization of suspicious activities promptly. By doing so, the system significantly reduces the time complexity associated with manual monitoring, offering an efficient and accurate approach to security management in the face of evolving threats.

II. LITERATURE SURVEY

1. Ashraf, T. Abbas and A. Ejaz, “Magnetic Anomaly-Based Detection of a Submarine”:

Seawater provides a natural hideout for seaborne vehicles and weapons. So the detection of seaborne objects/vehicles has been an area of strategic interest. Earth's magnetic field is a global phenomenon that travels in a straight path deviating only from the presence of permeable objects. This deviation from the straight path can be sensed passively by a magnetic sensor.

Magnetic anomaly detection (MAD) is a technology used to detect submarines based on the principle that a moving magnetic object will disturb the Earth's magnetic field.

2. S. K. Singh, M. H. Anisi, S. Clough, T. Blyth and D. Jarchi, "CNN-BiLSTM based GAN for Anomaly Detection from Multivariate Time Series Data,"

Continuous recording of sensor data for monitoring applications does require detection of data patterns which deviate from normal condition. Detection of such events is necessary for implementing early preventive methods to improve overall system performance and potentially identify sensor failure causes. Recently, deep learning techniques based on generative models such as generative adversarial network (GAN) are proposed for anomaly detection from multiple time-series data. In this research, a variant deep learning method-based GAN is proposed for anomaly detection from multivariate time series data. Based on our proposed approach, the generator block consists of both CNN and BiLSTM blocks whilst the discriminator uses BiLST.

3. H. Malik, S. K. Panda, K. Pootla and C. J. Spanos, "Data-Driven Hybrid Approach for Early Fault Detection of AHU using Electrical Signals,"

Available online health monitoring systems (HMS) using mechanical signals such as vibration & temperature for ACMV (Air-Conditioning and Mechanical Ventilation) system detect some of the critical faults only at high severity levels resulting in higher operation and maintenance (O&M) cost. Moreover, multiple monitoring systems are required one for each single component at the sub-system level further decreasing affordability. In this paper, a unique, single hybrid scheme involving both feature extraction and classification using electrical signals based holistic HMS for various types of critical faults of an AHU (Air Handling Unit) and its associated component in the ACMV system is proposed. The proposed approach is capable of detecting anomalies at an early stage and provides efficient condition monitoring and predictive maintenance (PdM) scheduling in advance using mostly electrical signals such as power.

4. S. Makkena, S. M. Buchi, L. A. K. Yalamati, M. R. L, S. V and S. K. Bommavaram, "Anomaly Detection for Wafer Manufacturing using IoT and Machine Learning Techniques,"

The development of brilliant sensor organizations and the Internet of Things (IoT), which have opened up before-hand unfathomable potential outcomes in the modern climate, has started the ascent of Industry 4.0 and savvy fabricating. Measurable AI (ML) approaches might be utilized to extract information from the information that would be challenging for even human experts to get since these advances grant a consistently expanding volume of information. Programmed irregularity discovery frameworks can help producing organizations by lessening how much personal time is welcomed on by machine glitches and by spotting issues before they make a sad difference. Without the cost of recruiting costly human topic subject matter experts, this is possible.

5. H. Anand, B. S. Sammuli, K. E. J. Olofsson and D. A. Humphreys, "Real-Time Magnetic Sensor Anomaly Detection Using Autoencoder Neural Networks on the DIII-D Tokamak,"

Magnetic diagnostics in tokamaks (a toroidal apparatus for producing controlled fusion reactions in hot plasma.) are key to plasma equilibrium control (plasma current, plasma shape, and position) and amelioration of plasma instabilities. Thus, real-time identification of the anomalous sensor is mandatory. A new real-time system based on autoencoder (AE) neural networks (NNs) for anomaly detection in magnetics signals, including both flux loops and magnetic probes, has been successfully implemented on the plasma control system (PCS) of the DIII-D tokamak. The AE NN is trained on over 4000 plasma discharges, with an optimized latent space representation of the input signals while minimizing the reconstruction loss. An algorithm determining anomalous sensors based on excessive deviations from accurate reconstruction from the trained NNs is constructed in a MATLAB/Simulink environment and is deployed on the real-time PCS using the embedded MATLAB coder (EMC) environment.

6. S. K. Gupta et al., "Anomaly Detection in Very Large Scale System using Big Data,"

Big data refers to a term that is used to describe vast amounts of data that have multiple kinds of Vs: velocity, variety, and volume. It could be semi-structured, unstructured, or even structured, making data analysis difficult. New architecture, methodologies, algorithms, and analytics are needed to extract hidden data and identify assaults on enormous amounts of data. It is quite challenging to identify assaults using conventional methods. This study provides a thorough analysis of malware detection in several sectors using deep learning and provides an overview of deep learning data. In networked computers, there have been more attacks. To protect a network, a strong intrusion detection system (IDS) is necessary.

7. AI & ML Based Anomaly Detection and Response Using Ember Dataset V. Rathod, C. Parekh and D. Dholariya

In the era of rapid technological growth, malicious traffic has drawn increased attention. Most well-known offensive security assessment todays are heavily focused on pre-compromise. The amount of anomalous data in today's context is massive. Analyzing the data using primitive methods would be highly challenging. Solution to it is: If we can detect adversary behaviors in the early stage of compromise, one can prevent and safeguard themselves from various attacks including ransomwares and Zero-day attacks. Integration of new technologies Artificial Intelligence & Machine Learning with manual Anomaly Detection can provide automated machine-based detection which in return can provide the fast, error free, simplify & scalable Threat Detection & Response System.

8. M. Rajeshwari and C. M. Rao, "Road Traffic Anomaly Detection using AI Approach: survey paper,"

Every day somewhere on the road's accident happens because of unexpected interference of vehicles and unpredictable driving by the driver. There is a ton of research about predicting and detecting vehicle accidents, yet there is no pre intimation to the drivers about the accident Streetcar crashes claim an enormous number of lives each day as a result It is usually the result of a driver's lapse of caution or a late response from emergency services. There is a need for an effective road accident identification system and data correspondence system for harmed people to be saved. It is not possible for a framework to convey data messages about an accident area to crisis management agencies for a quicker and more effective response. Numerous scientists propose different frameworks for programming accident recognition in exploration writing. Cell phones and GSM and GPS technologies aid in the identification of accidents.

9. “Real-World Anomaly Detection in Surveillance Videos” by Waqas Sultani; Chen Chen; Mubarak Shah

Through this paper, we propose to learn anomalies by exploiting both normal and anomalous videos. To avoid annotating the anomalous segments or clips in training videos, which is very time consuming, we propose to learn anomaly through the deep multiple instance ranking framework by leveraging weakly labelled training videos, i.e. the training labels (anomalous or normal) are at video-level instead of clip-level. In our approach, we consider normal and anomalous videos as bags and video segments as instances in multiple instance learning (MIL), and automatically learn a deep anomaly ranking model that predicts high anomaly scores for anomalous video segments. Furthermore, we introduce sparsity and temporal smoothness constraints in the ranking loss function to better localize anomaly during training.

10. J. D. S. W.S. and P. B., "Machine Learning based Intrusion Detection Framework using Recursive Feature Elimination Method,"

Intrusion detection has a prominent part for ensuring the information security. Machine learning approaches are followed to detect intrusion or anomaly of a network. The network traffic produce large amount of data, the Analyzing and monitoring the data is the biggest challenge here. To overcome that feature elimination or selection is done before classification. The dataset has some features which are irrelevant which makes the detection process slower and degrades the system performance. In order to improve the performance, this system identifies the features which are irrelevant and eliminated it. The feature selection is achieved by using Recursive Feature elimination method.

Conclusion

We are introducing an innovative Anomaly Detection System for surveillance, aimed at seamlessly integrating alerts through buzzer, OLED display, and notifications, specifically tailored for detecting potholes and fires. This conceptual application, meticulously designed for efficiency and user-friendliness, is seamlessly compatible with Android devices, offering a user-friendly interface for initiating anomaly alerts and receiving real-time notifications. Furthermore, notifications serve as an additional layer of timely information dissemination, enabling users to stay informed about detected anomalies even when they are not directly monitoring the surveillance feed. This comprehensive approach not only prioritizes user experience but also significantly enhances the effectiveness of response strategies to potholes and fires. In the context of pothole detection, our system employs advanced algorithms to accurately identify road defects, thereby mitigating potential hazards and ensuring road safety. The integration of buzzer alerts and OLED displays enables immediate notification of potholes to nearby vehicles or relevant authorities, facilitating prompt repair actions and reducing the risk of accidents. In the case of fire detection, our system leverages cutting-edge algorithms to swiftly identify fire outbreaks in surveillance footage. The integration of buzzer alerts and OLED displays ensures prompt notification of fire incidents to relevant stakeholders, enabling timely evacuation procedures and firefighting efforts to mitigate damages and safeguard lives. In summary, our Anomaly Detection System for surveillance offers a comprehensive solution for detecting potholes and fires, with a focus on reliability, user-centric design, and seamless integration with Android devices. By combining immediate alerts through buzzer and OLED display with timely notifications, we aim to elevate user experience and effectiveness in responding to anomalies, ultimately contributing to enhanced safety and security in various environments. Enhanced Edge Computing Capabilities: Optimizing edge computing on Raspberry Pi for complex anomaly detection by leveraging hardware accelerators like GPUs or custom ASICs, enhancing inference speed and reducing power consumption. Multi-Sensor Integration: Integrate additional sensors such as LiDAR or infrared cameras to provide complementary data for more robust anomaly detection. Combining data from multiple sensors can improve detection accuracy and reduce false positives/negatives, especially in challenging environmental conditions. Scalability and Deployment Flexibility: Design the system with scalability in mind to support deployment in diverse environments, like forests and remote areas. Considerations should include ease of installation, maintenance, and scalability to accommodate varying surveillance needs.

References

[1] Ashraf, T. Abbas and A. Ejaz, \"Magnetic Anamoly-Based Detection of a Submarine,\" 2023 International Multi-disciplinary Conference in Emerging Research Trends (IMCERT),vol. 1, IEEE,2023,pp. 1-5. [2] S. K. Singh, M. H. Anisi, S. Clough, T. Blyth and D. Jarchi, \"CNN-BiLSTM based GAN for Anamoly Detection from Multivariate Time Series Data,\" 2023 24th International Conference on Digital Signal Processing (DSP),IEEE,2023,pp. 1-4. [3] H. Malik, S. K. Panda, K. Pootla and C. J. Spanos, \"Data-Driven Hybrid Approach for Early Fault Detection of AHU using Electrical Signals,\" 2022 International Power Electronics Conference (IPEC-Himeji 2022-ECCE Asia), IEEE, 2022, pp. 1365-1371. [4] S. Makkena, S. M. Buchi, L. A. K. Yalamati, M. R. L, S. V and S. K. Bommavaram, \"Anamoly Detection for Wafer Manufacturing using IoT and Machine Learning Techniques,\" In 2022 6th International Conference on Electronics, Communication and Aerospace Technology, IEEE, 2022, pp. 428-431. [5] H. Anand, B. S. Sammuli, K. E. J. Olofsson and D. A. Humphreys, \"Real-Time Magnetic Sensor Anomaly Detection Using Autoencoder Neural Networks on the DIII-D Tokamak,\" IEEE Transactions on Plasma Science 50, no. 11 (2022): 4126-4130. [6] S. K. Gupta et al., \"Anamoly Detection in Very Large Scale System using Big Data,\" In 2022 International Conference on Knowledge Engineering and Communication Systems (ICKES), IEEE, 2022, pp. 1-6. [7] AI & ML Based Anamoly Detection and Response Using Ember Dataset V.Rathod,C. Parekh and D. Dholariya. In 2021 9th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions)(ICRITO), IEEE, 2021, pp. 1-5. [8] M. Rajeshwari and C. M. Rao, \"Road Traffic Anamoly Detection using AI Approach: survey paper,\" In 2021 5th International Conference on Electronics, Communication and Aerospace Technology (ICECA), IEEE, 2021, pp. 845-848. [9] “Real-World Anomaly Detection in Surveillance Videos” by Waqas Sultani; Chen Chen; Mubarak Shah. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2020,pp. 6479-6488. [10] J. D. S. W.S. and P. B., \"Machine Learning based Intrusion Detection Framework using Recursive Feature Elimination Method,\" In 2020 International Conference on System, Computation, Automation and Networking (ICSCAN), IEEE, 2020, pp. 1-4.

Copyright

Copyright © 2024 Dr. Prerana Chaithra, Nidhi A M, Madhu Shree S, Manjunath B N, Neha Kulkarni. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.