Person Re-Identification Using Machine Learning

Abstract

The goal of this research is to develop a system that can spot real-world anomalies in surveillance footage, like burglaries and assaults. Although different methods for anomaly detection have been explored in prior research, the significance of locality within the frame has not been fully explored. Given the rise in crime rates in many nations, accurate identification of people is essential for deterring and solving crimes. However, storing and processing information about suspects using conventional paper-based systems can be labor-intensive and time- consuming. This illustrates the need for more effective techniques for identifying dead bodies after a crime, which can be done using machine learning and image processing methods. Instead of using whole-frame video segments, this study explores the use of spatiotemporal tubes and presents a novel dataset for crime scene detection with bounding box supervision in both the train and test sets. The experiments show that a network trained with spatiotemporal tubes outperforms a model trained with whole-frame videos, with the locality remaining reliable even when extraction errors occur. Additionally, the network can expand the spatiotemporal crime scene dataset without the need for additional human labelling by producing spatiotemporal proposals for fresh surveillance videos using only video-level labels.

Introduction

I. INTRODUCTION
In order to improve public safety, surveillance cameras are now frequently used in public spaces like streets, intersections, banks, and shopping centres. However, law enforcement organisations are unable to keep up with the expansion of cam- eras, which leads to an unfeasible camera-to-human monitor ratio and underuse of surveillance technology. A crucial task in video surveillance is the detection of anomalous events, such as traffic accidents, crimes, and illegal activities. However, since crime scene incidents are uncommon, it is essential to create clever computer vision algorithms for automatic video object detection to cut down on labour and wasted time. A practical crime scene detection system’s main objective is to quickly identify abnormal activity and pinpoint the crime scene’s time window. In video surveillance, crime scene detection is a critical task for spotting anomalous events like accidents, crimes, or illegal activities. It is necessary to develop intelligent computer vision algorithms for automatic video object detection because current algorithms that detect specific crime-related events, like violence or traffic accidents, cannot be generalised to detect other crime scene events. The most current representative methods that produce cutting-edge object detection results use sparse-coding techniques, but these techniques have high false alarm rates for various normal behaviours. The crime scene/dead body identification detection algorithm should therefore be able to detect objects with little supervision and require little prior knowledge of the events.

A. Background

Identifying people is a crucial task with many applications, including security, human-computer interaction, and other areas. A person can be identified using a variety of methods, including fingerprint, vocal sound, or DNA tests. But because it is non-contact, simple to use, and a topic of research in the fields of pattern recognition and machine vision, facial image recognition is a widely used and well-liked technique.

B. Problem Statement

A Crime Scene Perception Network for Video Surveillance Based on Machine Learning has been proposed because it is getting harder to find and identify criminals and dead bodies. This model’s implementation is meant to be a component of any urban safety strategy meant to reduce and prevent issues with crime and insecurity.

II. LITERATURE REVIEW

A. Title: A Novel Deep Learning Framework for Person Detection.

Authors: Linxiang Zhao, Yi Wan.

Summary: Based on the Yolo3 network, the authors of this study present the PDnet, a brand-new person detection network model. They enhanced the anchor box clustering algorithm, unified the output ports to one, and the Yolo3 feature extraction network architecture to better extract person-specific features and speed up convergence. According to the results, PDnet outperforms the original Yolo3 algorithm in terms of person detection, resilience, and precision.

B. Title: Automated Criminal Identification by Face Recog- nition using OpenCV classifiers.

Author: Apoorva.P, Impana.H.C, Siri.S.L, Varshitha.M.R, and Ramesh.B.

Summary: The paper suggests a real-time face recognition system for automated surveillance that uses the Haar cascading algorithm. The system includes face detection, image comparison, and result analysis in addition to real-time image training. The suggested system has a high level of accuracy and robustness for identifying people on a watch list. It is effective for use in large-scale applications because it can quickly and efficiently recognise multiple faces with little computation time. Integrating the system with Aadhaar, the Indian citizenship database, will enable it to distinguish between Indian citizens and foreigners and look up the criminal history of the person who has been identified.

C. Title: Criminals and Missing Children Identification Using Face Recognition and Web Scraping

Author: S. Ayyappan, Dr. S. Matilda

Summary: The Haar-classifier algorithm is used in this paper to demonstrate a real-time face recognition system for automated surveillance. The system includes face detection, image comparison, face training, and result analysis. The suggested system is made to effectively and accurately identify people on a watch list. On the OpenCV platform, the system detects faces using the Haar cascading algorithm, which detects multiple faces quickly and with little computational effort. The system’s potential can be increased by linking it to the Aadhaar citizenship database in India, which allows users to distinguish between Indian citizens and foreigners and further explore an individual’s criminal history. The paper also suggests web scraping methods to gather data on missing children and criminals, which can be added to the watch list for more better      identification

D. Title: Remote Compact Seismic Sensor for Moving Per- son Detection

Author: Pavel P. Fastykovsky and Yaroslav I. Lepikh

Summary: The creation of a remote seismic sensor for tracking moving people is described in this letter. The authors present a person detection algorithm and discuss the selection of seismic transducer and signal processing unit. When the characteristics of seismic signals from moving subjects are examined, it is discovered that the sensor has high-level parameters suitable for object guarding and reconnaissance. Without requiring complicated or energy-intensive processing techniques, the device uses the temporal parameters of seismic signals.

E. Title: A Privacy-Protected Fall Detection IoT System for Elderly Persons Using Depth Camera

Author: Xiangbo Kong, Zelin Meng, Lin Meng and Hiroyuki Tomiyama

Summary: In order to address the privacy concerns of elderly people living alone, this paper presents a novel approach to fall detection using an IoT system. The suggested system employs a depth camera to take images of the skeleton, which are then encrypted using the Fast Fourier Transform (FFT), and then examined to look for falls. The system trains the model on the encrypted skeleton images using machine learning algorithms. The experimental testing of the system demonstrates that it is successful in identifying falls while protecting the users’ privacy. By quickly identifying falls, the proposed system has the potential to lower the risk of injury or even save lives.

F. Title: Research on Recognition of Criminal Suspects Based on Foot Sounds

Author: Fuxiang Liu, Qi Jiang

Summary: Based on frequency domain and Mel domain features, this paper presents a novel technique for differentiating between footstep events and non-footstep events. Additionally, the study looks at the peak and trough frequencies of footstep sound signals in the main frequency band for the same person under various experimental setups and discovers that they coincide one to one. The main frequency band of footstep sound signals, which can be used to identify suspects in criminal investigations, does not always contain the same frequencies for different people.

G. Title: Real-Person Re-identification Through Face Detection from Videos Using Deep Learning

Author: Vimala Mathew, Tom Toby, Anu Chacko, Udhayakumar A

Summary: With a focus on person re-identification from surveillance videos, this paper emphasises the value of real- time video analysis in enhancing security and preventing crimes.

The method for re-identification of real people through face detection is suggested in the paper. Convolutional Neural Network models are created using cropped face images from video frames. The suggested technique can greatly increase the effectiveness and accuracy of person re-identification in video analytics.

H. Title: Multiple Static Person Localization using UWB Radar for Respiratory Motion Detection

Author: Daniel Novak and Du ´san Kocur

Summary: This study describes a technique for locating multiple stationary people using ultra-wideband (UWB) radar to detect respiratory motion.

The suggested method detects respiratory motion and then localises the target using a two- stage detector and the Welch periodogram method. The ex- perimental findings demonstrate the viability of the suggested approach for locating three static individuals through walls. The technique may be used in many different contexts, such as security, health monitoring, and search and rescue operations.

I. Title: Person Identification using Smart Cameras

Author: Leon Rothkrantz

Summary: In order to ensure safety in smart cities, this paper discusses the rising demand for smart surveillance cameras. In order to identify and apprehend criminals during or shortly after the crime, the paper emphasises the necessity of real-time processing. The author suggests utilising surveillance cameras with facial recognition software to identify and find suspect people. The study points out the significance of striking a balance between security requirements and privacy concerns.

J. Title: Machine Learning-Based Person Identification in Group Photos.

Authors: Prof. Avani Sakhapara, Prof. Dipti Pawade, Mr. Saumil Dedhia, Ms. Twinkle Bhanushali, Mr. Vinit Doshi.

Summary: Group photos are more common now that smartphone photography is becoming more and more popular. How- ever, it can be time-consuming to manually identify everyone in a group photo. This study suggests “EuphoriaGrouping” (EUG), an Android app that employs neural networks to automatically recognise faces and identify people in group photos. In this paper, the implementation and effectiveness of two convolutional neural network models—Custom Built and OpenFace CNN—used for person identification are compared.

III. ALGORITHM

A. Haarcascade Algorithm

Based on features, the object detection algorithm known as Haar Cascade is used to identify objects in images. A large number of positive and negative images are used to train a cascade function in the algorithm’s detection step. This algorithm’s benefit is that it can operate in real-time and doesn’t call for a lot of computation. Furthermore, we are able to train our own cascade function for unique objects like animals, vehicles, bicycles, faces of people, etc.

The OpenCV library makes the implementation incredibly simple by offering a repository on GitHub for all popular pre-trained haar cascades files that can be used for a variety of object detection tasks, including human face detection. Creating a custom cascade function requires a lot of data to train on.

In order to function, the Haar-like feature takes into consideration nearby rectangular regions at a particular spot in a detection window. Then it calculates the difference between the sums of the pixel intensities in each region. Subdivisions of an image are then categorised using this distinction.The process of object detection can be significantly sped up by using a pre-trained Haar cascade model. This technology enables programmers to create robust computer vision applications that can recognise objects in real-time and have a variety of uses, including security systems, autonomous vehicles, and more.

1) Module

Input Dataset- Live camera

Since real-world data is frequently noisy, may contain missing values, and may be in an unusable format, data preprocessing is an essential step in machine learning. The data must be cleaned and transformed into a format that is appropriate for machine learning models through preprocessing. The machine learning model’s accuracy and effectiveness are enhanced by this step.

The selection of a subset of input variables to concentrate on while ignoring others is the process of feature extraction. For machine learning algorithms, this process can be difficult, but humans naturally use feature selection and feature extraction.

The accuracy of the model is significantly impacted by the feature extraction algorithms’ dimensionality reduction.

The process of classifying data into distinct classes is known as classification. Data that is unstructured or structured can both be processed using this method. By roughly estimating the mapping function from input variables to discrete output variables, classification predictive modelling aims to identify the class/category that new data will belong to. The Haarcascade algorithm is employed in this system for the classification of person identification.

2. Data Flow Diagram
A system’s data flow is shown visually in a data flow diagram (DFD). Rectangles are used as input and output in the base diagram (DFD0), and circles are used to represent the system itself. The DFD1 shows the system’s actual input and output, where the input can take the form of text or an image and the output is rumour detection. The DFD2 also shows how the system works for both users and administrators.

IV. RESULT

This study presents a novel machine learning-based method for identifying lost people, regular people, and actual criminal activity in real-time video footage. The researchers emphasise that in order to achieve the best results in identifying criminals or missing persons, it is necessary to use both regular data and crime surveillance videos due to the complexity of real-world crime scenes. With weakly labelled data, the researchers learn a general model of criminal detection using a deep multiple instance ranking framework. With this method, the model can gain knowledge from both positive and negative examples without requiring a lot of manual annotation. The suggested model has the potential to increase the precision and speed of criminal detection and lost person identification by utilising both routine and criminal surveillance videos. The study’s conclusions point to the possibility of using this machine learning-based strategy to enhance public safety and security in a variety of real-world situations.

Conclusion

In conclusion, our research has suggested a machine learn- ing method for detecting criminal activity, missing people, and dead bodies in actual situations using surveillance videos. The suggested method increases the precision of identifying criminals and bodies by using both regular and criminal surveillance videos. We have developed a general model for criminal detection without the need for time-consuming temporal annotations of crime segments in training videos by using deep multiple instance ranking frameworks with weakly labelled data. Our method may assist law enforcement organ- isations and search and rescue crews in more effectively and precisely identifying and locating missing people, criminals, and dead bodies, ultimately assisting in the prevention and solving of crimes.

References

[1] B. Amos, B. Ludwiczuk, M. Satyanarayanan, “OpenFace: A General Purpose Face Recognition Library With Mobile Applications”, Carnegie Mellon University, Pittsburgh, 2016 [2] E. Learned-Miller, G. Huang, A. RoyChowdhury, H. Li, G. Hua, “Labeled Faces in the Wild: A Survey”, Springer, pp. 189-248, 2016. [3] M. Chihuahua, A. Elke, W. Belly, Chore Ben Amar, “A Survey of 2D Face Recognition Techniques.” Computers, 2016 [4] Jurgen Sachs and Ralf Herrmann, “M-sequence- based ¨ ultra wideband sensor network for vitality monitoring of elders at home,” Radar, Sonar Navigation, IET, vol. 9, no. 2, pp. 125–137, 2015. [5] Wei Hsu, T. Yu Wu, W. Hung Wong and C. Yi Lee, ”CorrelationBased Face Detection for Recognizing Faces in Videos,” in IEEE International Conference on Acoustics, Speech and Signal Processing, Calgary, 2018 [6] Rui Zhao, Wanli Oyang, and Xiaogang Wang, ”Person ReIdentifi- cation by Saliency Learning,” IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 39, no. 2, pp. 356- 370,February 2017. [7] Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao, ”Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks,” IEEE SIGNAL PROCESSING LETTERS, vol. 23, no. 10, pp. 1499-1503, October 2016. [8] Muhammad Usman Yaseen, Ashiq Anjum, and Omer Rana , ”Deep Learning Hyper-Parameter Optimization for Video Analytics in Clouds,” IEEE TRANS ACTIONS ON SYSTEMS, MAN, AND CYBERNET- ICS: SYSTEMS, vol. 49, no. 1, pp. 253-264, January 2019. [9] Zhenfeng Shao, Jiajun Cai, and Zhongyuan Wang, ”Smart Monitoring Cameras Driven Intelligent Processing to Big Surveillance Video Data,” IEEE TRANSACTIONS ON BIG DATA, vol. 4, no. 1, pp. 105-116, January 2018. [10] Li Cuimei, Qi Zhiliang, and Wu Jianhua, ”Human face detection algorithm via Haar cascade classifier combined with three additional classsifier,” in

Copyright

Copyright © 2023 Yogesh Dange, Pratik Jadhar, Kshitija Shirke, Harshal Sangle, Prof. Girish Navale. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.