Face Detection from CCTV Footage using OpenCV and Haar Cascade

Abstract

With the rapid growth of surveillance systems, the need for efficient and accurate face detection in closed-circuit television (CCTV) footage has become increasingly crucial. Face detection plays a pivotal role in various applications, including security, law enforcement, forensic investigations, and social monitoring. This research aims to provide a comprehensive review and analysis of the existing techniques and approaches for face detection in CCTV footage. The study begins by examining the challenges and complexities associated with face detection in CCTV footage, such as variations in lighting conditions, camera angles, occlusions, and image quality. A thorough analysis of the existing literature is conducted, encompassing traditional methods such as Viola-Jones. The findings of this study contribute to enhancing the understanding of face detection in CCTV footage and provide valuable insights for researchers, practitioners, and system developers. The analysis of different approaches and their performance under varying conditions will assist in the design and implementation of robust face detection systems for real-world applications, ultimately improving the effectiveness and reliability of surveillance and security systems.

Introduction

I. INTRODUCTION

The rapid advancements in Machine Learning (ML) have revolutionized various fields by bridging the gap between human brain capabilities and machines. The human brain's innate ability to make decisions and interpret information has long been considered unique. However, ML algorithms that utilize leveraging mathematical foundations such as Calculus, Algebra, Probability, and Statistics have enabled machines to analyze data and generate predictions. This has resulted in the development of its applications ranging from facial recognition and weather prediction to self-driving cars and recommendation engines. Among the numerous applications of ML, facial analysis plays a pivotal role, involving the detection and extraction of facial features from images or videos. Facial analysis encompasses multiple phases, including face detection, facial feature extraction, face recognition, and subsequent refinements like motion capture and emotion analysis.

The related technologies of facial analysis, such as face alignment, face modelling, face relighting, face recognition, and others, rely on the initial step of face detection. Face detection, although seemingly effortless for humans, poses significant challenges for computer systems, making it a heavily researched academic area in recent years. Factors such as age, skin color, facial expressions, lighting conditions, resolution, occlusion, and disguises contribute to the complexity of this task. An effective face detector must be able to identify faces under various background and lighting conditions.

Initially, face detection involves a classification task that provides a binary output indicating the presence or absence of a face in an image. The subsequent stage of face localization determines the precise location of the face within the image, typically represented by bounding box coordinates.

Facial features are then identified through either feature-based techniques that extract and compare specific facial attributes or image-based methods that seek similarities between training and test photos. The focus of the presented prototype is specifically on face detection. This study aims to evaluate the effectiveness and accuracy of haar cascade classifiers in detecting faces from input videos under varying conditions.

By assessing the performance of these classifiers, this research seeks to contribute to the advancement of face detection techniques and their potential applications in real-world scenarios. In this thesis, we will explore the challenges and methodologies associated with face detection, provide an overview of existing algorithm, and conduct an empirical evaluation of haar cascade classifiers to assess their performance and effectiveness in detecting faces from input videos. The findings from this research will not only deepen our understanding of face detection techniques but also contribute to the development of more robust and accurate face detection systems.         

II. LITERATURE REVIEW

A review was done on face detection using haar cascade classifier by Dhirajkumar Kharat, et.al(2019)[1]. The study compares Local Binary Pattern (LBP), a texture descriptor and haar cascades. LBP was found to be an efficient technique for identifying and recognising faces but less precise than the Haar cascade classifier. Although Haar features are capable of detecting human face features, there are still some detection failures. The three essential parts of the Viola Jones algorithm are the adaboost, the integral representation, and the haar features. Adaboost is used to eliminate redundancy and to select features with a minimum error rate, whereas Haar features are used to extract features. The implementation of the algorithm worked well for face detection even in unfavorable environmental conditions. At the same time, the goal of the research has been to increase detection precision by altering the haar characteristics.

Another study was conducted on face detection using OpenCV and Haar cascades classifiers by Sidra Mehtab and Jaydip Sen (2020)[2]. The objective of the study was to develop a face detection framework that could be extended for the detection of eyes as well as nose. The study discusses the working of haar cascade algorithm. The algorithm is trained with both positive (containing faces) and negative (not containing faces) images. The model was trained with haar cascade frontal face xml file. Haar eyes and Haar nose xml files were later used for eyes and nose detection.

A study on face detection using haar cascade classifiers along with three additional classifiers was done by Li Cuimei et.al (2018)[3]. The three subpar classifiers were based on matching the skin hue histogram, detecting the eyes, and detecting the lips. First, a crude Haar cascade classifier processed photos of people, almost rejecting incorrect human face recognition (extremely low false negative rate) but accepting some incorrect recognition (false positive rate). Second, a weak classifier based on face skin hue histogram matching was used to remove the majority of non-human faces in order to eliminate these falsely accepted non-human appearances. After that, some remaining non-human faces were identified and discarded before another weak classifier based on eyes detection was added. The false positive rate was further reduced by applying a mouth detection technique to the remaining non-human faces. The suggested technique was successful and provided state-of-the-art performance, as demonstrated by test results on photos of individuals under various occlusions and illuminations and some degrees of orientations and rotations, using OpenCV. Additionally, it was effective due to how simple and easy it was to apply.

III. METHODOLOGY

Haar cascade, the trained classifier is organized in a cascade structure, which consists of multiple stages. Each stage contains a subset of Haar-like features and a weak classifier. The weak classifier uses a simple thresholding technique to classify image regions as either positive or negative. The cascade structure allows for efficient filtering of non-face regions, quickly discarding regions unlikely to contain the target object.

When applying the Haar cascade for object detection, the image is processed in a sliding window manner. The cascade evaluates regions of the image at multiple scales and positions. At each scale and position, the Haar-like features are computed and fed into the cascade stages. A region that passes all stages is classified as a positive detection (e.g., a face).

Haar-like features are simple rectangular filters that are applied to an image at different scales and positions. These filters capture local intensity variations in the image and can be used to distinguish between different types of objects or regions within an image. The filters are defined by their shape and size, which can be adjusted to detect specific patterns or features.[2]

A Haar-like feature is computed by subtracting the sum of the pixel intensities within the white region of the filter from the sum of pixel intensities within the black region. This computation is performed at each location and scale in the image to create a feature map. The feature map represents the responses of the Haar-like features at different locations and scales. In object detection applications, a cascade of Haar-like features is often used. A cascade is a series of classifiers where each stage consists of a set of Haar-like features and a corresponding threshold. The cascade progressively filters out non-object regions in the image based on the responses of the Haar-like features. This approach allows for efficient computation and high detection accuracy.[2]

A. Summary of the Process

1. Set up the environment: Install the required libraries, such as OpenCV, then configure your programming environment.
2. Load the Haar Cascade Classifier: Load the pre-trained Haar cascade classifier XML file specifically designed for face detection. This XML file contains information about the trained classifier stages and features.
3. Load and Preprocess the Image: Load the image or access the video stream where you want to detect faces. Preprocess the image as needed, such as resizing or converting it to grayscale. Grayscale conversion is commonly done because it simplifies the face detection process by reducing the complexity of color information.
4. Perform Face Detection: Face detection is carried out by employing the Haar cascade classifier to find faces in the preprocessed image/video. This entails applying the classifier to various scales and placements of picture subregions. To evaluate whether a region contains a face or not, the cascade classifier moves across the image and applies a sequence of feature evaluations.
5. Analyse Detected Faces: After faces are found, you can go ahead and process and examine the found regions. Bounding boxes can be created around faces that have been spotted, additional processing or analysis can be used, or facial feature extraction can be done.
6. Display or save the result. [5]

IV. TESTING AND RESULTS

The algorithm was tested when multiple people were present in the frame. Only half of the people were detected, and rest were not. A false positive was also seen.

The algorithm was effective in detecting frontal faces but not efficient enough when the angle of face was sideways. The algorithm lacked accuracy in covering bounding boxes exactly surrounding the face but there were some improvements after reducing min neighbor values.

V. DISCUSSION

The Haar cascade algorithm demonstrates computational efficiency[8], making it suitable for real-time or near real-time face detection on resource-constrained devices. It achieves reasonable accuracy[8] for frontal or near-frontal face detection and can handle variations in lighting conditions and occlusions to some extent. The algorithm utilizes Haar-like features with a clear geometric interpretation, allowing for interpretability and analysis of the learned features. However, the Haar cascade algorithm has limitations and disadvantages. It struggles with complex scenes, extreme poses, and large-scale variations. It is sensitive to changes in lighting conditions and has a high false positive rate, particularly in cluttered backgrounds. Detecting non-frontal faces can be challenging, and the algorithm's training process is complex and time-consuming. Additionally, the algorithm's effectiveness is primarily limited to face detection. It is worth noting that while higher-resolution images generally offer better detection performance, there is a trade-off between resolution and computational cost.

VI. ACKNOWLEDGEMENT

I extend my heartfelt gratitude to Dr. Vikram Palimar, Professor and Head of the Department of Forensic Medicine and Toxicology, KMC, Manipal. It is my privilege to thank our Dean, KMC, Manipal, all the faculty members, Department of Forensic Medicine and Toxicology, and Institutional Ethics Committee, Department of Statistics, KMC, Manipal, without which I would never be able to give my best. I also appreciate Mr.Suhas Kashyap, Mr.Rahul.N, and Mr.Subramanya Prajwal for backing me up throughout my thesis. I also express my gratitude to the Supreme Lord and my parents for the blessings showered on me during this period.

Conclusion

In conclusion, the Haar cascade algorithm is a widely used method for face detection in computer vision applications. It employs a cascade of classifiers trained on Haar-like features to detect faces in images or video streams. It is worth noting that the field of computer vision has seen advancements beyond the Haar cascade algorithm, such as deep learning-based approaches like convolutional neural networks (CNNs) and more recent architectures like the EfficientDet and CenterNet. These approaches have achieved state-of-the-art performance on face detection tasks and have become popular alternatives to the Haar cascade algorithm. In short, while the Haar cascade algorithm has been a popular choice for face detection in the past, it is important to consider the specific requirements of the application and explore other advanced techniques to achieve the best results in face detection tasks.

References

[1] Low Processing Fee Journal in Engineering?: Impact factor: 8.118. (n.d.). http://www.ijirset.com/upload/2019/april/11_Face [2] Mehtab, S., & Sen, J. (2020). Face detection using OpenCV and Haar Cascades classifiers. ResearchGate. https://doi.org/10.13140/RG.2.2.26708.83840 [3] Human face detection algorithm via Haar cascade classifier combined with three additional classifiers. (2017, October 1). IEEE Conference Publication | IEEE Xplore. https://ieeexplore.ieee.org/document/8265863 [4] Image taken from: https://training.atmosera.com/wp-content/uploads/2021/11/haar-1.png [5] GeeksforGeeks. (2023). Face detection using Cascade Classifier using OpenCV Python. GeeksforGeeks. https://www.geeksforgeeks.org/face-detection-using-cascade-classifier-using-opencv-python/

Copyright

Copyright © 2023 Radhika K. M, Shankar M. Bakkannavar, Arjun M.S, Samarth Bhaskar Bhat. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.