A Survey on Face Mask Detection and Social Distancing Measurement Using Machine Learning

Abstract

The purpose of the project “Implementation of Covid-19 Monitoring System using ML-based Face Mask Detectionand Social Distancing Measurement” is to create a tool that identifies the image of a human that can calculate the probability that he/she is wearing a mask or not and another motive of our project is to detect the people and check whether they are maintaining the social distancing norms. Using Machine Learning and Object Detection, this study seeks to detect face masks and social separation. The contribution is a strategy for merging more complex classifiers in a \"cascade\" that allows the image\'s background areas to be swiftly rejected while more computation is spent on promising object-like regions. While deep learning-based methods for general object detection have progressed dramatically in the previous two years, most approaches to face detection still use the CNN framework, which results in low accuracy and processing performance. We examine the use of MobileNet(pretrained model) and feature extraction in this project, which will yield outstanding results on a variety of object detection benchmarks. YOLO Object detection was used to detect persons in a frame and calculate the distance between them to check for social distancing. As a result, in a real-time situation, this technology tracks persons wearing or not wearing masks and provides social separation by triggering an alert if there is a violation in the scene or in public spaces. This can be combined with current embedded camera infrastructure to allow these analytics, which can be used in a variety of verticals as well as in offices and airport terminals/gates.

Introduction

I. INTRODUCTION

In order to break the chain of corona infection, it is critical to maintain social distance, wear a mask, and avoid mass gatherings at this time when COVID-19 is spreading rapidly. However, maintaining this is difficult. Many people gather and stroll the streets without wearing a mask, whether deliberately or unknowingly. It's not easy to keep track of all of these activities. To keep track of these rules, the authorities will need effective technology. This project can assist in the detection of face masks as well as the monitoring of social distances. We have the best possibility of imposing the mask policy on people with the help of AI and Computer Vision. Our method is designed to detect the presence of face masks on real time frames. We can measure people's social distance using Computer Vision technologies based on YOLOv3-based deep learning. Our main features include object detection, classification, object tracking and analysis and YOLOv3.

II. RELATED WORK

Most of the work done is based on using VGG-16, VGG-19, ResNet-50, Inception and few transfer learning concepts with the maximum accuracy limited to 94%. As the deep learning algorithms are gaining popularity, they have been used in most of the models for computer vision, object detection but they have limited accuracy for face mask detection. Combining the feature extraction and CNN would work to overcome the problem so we have proposed the combination of feature extraction and Mobilenet to increase the accuracy.

Object detection consists of various approaches such as fast R-CNN, Retina-Net, and Single-Shot MultiBox Detector (SSD). Although these approaches have solved the challenges of data limitation and modeling in object detection, they are not able to detect objects in a single algorithm run. The YOLO algorithm has gained popularity because of its superior performance over the aforementioned object detection techniques.

III. LITERATURE REVIEW

Safa Teboulbi, Seifeddine Messaoud, Mohamed Ali Hajjaji, and Abdellatif Mtibaa presented the following concept. focuses on developing an embedded vision system that implements a Face Mask and Social Distancing Detection paradigm. In this context, we use pre-trained models like MobileNet, ResNet Classifier, and VGG.

VGG-16, ResNet- 50, InceptionV3, VGG-19, and DenseNet-169, which have accuracy values of 82.1 percent, 89 percent, 60 percent, 53.4 percent, 94.52 percent, and 93.15 percent, respectively, are the two components of the proposed model. [1]. In comparison to other models, the VGG-19 has the highest accuracy.

Another technique proposed by Sammy V. Militante and Nanette V. Dionisio explains the theory behind CNNs by creating an artif icial model that resembles the visualization area of the human brain. [2] The most significant advantage of CNNs is that, rather than simply handmade attributes, they may extract more vital information across the entire image.

Different deep networks based on CNN were introduced by researchers, and these networks obtained state-of-the-art results in computer vision classification, segmentation object identification, and localization. It is advised that numerous CNN models be combined and that each model be evaluated with the highest performance accuracy during training to improve performance in detecting and recognizing people wearing facemasks and face shields.

Paul Viola and Michael Jones proposed a visual object detection system based on machine learning that can analyze images quickly and achieve high detection rates.

Three significant contributions distinguish this work. The first is the introduction of a new picture representation known as the "Integral Image," which enables our detector's features to be computed quickly.

The second is an AdaBoost-based learning method that selects a small number of key visual features from a larger set to produce exceptionally efficient classifiers.

The third contribution is a "cascade" method for merging progressively complicated classifiers, allowing background parts of an image to be swiftly dismissed while more computation is spent on potential object-like regions.

The cascade can be thought of as an object-specific focus-of-attention mechanism that, unlike prior approaches, ensures that dismissed regions are unlikely to include the object of interest statistically. The system achieves detection rates comparable to the best existing systems in the face detection domain.

The detector, which is used in real-time applications, works at 15 frames per second without using picture differencing or skin color detection.

One of the approaches proposed by Arjya Das, Mohammad Wasif Ansari, Rohini Basak implements face mask detection as a crucial task to help global society. This paper presents a simplified approach to achieve this purpose using some basic Machine Learning packages like TensorFlow, Keras, OpenCV and Scikit-Learn. [5] The proposed method detects the face from the image correctly and then identifies if it has a mask on it or not. As a surveillance task performer, it can also detect a face along with a mask in motion. The method attains accuracy up to 95.77% and 94.58% respectively on two different datasets. We explore optimized values of parameters using the Sequential Convolutional Neural Network model to detect the presence of masks correctly without causing over-fitting.

IV. PROPOSED SOLUTION

Our goal is to investigate how Machine Learning techniques may be used to determine whether a person is appropriately wearing a mask. As a result, this project must consider not just the presence or absence of a mask, but also its right use, significance, and the ability to tell when it is properly positioned and when it is not. To improve accuracy, feature extraction and the MobileNet algorithm were utilized.

The second objective aims to provide a more advanced measure than pure social distancing monitoring to further reduce the spread of Covid-19. This leads to our proposed approach of maintaining the social distance with high accuracy. The YOLOv3 algorithm is being used for higher accuracy.

It offers detection and warning regarding multiple possibilities of placement errors in the form of alerts and alerting the mass (community) about the consequences of not following the norms.

This work provides a comprehensive and effective solution to perform person detection, social distancing violation detection, face detection, face mask classification using object detection and spreading awareness through alerts.

V. SYSTEM ARCHITECTURE

The above Fig1. represents our proposed system architecture of face mask detection. It consists of three major stages. The first stage of our architecture includes a Face Detector, which localizes faces in images of varying size and processes the images. The detected faces (regions of interest) extracted from this stage are then batched together and passed to the second stage of our architecture, which is a “feature extraction” based Face Mask Classifier. The results from the second stage are decoded and the final output is the image with the face in the frame correctly detected and classified as either masked or unmasked faces. If the face is found unmasked, then alerts will be shown to users making them cautious to wear a mask.

Fig2. represents the third stage of our architecture which is based on the YOLOv3 algorithm. It will detect people from the image, calculate the Euclidean distance between them and the final output will be whether they are following or violating the rules of social distancing. The violation of social distancing will lead to alerts describing the social distancing norms

VI. METHODOLOGY

The above diagram represents the flow of our work. For maintaining the security of the application we register (new user) or login. The raw image is detected from the video stream and the preprocessing of the frame captured is done to convert it into the standard image. Face Detection and Face Mask Detection takes place with feature extraction and CNN algorithm. The classification is done based on whether the person is wearing a mask or not. For social distancing the image is captured using the live video stream and the image is converted into the format accepted by YOLOv3 algorithm. Object Detection takes place where the people in the given frame are identified. The Euclidean distance between them is calculated. And based on the safety conditions the result is given in the form of alerts.

1. Dataset: The dataset used comprises of nearly 2000 images containing specific images with facial annotations belonging to two classes, masked faces and unmasked faces.We manually added a 3rd class where we added images in the dataset of people covering their mouth with hand so that the algorithm could easily differentiate between a mask and a hand. The bounding box coordinates and labels were then marked on the live feed.
2. YoloV3 Architecture and Functioning: YoloV3 has Darknet 53 ,with these 53 layers, the model is more powerful to identify more than 80 different objects in one image. YoloV3 has brought down the error rate drastically. YoloV3 uses a thin sized boundary box. The previous version has been improved for an incremental improvement which is called YoloV3.

VII.  ALGORITHMS

A. Face Mask Detection
1. MobileNet and Haar-Cascade

The above image represents a face detection algorithm using Haar features. Further, the algorithm uses adaboost to eliminate the unnecessary features considering only the necessary ones. To reduce the computation, it uses integral image concept and then the cascade classifier is composed of stages each containing a strong classifier. The output of the above algorithm is to check the presence of face/faces in the given frame.

The input images were passed to the base model in the form of an array, as the algorithm required numeric data. The selected base model is the pre-trained MobileNet model and in the next step the output of the base model is given as an input to the head model. The head model is a fully connected layer. And finally, the last step of the algorithm consists of classification which has two layers: mask and without mask. The above algorithm produces a result where it detects the presence of a mask in the live feed.

B. Social Distancing Measurement

The above figure describes the implementation of Social Distancing Algorithm.

The algorithm takes the input in the form of live feed(through webcam) and then the frames are created from the videos where the presence or absence of the people in the frame is detected with the help of coco dataset’s person class. Lastly, after assuring the presence of people the distance between the is calculated using Euclidean distance formula and that is notified with the help of bounded boxes,if the distance is safe green bounded box is used and if unsafe red bounded box is used.

VIII. EXPERIMENTAL SETUP

The experiment is set up by using a tailored unbiased dataset with enough photos available online to train our algorithm. The dataset is divided into three sections: training, testing, and validation. Face detection is accomplished using the Haar Cascade technique, which is written in Python.

For social distancing, we have used the data collection which is divided into 70% and 30% training and testing sets respectively. There is no restriction on the mobility of the people throughout the whole implementation (live webcam). People in the frame move freely; their visual appearance is affected by radial distance and camera position. The algorithm is implemented using Python and social distancing is measured through the Yolov3 algorithm. If the classification is detected unsafe then the results are displayed in the form of audio alerts.

For training we set the hyper-parameters of the network as follows :

A. Face Mask Detection

1. Learning Rate : 1e-4
2. Epoch : 20
3. Batch-size : 32
4. Precision
5. Recall

B. Social Distancing Detection

1. Focal length : 4.5mm
2. Sensor Dimensions : 4.80mm X 3.60mm

IX. FUTURE SCOPE

1. In our project, we cannot detect face mask and measure social distancing at the same time as in public places there is very less possibility of facing the camera by every individual at the same time.
2. This can be achieved at product level by setting up multiple cameras having lenses with 360 degree rotation.

X. APPLICATIONS

A. Bus Station

B. Hospital

C. Shopping Mall

D. Medical

E. Airports

XI. OUTPUT SCREENSHOTS

References

[1] Safa Teboulbi , Seifeddine Messaoud , Mohamed Ali Hajjaji , and Abdellatif Mtibaa, “Real- Time Implementation of AI-Based Face Mask Detection and Social Distancing Measuring System for COVID- 19 Prevention-Raspberry Pi”, Sousse University, Sousse, Tunisia, 2021. [2] Sammy V. Militante, Nanette V. Dionisio, “Deep Learning Implementation of Facemask and Physical Distancing Detection with Alarm Systems”, University of Antique Sibalom, Antique, Philippines, 2020. [3] Paul Viola, Michael Jones, “Rapid Object Detection using a Boosted Cascade of Simple Features”, Mitsubishi Electric Research Labs 201 Broadway, 8th FL Cambridge, MA 02139, 2001. [4] Shashi Yadav, “iJRASET(International Journal)-Raspberry Pi.”, Department of Computer Science and Engineering, Goel Institute of Technology and Management, Dr. A.P.J. Abdul Kalam Technical University, Lucknow, India, 2020. [5] Arjya Das, Mohammad Wasif Ansari, Rohini Basak, “Covid-19 Face Mask Detection Using TensorFlow, Keras and OpenCV”, Jadavpur University, Kolkata, India, 2020. [6] Abdellah Oumina,Noureddine El Makhfi,Mustapha Hamdi, “Control The COVID-19 Pandemic: Face Mask Detection Using Transfer Learning”, PMIC Laboratory, Faculty of Science and Technology Al- Hoceima, Abdelmalek Essaadi University, Tetouan, Morocco, 2020. [7] Zhenfeng Shao, Gui Cheng, Jiayi Ma, Zhongyuan Wang, Jiaming Wang, Deren Li, “Real-time and Accurate UAV Pedestrian Detection for Social Distancing Monitoring in COVID-19 Pandemic”, State Key Laboratory of Information Engineering in Surveying Mapping and Remote Sensing, Wuhan University, 12390 Wuhan, Hubei, China, 430072, 2021. [8] Sudeep Tanwar , (Senior Member, Ieee), Rajesh Gupta , (Student Member, Ieee), Mohil Maheshkumar Patel , Arpit Shukla , Gulshan Sharma2 , And Innocent E. Davidson, Nirma University, Ahmedabad, Gujarat 382481, India, 2021. [9] Shabir Hussain ,Yang Yu Muhammad Ayoub Akmal Khan, Rukhshanda Rehman ,Junaid Abdul Wahid and Weiyan Hou, “IoT and Deep Learning Based Approach for Rapid Screening and Face Mask Detection for Infection Spread Control of COVID-19”, 2021. [10] Mohd Ezanee Rusli, Salman Yussof, Mohammad Ali, Ahmed Abdullah Abobakr Hassan, “MySD: A Smart Social Distancing Monitoring System”, University Tenaga Nasional Selangor, Malaysia, 2020.

Copyright

Copyright © 2022 Akansha Potharkar, Akanksha Songire, Snehal Late, Sania Mulani, Miss. Radhika Malpani. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.