Computer Vision Based System for Mask Detection

Abstract

Coronavirus disease is the latest epidemic that forced an international health emergency. It spreads mainly from person to person through airborne transmission. Community transmission has raised the number of cases over the world. Reports indicate that wearing face masks while at work clearly reduces the risk of transmission. Many countries have imposed compulsory face mask policies in public areas as a preventive action. Manual observation of the face mask in crowded places is a tedious task. Thus, researchers have motivated for the automation of face mask detection system. An efficient and economic approach of using AI to create a safe environment in a manufacturing setup. A hybrid model using deep and classical machine learning for face mask detection will be presented. The face mask recognition is developed with a machine learning algorithm through the image classification method: MobileNetV2. A face mask detection dataset consists of with mask and without mask images, we are going to use MobileNetV2 to do real-time face detection from a live stream via our webcam. We introduce face mask detection that can be used by the authorities to make mitigation, evaluation, prevention, and action planning against COVID-19. The steps for building the model are collecting the data, pre-processing, split the data, testing the model, and implement the model. The built model can detect people who are wearing a face mask and not wearing it at an accuracy of 96 percent. This will help track safety violations, promote the use of face masks, and ensure a safe working environment.

Introduction

I. CERTIFICATE

This is to certify that the Project work “COMPUTER VISION BASED SYSTEM FOR MASK DETECTION” done by “Dev Likhankar”, “Amit Swain”, “Harshavardhan Bhosale ’’, students of “Department of Computer Engineering” is a record of Bonafide work carried out of them. This Project is done as partial fulfilment of obtaining “Bachelor of Computer Engineering” degree to be awarded by “Mahatma Gandhi Mission of College of Engineering and Technology, Kamothe”. The matter in this project report has not been submitted to any other university for the award of any other degree.

Prof. Chandrashekhar Badgujar                                                      Prof. Vidya Bharde

 (Supervisor/Guide)                                                                         (Project Co-ordinator)                                                             

Dr. Ashok Kanthe                                                                                Dr. Geeta S. Latkar

 (Head of Department)                                                                          (Director of College)

II. DECLARATION

We declare that this written submission represents our ideas in our own words and where others’ ideas or words have been included, we have adequately cited and referenced the original sources. We also declare that we have adhered to all principles of academic honesty and integrity and have not misrepresented or fabricated or falsified any idea/data/fact/source in my submission. We understand that any violation of the above will be cause for disciplinary action by the Institute and can also evoke penal action from the sources which have thus not been properly cited or from whom proper permission has not been taken when needed.

Dev Likhankar                                                                                  

________________

Amit Swain

________________

Harshavardhan Bhosale

________________

ACKNOWLEDGEMENT

We would like to express our gratitude to all those who helped us to reach our goal, it would not have been possible without the kind support and help of many individuals and organizations. We would like to extend our sincere thanks to all of them. We are highly indebted to Prof. Chandrashekhar Badgujar, for the guidance and constant supervision as well as for providing necessary information regarding the system and also for the support in completing the system. We are thankful to Prof. Vidya Bharde, Project Co-Ordinator for the guidance during the system. We are thankful to Dr. Ashok Kanthe, Head, Computer Department, Mahatma Gandhi Mission’s College of Engineering and Technology, Navi Mumbai, for the guidance during the system. We are thankful to Dr. Geeta Lathkar, Director, Mahatma Gandhi Mission’s College of Engineering and Technology, Navi Mumbai, for her encouragement and for providing an outstanding academic environment. Special thanks to our colleagues and friends for providing suggestions and continuous encouragement. Finally, we would like to than our family, our parents for supporting us and endurance during this work.

III. PROJECT REPORT APPROVAL FOR B.E.

This project report entitled “Vision Based System for Mask Detection” by “Dev Likhankar”, “Amit Swain” and “Harshavardhan Bhosale” is approved for the degree of “Bachelor of Engineering”.

                                                                           Examiners

1. __________________________
2. __________________________

Date:

Place: MGMCET, Kamothe, Navi Mumbai

IV. INTRODUCTION

Coronavirus disease (COVID-19) is the latest epidemic caused by the newly discovered coronavirus. COVID-19 is an infectious disease that affects the respiratory system caused by the SARS-CoV-2 virus. It mainly spreads from person to person by airborne transmission, especially through close contact. COVID-19 had a massive impact on human lives. The pandemic led to the loss of millions and affected the lives of billions of people. Its negative impact was felt by almost all commercial establishments, education, economy, religion, transport, tourism, employment, entertainment, food security and other industries. After the person gets infected, it takes almost fourteen days for the virus to grow in the body of its host and affect them and in the meantime, it spreads to almost everyone who is in contact with that person. So, it is extremely hard to keep the track of the spread of COVID-19.  COVID-19 mainly spreads through droplets produced as a result of coughing or sneezing by an infected person. This transfers the virus to any person who is in direct close contact (within one-meter distance) with the person suffering from coronavirus. Because of this, the virus spreads rapidly among the masses. With the nationwide lockdowns being lifted, it has become even harder to track and control the virus. Face masks are an effective method to control the spread of virus. It had been found that wearing face masks is 96% effective to stop the spread of virus. The governments, all over the world, have imposed strict rules the everyone should wear masks while they go out. But still, some people may not wear masks and it is hard to check weather everyone is wearing mask or not. In such cases, computer vision will be of great help.

A. Description Of The Project

We introduce face mask detection that can be used by the NSO providing the data for the government, so the government can do some preventive action, mitigation, and evaluation of their programs. Moreover, this paper can be an early warning for the authorities in capturing the people’s habits in their region. On the other hand, this solution can be used by the industries to provide the face mask based on the people’s habit of wearing face masks; the more people get used to wearing a face mask, the more face mask needs to be supplied.

B. Project Goals & Objectives

The built model can be implemented on the surveillance cameras to impede the transmission of COVID-19 transmission by detecting the people who are not wearing a face mask. Each camera point is supplied with location data, so the data can be used to determine which locations require more attention from the authorities.

C.   Project Approach

The face mask recognition in this study is developed with a machine learning algorithm through the image classification method: MobileNetV2 also using OpenCV and Keras. MobileNetV2 is a method based on Convolutional Neural Network (CNN) that is developed by Google with improved performance and enhancement to be more efficient. OpenCV is an open-source library which is primarily used for Computer Vision Applications. Keras is an open-source software library that provides a Python interface for artificial neural networks. Keras acts as an interface for the TensorFlow library.

V. LITERATURE SURVEY

A. Existing System

A face mask detecting model named Retina Face Mask combining with a cross-class object removal algorithm is proposed by Jiang. The developed model includes one stage detector consisting feature pyramid network that results in slightly higher precision and recall than the baseline result. For reducing the shortage of datasets, they have applied transfer learning, a well-known deep learning technique. Gupta proposed a model to enforce the social distance using smart city and Intelligent Transportation System (ITS) during COVID-19 pandemic. Their model described the deploying sensors in different places of the city to monitor the real-time movement of objects and offered a data-sharing platform. A noticeable contribution of a smart city in controlling the spread of coronavirus in South Korea is explained by Won Sonn and Lee. A time-space cartographer speeded up the contact tracking in the city including patient movement, purchase history, cell phone usages, and cell phone location. Real-time monitoring has been carried out on CCTV cameras in the hallways of residential buildings.

B. Proposed System

The proposed system focuses on how to identify the person on image/video stream wearing face mask with the help of MobileNetV2 and deep learning algorithm by using the OpenCV, Tensor flow, Keras library.

C. Approach

1. Train Deep learning model (MobileNetV2)

Apply mask detector over images / live video stream

VI. PROPOSED SYSTEM

A. Problem Statement

The purpose of the project entitled as “Computer Vision based system for mask detection” to develop application which is user friendly, simple and fast, which will detect if the person is wearing a mask or not. So, we have come up with this project where we have tried to bring course material under one roof which is need for learning various programming languages. We have used OpenCV, Keras to make this project.

B. Objective

The project focuses on one primary objective, which is to detect if a person is wearing a mask or not, which will be done by scanning their faces individually through a camera and determines it.

C. Scope

The project application is used to determine if a person is wearing mask at a public place (restaurants, malls, gardens, religious places etc.). It is impossible to check it manually over each person. So, this application helps with it by scanning each person’s face individually at the entrance of the public place.

D. GANTT Chart

VII. REQUIREMENTS

A. Software Requirements

1. Python 3.10.4

2. TensorFlow

3. NumPy

4. SciPy

5. Matplotlib

6. Keras

7. OpenCV

8. Any Editor

B. Hardware Requirements

1. Windows 8 or Above
2. Processor – Intel(R) Premium(R) or Above
3. High Quality Webcam or inbuilt Camera
4. 64-bit Operating System
5. RAM – 4.00 GB or Above

VIII. DESIGN AND DETAILS

A. Description About The Project

The face mask recognition in this study is developed with a machine learning algorithm through the image classification method: MobileNetv2. MobileNetV2 is a method based on Convolutional Neural Network (CNN) that developed by Google with improved performance and enhancement to be more efficient. This study conducted its experiments on two original datasets. The first dataset was taken from the Kaggle dataset and the Real-World Masked Face dataset (RMFD); used for the training, validation, and testing phase so the model can be implemented to the dataset.

The model can be produced by following some steps which are

1. Data collecting

2. Pre-processing

3. Split the data

4. Building the model

5. Testing the model

6. Implement the model.

The second dataset is used to apply the model to the dataset from 25 cities in India. Some cities were chosen based on data availability. The dataset was taken from some sources, for instance, public place CCTV, shop, and traffic lamp camera. Considering the quota sampling, the images were chosen based on the population proportional size of the cities, while the duration of capturing the image is equal for every city.

IX. SYSTEM METHODOLOGY

A. Flow Chart

B. Building The Face Detection Model

1. Data Collecting: The development of the Face Mask Recognition model begins with collecting the data. The dataset train data on people who use masks and who do not. The model will differentiate between people wearing masks and not. For building the model, this study uses 1.916 data with mask and 1.930 data without a mask. At this step, the image is cropped until the only visible object is the face of the object. The next step is to label the data. The data which has been collected labeled into two groups; with and without a mask. After the data has been labeled, it is grouped into those two groups. The example of the data is as below

2. Pre-processing: The pre-processing phase is a phase before the training and testing of the data. There are four steps in the pre-processing which are resizing image size, converting the image to the array, pre-processing input using MobileNetV2, and the last is performing hot encoding on labels. The resizing image is a critical pre-processing step in computer vision due to the effectiveness of training models. The smaller size of the image, the better the model will run. In this study, the resizing an image is making the image into 224 x 224 pixels. The next step is to process all the images in the dataset into an array. The image is converted into the array for calling them by the loop function. After that, the image will be used to pre-process input using MobileNetV2.

3. Split the Data: After the pre-processing phase, the data is split into two batches, which are training data namely 75 percent, and the rest is testing data. Each batch is containing both of with-mask and without-mask images.

4. Building the Model: The next phase is building the model. There are six steps in building the model which are constructing the training image generator for augmentation, the base model with MobileNetV2, adding model parameters, compiling the model, training the model, and the last is saving the model for the future prediction process.

5. Testing the Model: To make sure the model can predict well, there are steps in testing the model. The first step is making predictions on the testing set.

X. SYSTEM IMPLEMENTATION

The model implemented in the video. The video is read from frame to frame, then the face detection algorithm works. If a face is detected, it proceeds to the next process. From detected frames containing faces, reprocessing will be carried out including resizing the image size, converting to the array, preprocessing input using MobileNetV2. The next step is predicting input data from the saved model. Predict the input image that has been processed using a previously built model. Besides, the video frame will also be labeled that the person is wearing a mask or not along with the predictive percentage.

XI. REUSLTS

A. Result Analysis

By preserving a reasonable proportion of different classes, the dataset is partitioned into training and testing set. The dataset comprises of 1539 samples in total where 80% is used in training phase and 20% is used in testing phase. The training and testing dataset contains 1231 and 308 images respectively. The developed architecture is trained for 100 epochs since further training results cause overfitting on the training data. Overfitting occurs when a model learns the unwanted patterns of the training samples. Hence, training accuracy increases but test accuracy decreases. The trained model showed 98.7% accuracy and AUC of 0.985 on the unseen test data.

B. Limitations

The developed system faces difficulties in classifying faces covered by hands since it almost looks like the person wearing a mask. While any person without a face mask is traveling on any vehicle, the system cannot locate that person correctly. For a very densely populated area, distinguishing the face of each person is very difficult. For this type of scenario, identifying people without face mask would be very difficult for our proposed system. In order to get the best, result out of this system, the city must have a large number of CCTV cameras to monitor the whole city as well as dedicated manpower to enforce proper laws on the violators. Since the information about the violator is sent via SMS, the system fails when there is a problem in the network. It also Faces issues when the camera is not pointed in the right direction of the face i.e., the alignment of the face is not proper to the view of camera. 

The proposed system mainly detects the face mask and informs the corresponding authority with the location of a person not wearing a mask. Based on this, the authority has to send their personnel to find out the person and take necessary actions. But this manual scenario can be automated by using drones and robot technology to take action instantly.

Furthermore, people near to the person not wearing a mask may be alerted by an alarm signal on that location, and displaying the violators face in a LED screen to maintain a safe distance from the person would be a further study.

XII. FUTURE SCOPE

The current ongoing system is gracing with MobileNetV2 classifier one of the best systems which would be implemented along with the interface of alarm and alerting system in future generation. This system will be integrated with the system implementing social distancing that would make it a complete system which can bring a dramatic impact on the spread of. The new world will be well-being of high demand of mask as faceless future and that will be a big security concern. Expertise says, CNN that using face mask proves to be the best solution to mitigate the spread of air borne virus like corona, but as a big security concern headed to challenge the nation as it would create a massive opportunity for people who cover their faces for nefarious reason. And also, experts say the mass no of mask wearing in could complicate in crime investigation in the coming days, as facial recognition is an important part in tracking of the criminals. When the pandemic covid getting over, then this system comes into play for chemical factories, banks, glass factories etc. If a person enters the bank while wearing a mask he would be not allowed to enter and also if the person does not wear masks in glass factories chemical factories and etc. then the person would not be allowed to enter to the industry. A mind concept of human being has been proved out to be very good at recognizing familiar faces and facial recognition algorithms are getting better in identifying pattern. So thus, this challenge would create a scope to new face mask detection algorithms which can identify aces which are covered with greater accuracies and precisions.

Conclusion

As the technology are blooming with emerging trends the availability so we have novel face mask detector which can possibly contribute to public healthcare. The architecture consists of Mobile Net v2 as the backbone it can be used for high and low computation scenarios. In order to extract more robust features, we utilize transfer learning to adopt weights from a similar task face detection, which is trained on a very large dataset. We used Python, OpenCV, tensor flow, keras and CNN to detect whether people were wearing face masks or not. The models were tested with images and real-time video streams. The accuracy of the model is achieved and, the optimization of the model is a continuous process and we are building a highly accurate solution by tuning the hyper parameters. This specific model could be used as a use case for edge analytics. Furthermore, the proposed method achieves state-of-the-art results on a public face mask dataset. By the development of face masks detection, we can detect if the person is wearing a face mask and allow their entry would be of great help to the society. In the name of the statistical organization that needs to move quickly to adopt and take advantage of machine learning and new digital data resources, this study can be an easy move for authorities to use more unstructured data resources for more data-based mitigation, evaluation, prevention, and action planning against COVID-19.

References

[1] Face Detection and Recognition using OpenCV and Python by Tejashree Dhawle1, Urvashi Ukey2, Rakshandha Choudante3 (oct 2020) [2] Face Detection and Recognition Using OpenCV by Maliha Khan, Sudeshna Chakraborty, Rani Astya, Shaveta Khepra (2019) [3] Face Detection and Tracking Using OpenCV by Kruti Goyal, Kartikey Agarwal, Rishi Kumar (2017) [4] B. QIN and D. Li, identifying facemask-wearing condition using image super-resolution with classification network to prevent COVID-19, May 2020. [5] M.S. Ejaz, M.R. Islam, M. Sifatullah, A. Sarker Implementation of principal component analysis on masked and non-masked face recognition 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT) (2019) [6] Jeong-Seon Park, You Hwa Oh, Sang ChulAhn, and Seong- Whan Lee, Glasses removal from facial image using recursive error compensation, IEEE Trans. Pattern Anal. Mach. Intell. 27 (5) (2005). [7] N. Ud Din, K. Javed, S. Bae, J. YiA novel GAN-based network for unmasking of masked face IEEE Access, 8 (2020)

Copyright

Copyright © 2022 Amit Swain, Dev Likhankar, Harshavardhan Bhosale, Prof. Chandrashekhar Badgujar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.