Improvised Surveillance System: Mask Detection, Social Distance Monitoring and Temperature Sensing

Abstract

The global pandemic of Covid-19 caused by novel corona virus has severely impacted the world. About 577 million cases have been reported worldwide [7]. Wearing face masks, proper sanitization and social distancing are the safety protocols to be followed strictly during this period. In this system, we propose a system where mask detection, hand sanitization, proper social distancing and temperature of the individual will be monitored. The model will be implemented by using Socket programming, Raspberry Pi Pico/ Arduino uno, CCTV cameras. Thus, the proposed system helps the society in lowering the spread of Corona Virus.

Introduction

I. INTRODUCTION

Coronavirus disease (COVID-19) is an infectious disease caused by a newly discovered coronavirus where most people infected with this virus experience a low grade to tolerable respiratory illness and recover without being in need of any special or exceptional treatment. People whose age is 60 and above with underlying medical problems such as cancer, cardiovascular disease, chronic respiratory disease, and diabetes are more likely to develop serious illness. Knowing the COVID-19 virus, what it causes and how it spreads is an effective way to prevent and slow transmission. To protect yourself and others against infection can be achieved by washing your hands or making frequent use of an alcohol-based disinfectant and avoiding touching your face. The surfaces, where you touch or come in contact with, have to be cleaned, disinfected regularly. Social distancing (physical distancing) includes infection control actions intended to slow down the spread of the disease by minimising the close contact between the individuals. People who are vaccinated may still get COVID-19 but are less likely to experience severe symptoms than the people who are unvaccinated. A person who has recovered from COVID-19 should maintain safe distance in crowd. Those who are diagnosed with COVID?19 or who believe they may be infected are advised by the CDC(Centre for Disease Control and Prevention) to stay at home except to get medical care, call ahead before visiting a healthcare provider, wear a face mask before entering the healthcare provider's office and when in any room or vehicle or closed space with another person, one should cover coughs and sneezes with a tissue, regularly wash hands with soap and water or sanitizer and avoid sharing personal household items. The WHO and the US CDC have recommended individuals to wear non-medical face coverings in public settings where there is an increased risk of transmission of the virus and where social distancing measures are difficult to maintain. This leads to reduce in the spread of the disease by asymptomatic and pre-symptomatic individuals.

As this virus spreads primarily  through droplets of saliva or discharge from the nose when an infected person sneezes or coughs. Therefore, it is important that one practices respiratory etiquette (for example, by coughing into a flexed elbow). Therefore, face mask detection along with proper social distancing has become a crucial task to help global society. In this model, we propose a system that restrict the growth of COVID-19 by finding out people who are not wearing any facial mask, maintaining proper social distance along with temperature sensing of the individuals. It can be implemented effectively in current situation when lockdown is ease to inspect persons in public gatherings, shopping malls, etc. This system reduces manpower to inspect the public and can be used in any place.

Following the onset of the global COVID-19 pandemic, there is a significant need for protective mechanisms. Face mask detection and social distancing has been proven as an effective measure against the spread of the infectious Coronavirus Disease 2019 (COVID-19). The basic aim of the project is to detect the presence of a face mask on human faces, monitoring social distance and checking the body temperature of the individual. Individuals, however, are not used to tracking the required 6-feet (2-meters) distance between themselves and their surroundings.

An active surveillance system capable of detecting distance between individuals and warning them can slow down the spread of the deadly disease. Thus, a system that will have features such as mask detection, temperature sensing and social distancing which will be implemented by using Raspberry Pi Pico/ Arduino uno, Surveillance cameras, Motion and IR temperature sensor to ensure public safety during this pandemic.

The ultimate goal of our system is to prevent the spread of this virus. Our System will ensure that people are wearing masks and maintaining social distance.

The remainder of the paper is arranged accordingly. The most recent works for facial mask detection is described in Section II. In Section III, the proposed system design for developing the whole system is described. Section IV analyses the results obtained from the developed system. The conclusion is drawn in Section V. Lastly, the limitations with potential works are depicted in Section VI.

It is hoped that this study would be a useful tool to reduce the spread of this communicable disease for many countries in the world.

II. RELATED WORK

In the meantime, many systems have been developed for mask detection and social distance monitoring during this COVID-19 pandemic. An automated system to limit COVID-19 using facial mask detection in smart city network is proposed by M. Rahman. Now a days, in every city, all public places are monitored by CCTV cameras. These cameras capture images from public places, then these images are feed into a system that identifies if a person without mask appears within the image. If a person without a mask is detected then this information is shipped to the right authority to require necessary actions. The architecture of the learning technique highly depends on CNN [1][9][10].

Arjya Das has researched and developed a system where face mask detection during COVID-19 is done using TensorFlow, Keras, and OpenCV. TensorFlow an interface for expressing machine learning algorithms is utilized for implementing ML systems into fabrication over a bunch of areas of computer science, including sentimental analysis, voice recognition, to pursue research. Keras gives basic reflections and construction units for the creation and transport of ML arrangements with high iteration speed. OpenCV (Open-Source Computer Vision Library) is utilized to differentiate and recognize objects, faces, group movements in recordings, trace progressive modules, follow eye gesture, track camera actions, find comparative pictures from an image database, expel red eyes from pictures taken using flash, perceive landscape and set up markers to overlay it with increased reality and so forth. [2].

Person Detection System for Social Distancing and Safety Violation Alert supported Segmented ROI developed by A. Ahamad. In this study, the proposed idea is developed based on Python 3, OpenCV and Caffe framework. The OpenCV library is used to use the image processing methods that will be described later in this section. The primary purpose of this system is to process captured video footage for person detection and subsequent processing for social distancing or safety violation. So, the process starts with reading the frames of a video feed one by one.

III. System Design AND ARCHITECTURE

An active surveillance system capable of detecting the presence of a face mask on human faces, monitoring social distance between individuals and warning them can slow down the spread of the deadly disease. Measuring social distance in a ROI (Region of Interest) and modulating inflow can decrease social distancing violation occurrence chance. Thus, a system that will have features such as mask detection, temperature sensing and social distancing which will be implemented by using Surveillance cameras, Raspberry Pi Pico/ Arduino uno, Motion and IR temperature sensor to ensure public safety during this pandemic.

The overall system design will consist of following modules:

A. Outdoor Module

This is the module which will be fixed outside any premises. Instead of the normal foot pedal style sanitizer dispenser, this outside module will be placed. This module will be based of Software as well as Hardware. It will have a Raspberry Pi Pico/ Arduino uno, IR temperature sensor, motion sensor, camera and a sanitizer dispenser. This Module will be connected into the output Display which will show the result/status/readings and the camera view.

Firstly, the motion and temperature are sensed using the Motion and IR Temperature sensors. The cameras will be used for capturing images of people in the premise. And all this will be deployed into Raspberry Pi Pico/ Arduino uno. If the Output Display shows that the mask is properly worn and temperature is less than the specified value then only the Sanitizer Dispenser will operate using the Relay circuit which will be connected to the Raspberry Pi Pico/ Arduino uno and now the individual can enter the premise and sanitize their hands. Raspberry Pi could have handled the whole system on itself but there could be issues; as we are using CNN model for mask detection and there is no GPU in Raspberry Pi as well as the CPU of Raspberry Pi is not strong enough, the results may take more time than expected. There would be difficulties for collecting all of the data in one place. Also, there may be multiple nodes (entry/exit points) in a premise, using Raspberry Pi at every node is not cost efficient. To overcome these problems, we came to a solution that there will be a client-server architecture where every node will be a client connected to the server where the model will be deployed, storing all the data from every node in one place and also making front-end deployment easier to implement i.e., by using Socket Programming.

To establish communication between nodes on a network, socket programming is way to connect two or more nodes. One socket/node listens on a particular port at an IP, while other socket reaches out to the listening socket to form a connection. The server forms the listener socket while client reaches out to the server [8].

In our system, Server section will consist of Mask detection, Database and Front-end. Mask detection and Database will work together and run in parallel with the Front-end. Client section will consist of Raspberry Pi Pico/ Arduino uno, Motion sensor, Camera, IR temperature Sensor, Output Display, Relay circuit, Sanitizer dispenser.

A Mask detection model will check the frame sent by the Client. The model will use Convolutional Neural Network (CNN) and gives the Result back to the Client. If an individual is not wearing the mask/temperature of individual is more than the specified value then, violation is detected and the current frame & temperature will be sent back to the Server. The Face Mask Detection model will be developed using Keras, TensorFlow, MobileNet and OpenCV. The algorithm used will be CNN i.e., Convolutional Neural Networks which is useful for pattern recognition from images, and since the MobileNetV2 architecture is used. The network comprises of an input layer, several hidden layers and an output layer. The hidden layers are composed of several convolution layers that learn suitable filters for extracting important features from the given samples. The features extracted by CNN are used by the multiple dense neural networks for classification purposes.

The Database Software used in this system will be SQLite. SQLite is a relational database engine which is open-source, self-contained, zero-configuration, etc. designed to be embedded into an application. It is used to manage structured data in applications including desktop, web, and mobile apps. The Database will store only the violations (if occurred) at particular node. The data stored will be the address of the client/node, time & date, temperature (in Fahrenheit) of the person who violated and the location of that person’s image which will be stored in the server.

Front-end deployment is done using HTML, CSS and deployed using Flask. Flask is a lightweight WSGI web application framework and has the ability to scale up to complex applications. It began as a simple wrapper around Werkzeug & Jinja and has become one of the most popular Python web application frameworks. When the Front-end deploys, then we can access the data on any device provided they are all in the same network.

B. CCTV Module

This is a completely Software based module, where the existing CCTV cameras of a premise will be integrated to the system.

Social Distancing Model:

The existing system of the social distancing monitoring algorithm consisted of following basic steps:

1. Get Region of interest from the user.
2. Get input to get average distance of 6 feet as camera’s distance matters.
3. Convert the view from 3D perspective to 2D perspective i.e., to bird’s eye view.
4. Detect humans and apply tracking.
5. Calculate the center points of every detected humans.
6. Compare the distance between every two center points with the average distance got from step 2
7. Print the bounding box Red or Green according the result of step 6 [3][4][5][8].

The problem faced in this algorithm is that to get ROI from the user which will be different for every other camera placement. The average distance will be different it the camera is of a rotating/moving type.

The average distance can be calculated with the width of the detected human

To overcome these issues, we’ve come up with an updated algorithm.

a. Detect the humans.

b. Get the co-ordinates of every detected human and store them.

c. For every human, calculate the center point.

d. Calculate the distance between two center points and compare it with the mean of both the widths of the considered humans.

e. Print the bounding box Red or Green according the result of step 4.

Eventually this algorithm reduced the number of steps involved in the previous system and also decreasing the time taken as step 1,2,3 is not necessary in this situation.

The main advantage of this system will be that is can be applied to any camera, with any type of FOV and with all different angle placement without any recalibration of the system.

The detection of a person or a human will be done by YOLO i.e. You Only Look Once object detection model and COCO dataset. This is one of the models which is very efficient resulting good number of frames per second (FPS).

IV. RESULT ANALYSIS

A. Outdoor Module

For the mask detection model, the dataset was partitioned into training and testing set. The dataset comprises of 3535 samples in total where 80% is used in training phase and 20% is used in testing phase. The trained model gave 99% accuracy.

This model was deployed in the server with the front end. The client, when it senses a motion, takes a picture of the person and sends it to the server for analysis. The result is then sent back to the client and depending upon the result, the decision is made. The client also records the body temperature of that person. The whole communication took about 1.2 to 1.5 seconds back and forth.

Based on the conditions the data including image and temperature was also stored in the database and was accessible through front end.

This architecture was tested with three clients simultaneously and was able to perform the tasks without any difficulties.

B. CCTV Module

In this updated algorithm, there is no need to set the region of interest and it works as expected. It was able to detect 80% of the pedestrians in a frame and calculate the accurate distance between every person.

This algorithm was also tested with a horizontally rotating camera and the result was as expected.

It was also tested in different kind of systems, even with low configurations, and the minimum frames per second recorded was 20 - 22 FPS, which is a good number of frames for maintaining the continuity between frames in a footage, and as high as 60 FPS, capped, in a system with high-end specifications.  

Conclusion

We have proposed an approach for Mask Detection and Social Distance Monitoring to help maintain a secure environment and ensure individuals protection by monitoring public places to avoid the spread of the COVID-19 virus. Thus, this system will operate in an efficient manner in the current situation. The solution has the potential to significantly reduce violations by real-time interventions, so this system which ensure public safety through saving time. This solution can be used in places like temples, banks, shopping complex, airports, etc.

References

[1] M. Rahman, S. Mahmud, Md. M. Manik, Jong-Hoon Kim, Md. M. Islam, “An Automated System to Limit COVID-19 Using Facial Mask Detection in Smart City Network,” 2020. [2] A. Das, M. Ansari, R. Basak, “Covid-19 Face Mask Detection Using TensorFlow, Keras and OpenCV,” 2020. [3] A. Ahamad, N. Zaini, M. Latip, “Person Detection for Social Distancing and Safety Violation Alert based on Segmented ROI,” in 2020 10th IEEE International Conference on Control System, Computing and Engineering (ICCSCE2020), 21–22 August 2020, Penang, Malaysia. [4] Y. Hou, M. Baharuddin, S. Yussof, S. Dzulkifly, “Social Distancing Detection with Deep Learning Model,” in 2020 8th International Conference on Information Technology and Multimedia (ICIMU). [5] Shashi Yadav “Deep Learning based Safe Social Distancing and Face Mask Detection in Public Areas for COVID-19 Safety Guidelines Adherence”. [6] Face Mask Detection | Kaggle. [Online]. Available: https://www.kaggle.com/andrewmvd/face-mask-detection [Accessed 10 Jul 2022]. [7] Worldometer , “Coronavirus Cases,”. [Online]. Available: https://www.worldometers.info/coronavirus [Accessed 18 Jul 2022]. [8] K. Bhambani, T. Jain and K. A. Sultanpure, “Real-time Face Mask and Social Distancing Violation Detection System using YOLO” in IEEE Bangalore Humanitarian Technology Conference (B-HTC), 2020, pp. 1-6, doi: 10.1109/B-HTC50970.2020.9297902. [9] S. Srinivasan, R. Rujula Singh, R. R. Biradar and S. Revathi, “COVID-19 Monitoring System using Social Distancing and Face Mask Detection on Surveillance video datasets” in 2021 International Conference on Emerging Smart Computing and Informatics (ESCI), pp. 449-455, 2021, doi: 10.1109/ESCI50559.2021.9396783. [10] ZHANG J, HAN F, CHUN Y. and CHEN W, “Novel Detection Framework About Conditions of Wearing Face Mask for Helping Control the Spread of COVID-19” 2021, doi:10.1109/ACCESS.2021.3066538.

Copyright

Copyright © 2022 Srinivasan Poonkundran, Uzma Patil, Pooja Rahate, Nikhil Thorat, Prof. A. A Gupta. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.