Covid Safety System

Abstract

In this paper a Raspberry Pi based automated solution system focused on the real-time face monitoring of people to detect both face masks and body temperature with the help of MLX90614 sensor has been proposed. This is implemented using Python Programming with OpenCV Library, TensorFlow, Dlib Module. A security clearance system is deployed that will allow that person to enter if they are wearing a face mask and their body temperature is in check with WHO guidelines. A programmed hand sanitizer apportioning machine is mechanized, non-contact, liquor-based hand sanitizer gadget. Liquor is essentially a dissolvable, and furthermore a generally excellent sanitizer when contrasted with fluid cleanser or strong cleanser, likewise it needn\'t bother with water to wash off since it is unpredictable furthermore, disintegrates in a split second after application to hands. It is too demonstrated that a convergence of >70% liquor can execute Covid in hands. Here, we have used IR sensor detects the hand put close to which detects the distance and the outcome is the pump starts running out the hand sanitizer. Thus, the above said system will help the society by saving time and also helps in contaminating the spread of coronavirus. This can be implemented in public places such as colleges, schools, offices, shopping malls, etc. to inspect people.

Introduction

I. INTRODUCTION

According to the World Health Organization (WHO)’s official Situation Report – 205, coronavirus disease 2019 (COVID-19) has globally infected over 20 million people causing over 0.7million deaths [1]. Individuals with COVID-19 have had a wide scope of symptoms reported – going from mellow manifestations to serious illness. Respiratory problems like shortness of breath or difficulty in breathing is one of them. Elder people having lung disease can possess serious complications from COVID-19 illness as they appear to be at higher risk [2]. Some common human coronaviruses that infect public around the world are 229E, HKU1, OC43, and NL63. Before debilitating individuals, viruses like 2019-nCoV, SARS-CoV, and MERS-CoV infect animals and evolve to human coronaviruses [3]. Persons having respiratory problems can expose anyone (who is in close contact with them) to infective beads. Surroundings of a tainted individual can cause contact transmission as droplets carrying virus may withal arrive on his adjacent surfaces [4].To curb certain respiratory viral ailments, including COVID-19, wearing a clinical mask is very necessary. The public should be aware of whether to put on the mask for source control or aversion of COVID-19. Individuals have to put on facial veils to keep away from the hazard of contamination transmission, and a social hole of as a minimum of 2m [2] have to be maintained among human beings to save you character to character unfold of sickness, consistent with WHO. Furthermore, numerous public provider establishments require customers to apply their offerings simplest if they put on veils and cling to secure social segregation. As a result, face veil identity and secure social separation checking have ended up an essential PC vision

project on the way to help the worldwide society. This look illustrates a technique for stopping the transmission of infection via way of means of constantly looking if people are adhering to secure social practices which include casting off their face coverings and carrying them openly. The World Health Organization (WHO) reports proposed that the two primary courses of transmission of the COVID-19 infection are respiratory beads and actual contact. Potential points of interest of the utilization of masks lie in reducing vulnerability of risk from a noxious individual during the "pre-symptomatic" period and stigmatization of discrete persons putting on masks to restraint the spread of virus. WHO stresses on prioritizing medical masks and respirators for health care assistants[4]. Therefore, face mask detection has become a crucial task in present global society. Face mask detection involves in detecting the location of the face and then determining whether it has a mask on it or not. The issue is proximately cognate to general object detection to detect the classes of objects. Face identification categorically deals with distinguishing a specific group of entities i.e. Face. It has numerous applications, such as autonomous driving, education, surveillance, and so on [5]. This paper presents a simplified approach to serve the above purpose using the basic Machine Learning (ML) packages such as TensorFlow, Keras, OpenCV and Scikit-Learn. The world is suffering more because of this Covid. There is an exacting assessment wherever to control the corona illness and spread to the country. The emergency clinic and the medical caretaker people are enduring to fix the influenced people and stop spreading the infection to the neighbouring people. The mask and the sanitizer is given wherever to shield people from spreading the infection and to execute the infection from the human hand.

The infection is spreading from the human hand also, mouth spit. The mouth spread is controlled with the covering mask and the human hand is constrained by the hand wash sanitizer. The hand contact while taking hand sanitizer from bottle can spreads from one human to another. There ought to be a programmed hand wash sanitizer allocator, to control and keep up the spread from one human to another. As there is an effect in utilizing the hand washes disinfection by foot or by pressing the sanitizer bottle used to have a spread of the infection sickness with one human to another.

II. EXISTING WORK

As widely as Face detection and recognition is used in modern day, it is seen to be very vital in normal day to day life concerning mainly in grounds of security and safety of people. Though this has been used since decades, the accuracy, and the speed at which the results were found were not the same. There are different methods proposed for facial recognition. Each have their own advantages and disadvantages. Some of these methods are: (1) Classical Face Recognition Algorithms (2) Artificial Neural Networks (3) Face Description Based Methods, etc. Facial recognition and detection play a vital role in various fields. As there can be more such disease like the COVID-19 in future, it is important to recognize and understand people even if they are wearing masks. To be par with the existing technology and methods, we focus on recognizing human faces covered with facemasks, which is a very common thing practiced during this pandemic.

III. METHODOLOGY

Firstly we started with our two-phase COVID-19 face mask detector, detailing how our computer vision/deep learning pipeline will be implemented. From there, we’d reviewed the dataset and used it to train our custom face mask detector. Then we implemented a Python script to train a face mask detector on our dataset using Keras and TensorFlow.

We’d used this Python script to train a face mask detector and review the results. Below given is the two-phase COVID-19 face mask detector block diagram.

Any analysis-based technique requires the identification, definition and design of suitable steps followed as a methodology. Fig 1 shows the sequence of steps which are required for facial recognition. Our program basically follows a simple algorithm. To study the details of a given image different sets of images from different angles, with/without body etc. are provided for reference. For our program we have provided 615 images for reference per person collected from different sources. These reference images are images without masks.

Python Programming Language is a free, object-oriented, high-level dynamic language with dynamic semantics. It has high-level built-in data structures combined with dynamic typing and dynamic binding which makes it attractive for Rapid Application Development as well as for scripting or connect existing components together. Python has a huge collection of libraries used for various purposes. For processes such as Image Learning and recognizing, we have used some of the most prominent and powerful libraries to aid in our process. face_recognition is a module used in our program, which is built using Dlib’s facial recognition with deep learning and has remarkably high accuracy. Dlib is a contemporary toolkit developed using C++ which contains numerous machine learning algorithms and tools for creating complex software for real world problems. PIL or pillow is another main module used by us, which stands for Python Image Library is immensely powerful library which is used in our program which supports various formats for images like JPG, PPM, PNG etc.

IV. WORKING OF SOFTWARE

In order to overlay face masks, we will need to perform face detection. There are many methods available to perform this task. We can use OpenCV’s built-in Haar Cascade XML files or even TensorFlow or using Keras. In our project, we are using dlib’s face detector. Before we proceed further, it is important to understand how dlib’s face detector and facial landmark detection works. The frontal face detector in dlib is based on histogram of oriented gradients (HOG) and linear SVM. We are using dlib’s frontal face detection to first detect face and subsequently detect facial landmarks using facial landmark predictor dlib.shape_predictor from dlib library.

Facial landmarks are used to localize and represent salient regions of the face, such as eyes, eyebrows, nose, jawline, mouth, etc.It is a technique that have been applied to applications like face alignment, head pose estimation, face swapping, blink detection, drowsiness detection, etc.In the context of facial landmarks, it is necessary to detect the important facial structures on the face using shape prediction methods. Facial landmarks detection involves two steps:

1. Localizing the face detected in the image.
2. Detection of key facial structures on the face

As mentioned earlier, we can perform face detection in a variety of ways but every method essentially tries to localize and label the following facial regions: Nose, Jaws, Left and right eyes, Left and right eyebrow, Mouth. We have used Deep Learning-based algorithms which are built for face localization. This algorithm will also be used in the detection of the faces in the image. We will also obtain face bounding box through some method where we use the (x, y) coordinates of the face in the image respectively. Once the face region is detected and bounded, we will proceed to the next step of detecting the key facial structures in the face region. We are using the pre-trained facial landmark detector that is included in the dlib library. It is an implementation of the One Millisecond Face Alignment with an Ensemble of Regression Trees paper by Kazemi and Sullivan (2014) where it estimates the location of 68 (x, y)- coordinates that map to facial structures on the face. We can visualize these indexes of 68 coordinates or points using the image below:

From Figure 3, the locations of the facial features can be assessed via the different sets of points [start point, end point]:

Left eye : points [42, 47]

Mouth: points [48, 67]

Left eyebrow: points [22, 26]

Nose: points [27, 34]

Right eyebrow: points [17, 21]

Right eye: points [36, 41]

Jawline: points [0, 16]

The landmark points start from 0. These annotations are part of the 68 point iBUG 300-W dataset which the dlib facial landmark predictor was trained on. The next step involves initializing dlib’s pre-trained face detector based on a modification to the standard Histogram of Oriented Gradients + Linear SVM method for object detection. This detector will handle the detection of the bounding box of faces in our image.

The first parameter to the detector is our grayscale image. (This method works with color images as well).

The second parameter is the number of image pyramid layers to apply when upscaling the image prior to applying the detector.

The advantage of increasing the resolution of the input image prior to face detection is that it may allow us to detect more faces in the image. However, the disadvantage is that the larger the input image, the more computationally expensive and the slower the speed the detection process will be. We will also print out the coordinates of the bounding boxes as well as the number of faces detected. We can also draw the bounding boxes using cv2 around the detected faces using a for loop.

The following step involves identifying the points required to drawing the different type of facemasks. The types of facemasks that we are replicating are defined by different sets of points mentioned in the NIST study paper Appendix

A. See figure 4 for the visuals. We will define the shape of the facemasks by connecting the landmark points as defined in the paper Appendix A. For example, to form wide and medium coverage mask, we will connect (draw) landmark points for jawline [0,16] with landmark coordinates of point 29.

The facemasks outline can be drawn using Drawing Functions in OpenCV for the ellipse and three other types of regular shaped masks. We can then use cv2.fillpoly function to fill the drawn facemasks with color. The color and the type of the facemasks will be pre-determined by the user with selection prior to the start of image detection. We have pre-selected two colors for facemasks —blue and black in the user input function.

V. WORKING OF HARDWARE

Firstly, the Tensor flow based Deep Learning Neural Network tries to detect if the person is wearing a Mask or not. The System has been made Robust by training it with many different examples to prevent False Positives. Once, the System has detected the Mask it asks the user to remove the mask so that it can perform Facial Land marking. The System is using DLIB Module for Facial Land marking to find the best Spot on the Forehead of the person to take Temperature from. Then by using PID Control System with Servo Motors, the system tries to align the Selected Spot on the Forehead with the Sensor. Once aligned the system takes Temperature Reading using Contactless IR Temperature Sensor.

If the temperature is within normal Human Body Temperature range it allows the Person to Proceed and sendsan email to the Authority with a Picture and other Details like Body Temperature, etc. If temperature of person is not normal then the person will be notified along with the nearby authorities. To use the email Alert Service we added a Gmail Account. All the connections are shown in the diagram. The Camera has to be connected to the Camera Connector and Raspberry Pi Display can be connected to the Display Connector. If you are using a Generic 3.5 inch Touch Screen, you can connect the screen to the required GPIOs. The GPIOs will still be free, but you will need to solder to get 5v and SDA, SCL pins for the Sensor. Connecting IR sensor such that it will face to the area where hands are detected for sanitizer.

5V relay is used here as an electrically controlled electromechanical switch. The Relay has six connection points in total, three on each side. The appliance to be controlled that is in this case a DC pump is to be connected to the left side of the relay as shown in the image above. The appliance must be connected to the common, while the positive VCC cable must be connected to Normally Open (N/O) or Normally closed (N/C) depending on whether you want your appliance to be always OFF or always ON. For our case, we require the DC pump to be always OFF, so we will connect the VCC wire to N/O. The connections on the right side of the relay, is for the

relay module itself. 5V and GND must be connected to power on the relay and signal pin is the pin that will control the output of the relay. When a logic high (5V) is given to the signal pin, a connection establishes between N/O and Common pin but if the there’s’ no signal at signal pin, i.e. logic low, the connection gets disconnected between the N/O and the common terminal, thus giving us the ability to electrically control the relay as an ON/OFF switch. The IR proximity sensor will be used to detect a hand near the nozzle and will be controlling the position of the relay in the sanitizer dispenser kit. The IR proximity sensor uses the IR emitter to emit out signal and listens on in the receiver side. The module is tuned to receive and detect the emitted signal. If an object blocks the emitted signal, the signal bounced off the object and falls on the receiver side. This is how the module detects if an object is placed in front of the sensor. The built in potentiometer can be used to adjust the sensitivity of the module and can be adjusted with a star screwdriver based on the user needs. You can tune the sensor using the potentiometer to vary it's sensitivity. You can turn it all the way towards the opposite end and ensures it turns off. You can then keep your hand below it. VCC and GND pins on the left side of the module is used to power up the module itself while the OUT pin gives the output of the sensor. When an object is detected, it gives, out HIGH signal, otherwise low. This will be connected to the relay signal pin, so that the relay can be switched on/off depending on whether an object is detected at the sensor side or not. The set-up of the DIY sanitizer dispenser kit with the components described above is a fairly straightforward. The IR proximity sensor will be the sensor used to detect if a hand is detected under the nozzle. When the IR detects an object, it will output a high (5V signal) to the relay, opening up the relay channel and turning on the DC pump, thereby releasing the sanitizer through the nozzle. After the above connections are made, the pump with plastic tube attached can be placed inside the plastic jar while the electronic components can be taped on top of the lid of the Jar. Two holes are made on top of the jar such that plastic pipe and the electrical wires can be brought out of the jar. Make sure all exposed wires are covered with electrical tape so as to avoid short circuits. A nozzle can also be added to the end of the plastic tube connected to the pump, to restrict the flow of the sanitizer. After all the connections and set up is ready, the main power was given and the relay and the IR having their power LEDs glow on. When powered on, as soon as the IR detects an object, the pump turns on and the sanitizer is poured out.

VI. RESULT

Some pictures were not recognized due to several reasons such as lighting conditions, pixelated images etc. While testing we received 70% to 85% of accuracy (i.e., the program was able to recognize the person correctly), while in some situations the program was not able to recognize the person.

VII. DISCUSSIONS

In this research, we were able to detect faces using DLIB module and face_detection library available for Python. This specific module and library have been chosen because we face less difficulties compared to other methods since these inoperative methods were not able to detect a face because they needed the whole face, if a face covered with mask was imported, the model had hard time identifying the face, i.e., it needs all the facial features to recognize faces. Another possible problem to be faced is the resolution of the imported image itself, for example, if the image were to be blurred or has pixels too huge causing staircase effect or other seeable distortions, inaccurate values would be stored causing more complications. The image needs to readable by the model or else the model will not be able to get accurate data which further does not result in accurate outcome. Other challenge we had to come across was the unavailability of sufficient dataset with the required details. Keeping dilemmas aside, this model or system gives accurate result even when applied with small constraints such as different angle, low resolution image, it is also able to recognize people with masks when the model has learnt the same person’s face structure without their face covered.

VIII. FUTURE SCOPE

As seen since the past year, the usage of mask has been made compulsory due to the COVID-19 pandemic for the ensuring health safety. Therefore, recognizing people wearing mask, while covering most of their facial features has never been of more prominent importance. In future face recognition while wearing masks can be implemented in smartphones and other devices where they use facial recognition for security reasons (Current versions of face recognition system in smartphones does not recognize a person’s face if he/she is wearing a mask). This practice may or may not be continued but in some other circumstances it seems to be very vital in recognizing people in such aspect. Currently our program works on a dataset where the person has 100% worn mask or not worn the mask at all. So, we would like to work on and understand how to identify people’s mask in such scenarios where the face is not masked properly. We would also like to improve the functionality of our project by implementing emotion recognition while wearing masks. The speed of recognizing a person is an important factor while performing facial recognition. Currently our program takes around 20 seconds to 30 seconds to recognize a person. So, we would like to implement features and work on our code for the same.

Conclusion

In this research, we are able to detect and recognize faces of people with mask with the help of Python Image Library. Firstly we feed the model with the reference images, which are of people without mask and images taken at different angles. Using these images a consolidated data array is made with 128 specific points of the face, capturing all the unique details. Now when we feed the model to recognize the image of a person wearing the mask, the model compares the data array and makes an accurate guess. Considering our challenges faced and future developments in creation of new modules, the new techniques would definitely ease the work needed in analyzing and understanding an image. But this method restricts or limits our capability to use large number of sample pictures to study the features. This further could be modified to fix the limitations to load/ feed numerous reference images to the model, for its facial recognition training.

References

[1] Qiao, W., Chen, B., Zhao, J. and Guo, S., 2021, March. Face Recognition Based on CD-RBM and BM-ILM. In Journal of Physics: Conference Series (Vol. 1802, No. 3, p. 032077). IOP Publishing. [2] Li, X., Luo, J., Duan, C., Zhi, Y. and Yin, P., 2021, March. RealTime Detection of Fatigue Driving Based on Face Recognition. In Journal of Physics: Conference Series (Vol. 1802, No. 2, p. 022044). IOP Publishing. [3] Li, Y., Guo, K., Lu, Y. and Liu, L., 2021. Cropping and attention based approach for masked face recognition. Applied Intelligence, pp.1-14. [4] Damer, N., Boutros, F., Süßmilch, M., Fang, M., Kirchbuchner, F. and Kuijper, A., 2021. Masked Face Recognition: Human vs. Machine. arXiv preprint arXiv:2103.01924. [5] Loey, M., Manogaran, G., Taha, M.H.N. and Khalifa, N.E.M., 2021. A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic. Measurement, 167, p.108288. [6] Maurya, P., Nayak, S. and Vijayvargiya, S., 2021. COVID-19 Face Mask Detection. [7] Adusumalli, H., Kalyani, D., Sri, R.K., Pratapteja, M. and Rao, P.P., 2021, February. Face Mask Detection Using OpenCV. In 2021 Third International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV) (pp. 1304-1309). IEEE. [8] Liu, G. and Zhang, Q., 2021. Mask Wearing Detection Algorithm Based on Improved Tiny YOLOv3. International Journal of Pattern Recognition and Artificial Intelligence, p.2155007. [9] Vikas K. Solegaonkar, 2019. https://solegaonkar.github.io/FaceRecognition.html [10] R. Ramachandran, S. Velan S and D. Imtiyaz Wadekar, \"Statistical Comparison of COVID-19 Infections Based Upon the Food Habits/Diets in Countries Using RStudio,\" 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence), 2021, pp. 1095-1101. [11] Suganantham, S., Babu, C. and Raju, M., “A Quantitative Evaluation of Change Impact Reachability and Complexity Across Versions of Aspect Oriented Software”, International Arab Journal of Information Technology (IAJIT), 14(1), 2017. [12] D. Meena and R. Sharan, \"An approach to face detection and recognition\", 2016 International Conference on Recent Advances and Innovations in Engineering (ICRAIE), pp. 1-6, 2016. https://ieeexplore.ieee.org/abstract/document/7939462. [13] S. Ge, J. Li, Q. Ye and Z. Luo, \"Detecting Masked Faces in the Wild with LLE-CNNs\", 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 426-434, 2017. https://ieeexplore.ieee.org/document/8099536. [14] Mrs. Supriya Kurlekar, Mr. Aniket A. Omanna., Mr. Onkar A. Deshpande, Mr. Dinesh B. Patil, “ Face Mask Detection System Using Deep Learning ”, Turkish Journal of Computer and Mathematics Education Vol.12 No. 7 (2021), 1327 – 1332. https://turcout.com/volume12/1327-1332. [15] A.G. H. Dong Chen, F. Wen and J. Sun, \"Supervised transformer network for efficient face detection\", ECCV, pp. 122-138, 2016. https://link.springer.com/chapter/10.1007/978-3-319-46454-1_8. [16] Dr Y. B. Gurav, Tejas Giri, Nupur Karve, Ifrah Ansari, Atish Oswal,” Touchless Sanitizer Dispenser”, International Journal of Scientific Research & Engineering Trends Volume 7, Issue 3, May-June-2021. https://ijsret.com/2021/05/IJSRET_V7_issue3_473. [17] Jessica Hillburn, Akshay Sharma A S, “Review on Automatic Sanitizer Dispensing Machine”, International Journal of Engineering Research Technology (IJERT), 2020. https://www.ijert.org/review-on- automatic-sanitizer- dispensing-machine. [18] Satoru Mitsuboshi, Masami Tsugita, “A Novel Automatic Sanitizer Dispenser”, International Journal of Engineering Research Technology (IJERT). https://www.ijert.org/S1341-321X(18)30271. [19] Golin, A. P., Choi, D., Ghahary, “The role of imaging in the detection and management of COVID-19: a review”, IEEE 2020. https://ieeexplore.ieee.org/document/8098536. [20] M. M. Srihari, “Self-Activating Sanitizer With Battery Imposed System For Cleansing Hands”, IEEE 2020. https://ieeexplore.ieee.org/document/9183347.

Copyright

Copyright © 2022 Hinal Sodagra, Dhruv Umraliya, Ruchi Shah, Shripad Nagpurkar, Prof. Prajakta Pawar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.