Facial Recognition Attendance System Using Deep Learning

Abstract

Face authentication is one of the critical unsolved problems in computer science, a lot of time and energy is spent to invent a robust solution for it. Face authentication can play a significant role in security, Biometric verification, auto detection of criminals in a crowd, etc. The paper presents a unique model that authenticates the faces in the image, and securely shares the images. Firstly, the deep learning algorithm identifies all the faces in the image, extracts the invariant features of each face. The invariant feature of the face is studied to compute the emotions. The model captures the friends list of the user and securely shares the image to the known persons in the image.The main purpose of this project is to enhance the relationship between the student and the teaching staff. Due to pandemic every student are forced to attend the online classes, even though some students are tend to bunk the classes by attending those session by use some other else. In order to stop this we developed an windows application to take facial attendance and attending the class. The current old system has a lot of ambiguity that caused inaccurate and inefficient of attendance taking. Many problems arise when the authority is unable to enforce the regulation that exists in the old system. Thus, by means of technology, this project will resolve the flaws existed in the current system while bringing attendance taking to a whole new level by automating most of the tasks. The technology working behind will be the face recognition system. In the old system they use LDA. Although LDA is one of the most common data reduction techniques, it suffers from two main problems: The Small Sample Size (SSS) and linearity problems. We improvised the facial recognition using LBPH – (Local Binary Pattern Histograms) - Local binary patterns (LBP) is a type of visual descriptor used for classification in computer vision. This algorithm is so fast and reliable and the result prediction we have utilized a regression algorithm.

Introduction

I. INTRODUCTION

Today’s institutions and organizations are facing major security issues consequently, they need several specially trained personnel to attain the desired security. These personnel, as human beings, make mistakes that might affect the level of security. So, we need a system with is automated and efficient, Face recognition play a vital role in variety of applications from biometrics, surveillance, security, identification to the authentication. In this paper we design and implement a smart security system for restricted area where access is limited to people help in minimizing human error except for whose faces are available in the training database. First, we are going to detect the face by detecting the human motion. This system is composed of two parts:

hardware part and software part. The hardware part consists of a camera, while the software part consists of face-detection and face-recognition algorithms software. When a person enters to the zone in question, a series of snapshots are taken by the camera and sent to the software to be analyzed and compared with an existing database of trusted people. An alarm goes off if the user is not recognized.  Experimental results demonstrate the effectiveness of proposed security system in order to restrict the unauthorized access and enhanced reliability by use of face recognition.

II. RELATED WORK

One of the first studies on facial recognition was done by K. James Mathai [2].This paper work is focused on Performance comparison of intrusion detection system between DBN Algorithm and SPELM Algorithm. Researchers have used this new algorithm SPELM to perform experiments in the area of face recognition, pedestrian detection, and for network intrusion detection in the area of cyber security. The scholar used the proposed State Preserving Extreme Learning Machine (SPELM) algorithm as machine learning classifier and compared it's performance with Deep Belief Network (DBN) algorithm using NSL KDD dataset. The NSL- KDD dataset has four lakhs of data record; out of which 40% of data were used for training purposes and 60% data used in testing purpose while calculating the performance of both the algorithms.

The experiment as performed by the scholar compared the Accuracy, Precision, recall and Computational Time of existing DBN algorithm with proposed SPELM Algorithm. The findings have show better performance of SPELM; when compared its accuracy of 93.20% as against 52.8% of DBN algorithm;69.492 Precision of SPELM as against 66.836 DBN and 90.8 seconds of Computational time taken by SPELM as against 102 seconds DBN Algorithm.

Author Vimala Mathew [3] proposed paper work is focused on Performance comparison of intrusion detection system between DBN Algorithm and SPELM Algorithm. Researchers have used this new algorithm SPELM to perform experiments in the area of face recognition, pedestrian detection, and for network intrusion detection in the area of cyber security. The scholar used the proposed State Preserving Extreme Learning Machine (SPELM) algorithm as machine learning classifier and compared it's performance with Deep Belief Network (DBN) algorithm using NSL KDD dataset. The NSL- KDD dataset has four lakhs of data record; out of which 40% of data were used for training purposes and 60% data used in testing purpose while calculating the performance of both the algorithms. The experiment as performed by the scholar compared the Accuracy, Precision, recall and Computational Time of existing DBN algorithm with proposed SPELM Algorithm.

A. Linear Discriminate Analysis

LDA is a method to find a linear combination of features which characterize or separate two or more classes of objects or events. Linear classifier can be obtained from the resultant. Large number of pixels are used to represent face in computerized face recognition. Before classification Linear discriminant analysis is used to reduce features and makes it more manageable. New dimensions are a linear combination of pixel values which forms a template.

B Convolutional Neural Networks

Convolutional neural networks (CNNs) are multilayer neural network architectures [19]. CNN inputs and outputs are array vectors called feature maps. The size of the array depends on the input type. For example, audio inputs include one-dimensional arrays and text inputs. I have a 2D array in my image. The output object map describes the objects retrieved from the input. CNN consists of three main layers: Convolutional Filter Layer, Pool/Sub-sampling Layer, and Classification Layer.

C. Facial Expression Recognition Methods

The major drawback of the function-based approach is that it requires a lot of effort to design and use various human-generated function extraction methods. To overcome these shortcomings, we propose a new approach based on deep learning, a machine-generated function that automatically extracts facial features.

III. PROPOSED METHOD

In this paper, we propose an Facial Recognition Attendance System Using Deep learning, Here we are using OpenCV and Haar-cascades to overcome the drawbacks of LDA. By using these methods, we can detect the faces even when there are any alterations on the face like changing pose or illumination, degraded images.

The Procedure contain four modules

A. Image Data Collection.

In this module, we are going to collect the images which are needed to train and test the model we are going to use. We can collect the images of the student through the take image function which will be stored in a folder and can be used for tracking process.

B. Data Implementation.

Here the collected images are classified into different student names  (i.e.  ) and these images are used to train the models. 

The proposed DCNN is trained using 70% of normalized

ROI dataset images that are rotated to ±45 and ±75 degrees.

C. Model Training.

Now, the classified images are given as input to the models which are used to train the machine to detect whether the person is authorized or not.

The facial emotion of the input image is extracted using pre-trained DCNN.

D. Image predictions

Only the authorized person's details will be displayed and when an unauthorized person is detected by the camera it sends an e-mail to the concerned e-mail address. Now, the machine can able to detect and send the e-mail.

IV. RESULTS

In this chapter we have described the results we have obtained on the

A. Basic opencv Data Structures

1. Point2f - 2D Point: int x,y several operators and fuctions available
2. - 2D size structure: int width, height

1. - 2D rectangle object int x, y, width, height
2. - Rect object with angle

Mat - image object mainly - rows, cols - length and width, channels, depth,a large number of functions for manipulation, a critical being

Mat.type() returns the TYPE of a matrix

B. OpenCV: Types

The TYPE is a very important aspect of OpenCV

Represented as CV_<Datatype>C<no of Channels>  OpenCV: PixelTypes shows how the image is represented in data

1. - The default color of imread(). Normal 3 channel color

1. - Hue is color, Saturation is amount, Value is lightness. 3 channels
2. - Gray values, Single channel

Converting colorspaces: cvtColor( image, image, code): codes

CV_<colorspace>2<colorspace>, 

Image Normalization: process of stretching the range of an image from [a, b] to [c, d], important for visualization, normalize(imagein, imageout, low, high, method). This is incredibly important for visualization because if the image is beyond [0,255] it will cause truncation or unsightly effects.

Thresholding threshold( image, image, thresh, maxVal,

CODE), codes e.g. THRESH_BINARY

Edge Detection: several methods available: Sobel,

Scharr, Laplacian and Canny

• Sobel Edge Detection void cv::Sobel(image in, image out, CV_DEPTH, dx, dy);
• Scharr Edge Detection

void cv::Scharr(image in, image out, CV_DEPTH, dx, dy);

• Laplacian Edge Detection void cv::Laplacian( image in, image out, CV_DEPTH);

Keras

Keras is an open-source software library that provides a Python interface for artificial neural networks. Keras acts as an interface for the TensorFlow library.

Up until version 2.3 Keras supported multiple backends, including TensorFlow, Microsoft Cognitive Toolkit, Theano, and PlaidML. As of version 2.4, only TensorFlow is supported. Designed to enable fast experimentation with deep neural networks, it focuses on being user-friendly, modular, and extensible. It was developed as part of the research effort of project ONEIROS (Open-ended Neuro-Electronic Intelligent Robot Operating System), and its primary author and maintainer is François Chollet, a Google engineer. Chollet also is the author of the XCeption deep neural network model.

Keras contains numerous implementations of commonly used neural-network building blocks such as layers, objectives, activation functions, optimizers, and a host of tools to make working with image and text data easier to simplify the coding necessary for writing deep neural network code. The code is hosted on GitHub, and community support forums include the GitHub issues page, and a Slack channel.

In addition to standard neural networks, Keras has support for convolutional and recurrent neural networks. It supports other common utility layers like dropout, batch normalization, and pooling.

Keras allows users to productize deep models on smartphones (iOS and Android), on the web, or on the Java Virtual Machine.[3] It also allows use of distributed training of deep-learning models on clusters of Graphics processing units (GPU) and tensor processing units (TPU).

C. TensorFlow

TensorFlow isa free and open-source software library for machine learning. It can be used across a range of tasks but has a particular focus on training and inference of deep neural networks.

TensorFlow is a symbolic math library based on dataflow and differentiable programming. It is used for both research and production at Google.

TensorFlow was developed by the Google Brain team for internal Google use. It was released under the Apache License 2.0 in 2015.

TensorFlow is Google Brain's second-generation system. Version 1.0.0 was released on February 11, 2017. While the reference implementation runs on single devices, TensorFlow can run on multiple CPUs and GPUs (with optional CUDA and SYCL extensions for general-purpose computing on graphics processing units). TensorFlow is available on 64-bit Linux, macOS, Windows, and mobile computing platforms including Android and iOS.

Its flexible architecture allows for the easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices.

TensorFlow computations are expressed as stateful dataflow graphs. The name TensorFlow derives from the operations that such neural networks perform on multidimensional data arrays, which are referred to as tensors. During the Google I/O Conference in June 2016, Jeff Dean stated that 1,500 repositories on GitHub mentioned TensorFlow, of which only 5 were from Google.

In December 2017, developers from Google, Cisco, RedHat, CoreOS, and CaiCloud introduced Kubeflow at a conference. Kubeflow allows operation and deployment of TensorFlow on Kubernetes.

In March 2018, Google announced TensorFlow.js version 1.0 for machine learning in JavaScript.

In Jan 2019, Google announced TensorFlow 2.0. It became officially available in Sep 2019.

In May 2019, Google announced TensorFlow Graphics for deep learning in computer graphics.

D. LBPH

Local Binary Pattern Histogram algorithm is a simple approach that labels the pixels of the image thresholding the neighborhood of each pixel. In other words, LBPH summarizes the local structure in an image by comparing each pixel with its neighbors and the result is converted into a binary number. It was first defined in 1994 (LBP) and since that time it has been found to be a powerful algorithm for texture classification.

This algorithm is generally focused on extracting local features from images. The basic idea is not to look at the whole image as a high-dimension vector; it only focuses on the local features of an object.

In the above image, take a pixel as center and threshold its neighbor against. If the intensity of the center pixel is greater-equal to its neighbor, then denote it with 1 and if not then denote it with 0.

Conclusion

This whole work is to access the details about the student attendance information and generate a final report. This project “FACIAL RECOGNITION ATTENDANCE SYSTEM USING DEEP LEARNING” is a collection of static and dynamic web-based application. This project provides an offer to the user to enter the data through their respective registration forms. It is very helpful for the teachers and admin to keep and maintain the information about the students easily. In future this work can be expanded to store the internal marks, semester marks, college events and college placement activities of the students to get minimized all stuffs at one place in a systematic way to import and export the data through the admin and authorized persons whenever it will be needed in future by the educational organization. A. Future Work The project has a very vast scope in future. The project can be implemented on intranet in future. Project can be updated in near future as and when requirement for the same arises, as it is very flexible in terms of expansion. With the proposed software of database Space Manager ready and fully functional the client is now able to manage and hence run the entire work in a much better, accurate and error free manner. The following are the future scope for the project. 1) A feature which can give intruder alert can be included in the system. Furthermore, the images of unknown people can be saved in an efficient manner and displayed in the system for better security. 2) The number of training images can be reduced so that less storage is required. This can be done by removing duplicate images of the same person, or images with similar embeddings. 3) The training time can be reduced by retraining the classifier only for the newly added images. 4) A feature can be added where an employee is automatically sent a warning if his attendance or working hours are below the threshold. 5) Wrongly classified images can be added to the training dataset with the correct label so as to increase the accuracy of the recognition model.

References

[1] Akbulut, Yaman, et al. \"Deep learning based face liveness detection in videos.\" 2017 international artificial intelligence and data processing symposium (IDAP). IEEE, 2017. [2] Mathai, K. James. \"Performance comparison of intrusion detection system between deep belief network (DBN) algorithm and state preserving extreme learning machine (SPELM) algorithm.\" 2019 IEEE International Conference on Electrical, Computer and Communication Technologies (ICECCT). IEEE, 2019. [3] Mathew, Vimala, et al. \"Person re-identification through face detection from videos using Deep Learning.\" 2019 IEEE International Conference on Advanced Networks and Telecommunications Systems (ANTS). IEEE, 2019. [4] Radhika C.Damale, Prof.Bageshree.V.Pathak.“Face Recognition Based Attendance System Using Machine Learning Algorithms.\" Proceedings of the Second International Conference on Intelligent Computing and Control Systems (ICICCS 2018) IEEE Xplore Compliant Part Number: CFP18K74-ART; ISBN:978-1-5386- 2842-3. IEEE 2018 . [5] Omar Abdul, Rhman Salim, Rashidah Funke Olanrewaju, Wasiu Adebayo Balogun. “ Class Attendance Management System Using Face Recognition.\" 2018 7th International Conference on Computer and Communication Engineering (ICCCE) IEEE 2018. [6] Adrian Rhesa Septian Siswanto, Anto Satriyo Nugroho, Maulahikmah Galinium. “Implementation of Face Recognition Algorithm for Biometrics Based Time Attendance System\" Center for Information Communication Technology Agency for the Assessment Application of Technology (PTIK-BPPT) Teknologi 3 BId., 3F, PUSPIPTEK Serpong, Tangerang, INDONESIA, 15314. 5 ITM Web of Conferences 32, 02001 (2020) https://doi.org/10.1051/itmconf/20203202001 ICACC-2020 . [7] Jinsu Kim, Usman Cheema, Seungbin Moon. “,Face Recognition Enhancement by Employing Facial Component Classification and Reducing the Candidate Gallery Set. Department of Computer Engineering, Sejong University, Seoul, 143-747, Korea (sbmoon@sejong.ac.kr). [8] Nusrat Mubin Ara, Nishikanto Sarkar Simul, Md. Saiful Islam.“ \"Convolutional Neural Network(CNN) Approach for Vision Based Student Recognition System.\" 2017 20th International Conference of Computer and Information Technology (ICCIT), 22-24 December, 2017. [9] Xinxiong Liu, Wanru Wang “User Centered Design of Attendance Record System Based on Mobile Terminals\" 2017 International Symposium on Computer Science and Intelligent Controls. [10] Turgut Özseven, Muharrem Düenci “Face Recognition by Distance and Slope between Facial Landmarks\" Gaziosmanpaúa University Turkey, Tokat ; Karabuk University, Turkey, Karabük. .

Copyright

Copyright © 2022 Md Abu Talha Reyaz, K Sunil Kumar, Arun Kumar, Aathimuthu , Z. Qulitha Begum. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.