Multimodal Biometric Authentication for Smartphones

Abstract

Smartphones have become a crucial way of storing sensitive information; therefore, the user\'s privacy needs to be highly secured. This can be accomplished by employing the most reliable and accurate biometric identification system available currently which is, Eye recognition. However, the unimodal eye biometric system is not able to qualify the level of acceptability, speed, and reliability needed. There are other limitations such as constrained authentication in real time applications due to noise in sensed data, spoof attacks, data quality, lack of distinctiveness, restricted amount of freedom, lack of universality and other factors. Therefore, multimodal biometric systems have come into existence in order to increase security as well as to achieve better performance.[1] This paper provides an overview of different multimodal biometric (multibiometric) systems for smartphones being employed till now and also proposes a multimodal biometric system which can possibly overcome the limitations of the current biometric systems.

Introduction

I. INTRODUCTION

In the age of information technology, verification of identity has become crucial.Verification requires high security standards for personal data and protection of privacy against unauthorized usage. As of now, using a Personal Identification Number (PIN), Patterns are some of the most common methods used, but these methods have a myriad of vulnerabilities, especially with the progress of technology. Employing biometric techniques on smartphones offer more dependable identification methods based on unique traits or characteristics possessed by humans. The possible impacts of unauthorized access could be threatening personal data.  Thus, there is a trend of adding a novel authorization application which depends on an iris pattern to identify the smartphone user. Iris recognition is considered one of the most reliable biometric techniques to protect the device, its applications, and sensitive data. But no single biometric system is expected to effectively satisfy all the requirements when it is deployed in real world applications. Unimodal biometric systems like iris recognition systems have their own set of limitations.

The Unimodal biometric system has to tackle a diversity of conundrums like noisy sensor data, non-universality, intra-class variation, inter-class similarities, failure-of-enrolment, spoof attacks, restricted degree of freedom, unacceptable error rate, and many more. Multimodal biometric system utilizes information from more than one modality or multiple processing techniques or even both.  Therefore, Multimodal biometric systems are ones which coalesce more than one physiological or/and behavioural characteristics for enrolment, verification, or identification to enhance performance and reliability of the recognition system.

Some commonplace multimodal biometrics are: face and iris, iris and fingerprints, face and fingerprints, face and voice, face, fingerprints and iris, face, fingerprint and signature, etc. In the paper we will be focusing on a multimodal eye biometric system which combines the features of iris, pupil, and sclera multiple modalities or multiple processing techniques or both.[1,  5]

Therefore, Multimodal biometric systems are those which  integrate more than one physiological or/and behavioral  characteristics for enrollment ,verification, or identification to improve performance and reliability. Some common  multimodal biometrics are :face and iris, iris and fingerprints,  face and fingerprints, face and voice, face, fingerprints and  iris, face, fingerprint and signature, etc.

II. BIOMETRIC SYSTEM

Biometrics is constantly evolving especially in the field of secure IDs such as national identity documents, passports, or driver's licenses. Biometric systems make life safer and practical. Passwords, PINs, Smart Cards are some of the traditional ways of authentication. [3]General biometric systems capture the trait in the form of raw biometric data,processes data, and build an extract feature set that is a representation of the trait, the matching and comparing process generates score based on how closely the sample images in database,decision making phase decides whether to accept or decline the input based on score of input and the data in the database.[5]

III. MULTIMODAL BIOMETRIC SYSTEM

 Multimodal biometric system takes information from two or more biometric traits. Multimodal biometric systems are way more secure than unimodal biometric systems for authentication. Unimodal biometric systems can face problems such as lack of confidentiality, lack of universal space for samples, maximum comfort and user freedom while working with the system, attacks of database data fraud, etc. In a multimodal biometric system if one of the traits are absent due to some noise following trait value can be discarded and other trait values can be taken into consideration for matching the score. Hence unimodal biometric systems are preferred considering the security of the user.

 Figure 1 outlines the complete execution of the multimodal  biometric authentication system. The input modalities which are biometric traits, iris, pupil and sclera are taken as inputs by the system. The next step in the execution flow of the system is feature extraction. The prominent and important features required by the authentication system are extracted by this stage. In the next step,which is Fusion, values of the features of all three modalities are combined together. Here, a score of the image is created, which is further used in the last stage to compare it with the score of the test image. Depending on the similarity between the score of the test and train  image, the decision of accepting or rejecting the authentication takes place.

IV. LITERATURE SURVEY

A. Preprocessing

Preprocessing enhances the input image by eliminating unwanted extra  information like shadow, eyelash, reflection from the input image. One of the papers discusses  Feature Level Fusion of Iris and Sclera using Entropy Based CNN Features to Improve the Performance of Biometric Authentication. Initially, input image is taken from the database which is then transformed from RGB photograph to gray photo to maintain the computational complexity. Min-max normalization is used to carry out linear transformation on input photos to fit the data in particular range.[1] Later, the non linear bilateral filtering is used for smoothing  purposes by keeping the edges as it is. In one of the proposed systems, MSRCR is used to minimize the effect of nonlinear illumination with canny edge detection to identify the iris inner and outer circle boundaries .   

B. Segmentation

Eye segmentation is done to find out iris and sclera area boundaries for front and off angle images.  One of the proposed methods used CDM for segmentation to obtain required parts of iris and sclera and excluding noises like eyelashes, shadow or reflections. As a result  valid regions of iris and sclera are obtained  from an input image.Two binary maps obtained from eye images with noise are fused together based on their CDM for better sclera segmentation.

In addition to this, the proposed system also used the Blurred Image Detection method in which a blurred filter was applied for smoothing the blurry eye images in which a high energy frequency component was suppressed to increase clarity of image. Another segmentation method for iris ,sclera and pupil regions is by using convolutional neural networks(CNN) based on entropy values. In the proposed system, entropy is derived from contour based colour ,texture and brightness features. Convolutional neural community is used to cluster iris, sclera and pupil vicinity based on similarity obtained by using entropy measures.

C. Feature Extraction

In the eye biometric system color, texture and shape are the promising and reliable features which are stable over its lifetime for iris, sclera and pupil which can be used to generate the feature vector for correct authentication. In one of the  proposed systems we extracted the color and texture information from segmented iris and pupil region which is combined with the Y-shaped features extracted from sclera. Color histogram algorithm is used to extract the color features whereas log Gabor filter is used to extract the texture features. Sclera of the human eye is compromised with different layers  which form the stable blood vessel pattern which is unique for every person . These blood vessels form Y shaped branches which are stable and used to calculate sclera descriptor value. In the proposed system to find out Y-shape features (Shape), we look for the nearest set of neighbors in all line segments at regular distance and classify the angles between these neighbors to derive structure.In another method texture features for both iris and sclera are extracted by combining GLCM and Wavelet Transform.

1. Grey Level CO-OCCURRENCE MATRIX(GLCM): Grey Level Co-Occurrence Matrix (GLCM) is a texture feature extraction method results of which are better as compared to other textural feature extraction algorithms.[3] Difficulties in this are high computational complexity and also lack of global information. In this method two pixels separated by a definite distance in an image always have spatial relationships due to variation in their gray scale which describe the image from the interval of neighboring pixels, direction and degree of variation.
2. Contour Feature Extraction: Contours are characteristics of visual patterns available in the image which are obtained based on features like color, texture and brightness.[3] So contours are the subsets of features which helps to perform segmentation accurately with less quantity of features.
3. Entropy Features Extraction: It is a measurement of the degree of uncertainty that exists in a system. [3]Shannon's entropy is an important measure for evaluating  patterns and structures in the data which can be used to characterize texture in the input image. The entropy is obtained for the potent segmentation of iris pupil and sclera regions. In the proposed system these entropy values are calculated to recognize iris, sclera and pupil regions based on texture in the contour image.

D. Matching

We use these methods to compare the support match score value of test data with the values stored in a trained  database.In one of the papers, matching scores for both iris and sclera is calculated separately using Hamming Distance (HD) method. HD measures the similarity between two bit patterns. Then iris and sclera scores are normalized using the min-max rule. Then weighted score level fusion based on the sum rule is used and before addition weighting factor is multiplied with individual normalized scores corresponding to the iris and sclera.

Euclidean distance is used for pixel wise comparison of images. If the calculated distance is lesser than threshold value it is recognized otherwise rejected. Euclidean distance algorithm compares the stored data with the test data.

E. Fusion

Multimodal fusion in biometric system is generally classified into two categories:

1. PRE-Classification: In this, before applying any classification method or matching algorithm the  information is integrated. Pre-classification fusion categories are:

a. Data Level (Sensor Level): This is the process of combining multiple data  representing signals from a similar modality source. It is vulnerable to noise and failures because there is no preprocessing involved here. It is not very oftenly used.[3]

b. Feature Level (Early Fusion): Raw input data from different biometric traits is used in this.[3] Features extracted from  tightly coupled or time synchronized modalities are initially blended and then analysis is executed.

• Pre Classification: In this, information is combined before the decision of classifiers.
• Post Classification: In this, information is combined after the decision of classifiers.

c. Dynamic Classifier Selection: It estimates the accuracy of each classifier in the local region surrounding the input pattern which is to to be classified and chooses the classifier which is most likely to give the correct decision . It requires large data sets for estimating local classifier accuracy.

d. Rank Level Fusion: In this fusion, each classifier has a rank with respect to  every enrolled identity. The output from each biometric matcher is a subset of possible matches ranked in descending order of confidence values. Fusion can be done by taking into consideration  more than two biometric matching scores associated with an identity and finding out a new rank that would be used in the final decision.

e. Matching Score Level Fusion: In match score similarity between input data and data from database are compared and matched. Combining can be done at matching score level. It is also known as measurement or confidence level fusion or confidence level fusion.

f. Decision Level Fusion: Here, feature extraction from each biometric trait is done and after matching modules these extracted features are classified as accept or reject. The final output of multiple classifiers for different modalities is then  fused.

V. MODES OF OPERATION

Multimodal biometric system works in three modes of operation:

A. Parallel Mode

Numerous  sources of information are simultaneously obtained for recognition.

B.  Serial Mode

The output of one biometric trait is used  at first to reduce the number of possible identities, after that it moves to the next trait. As more than one source of information is not acquired simultaneously, recognition time is reduced.

C. Hierarchical Mode

In this, individual classifiers are fused in a tree-like structure. It is used when there are a large number of classifiers matching score level ie. when output from each biometric matching module is a set of possible matches along with the quality of each matching score and with confidence values.

VI. COMPARATIVE ANALYSIS

A. Pre-Processing

Table I: Comparative analysis of Preprocessing methodologies

In the above table, various preprocessing techniques are stated and are described in detail. The preprocessing is done according to the traits used for the system. For instance, in the first paper, the traits are iris, pupil and sclera, and the preprocessing includes RGB to gray image transformation, min-max normalization,and  bilateral filter.

In one of the papers for iris and sclera preprocessing MSRCR was used for image enhancement and to reduce noise in the image. For fingerprint preprocessing OCT technique is used and for iris preprocessing an iris camera was used.

B. Segmentation

Table II: Comparative analysis of Segmentation methodologies

Segmentation is necessary to segment the regions of interest. Various segmentation techniques like Entropy based segmentation, Color Distance Map(CDM) and RANSAC are studied in this paper.

C. Feature Extraction

Table III: Comparative analysis of feature extraction methodologies

Various feature extractions methodologies are employed in different papers mentioned above. For instance, the first paper which has employed iris, sclera and pupil as traits, has used Color histogram for extraction of  the color features of iris and pupil and Log Gabor Filter for texture features. For sclera, Y-shape features which are stable during eye movements are used.

D. Matching

Table IV: Comparative analysis of matching methodologies

Various matching techniques are used taking the type of fusion used into consideration. Most prominently used methods for matching are the Euclidean distance and Hamming distance methods.

E. Fusion Method

Table 4: Comparative analysis of  fusion techniques

By using appropriate fusion techniques it improves recognition rates. Various fusion level techniques are discussed. Fusion at feature level fusion works better because of easy implementation. Suitable fusion techniques should be chosen considering the time, space and computational complexity.

F. Accuracy And Equal Error Rate

Table V: Analysis of  Accuracy and Equal Error Rate

Conclusion

In the earlier section, we have seen various papers with their respective biometric traits for the multimodal biometric systems. We even compared their preprocessing, segmentation methodologies, feature extraction techniques, fusion and matching methods. We also found out the multimodal biometric system with highest accuracy among other systems.Various Traits and their fusion methods are studied in this paper. Analysis on the basis of accuracy and equal error rate is analysed. We will use feature level fusion in our proposed model as it fulfils all the requirements and can be considered the best for our multimodal biometric system. Finally, we have come up with our proposed method for a multimodal eye biometric system. This system will take biometric traits iris, sclera, and pupil as their regions of interest. Being a multimodal system, the biometric system will have higher accuracy compared to a unimodal system. In this system,we will be using entropy based CNN in order to segment the iris, pupil, and sclera. Color histogram algorithm is used to extract the color features whereas the log Gabor filter is used to extract the texture features. The Y-shape features of sclera can be used for the feature extraction. The fusion method feasible for this biometric system is feature level based fusion method.In feature level fusion, different biometric traits are pre-processed and feature vectors are extracted separately. All these feature vectors, in this case iris, pupil and sclera are combined to form a composite feature vector. This composite vector is used in the classification process. Due to rich information available in this level of fusion it is expected to perform better than score level and decision level fusion. Unimodal biometric systems are not completely reliable, they also suffer from spoofing attacks due to lack of invariant representation and noisy input. Therefore multiple biometric traits are combined together to achieve better performance of eye biometric authentication. We saw different multimodal biometric systems in this paper. Our proposed eye biometric system combines the prominent features of iris, sclera and pupil to improve the accuracy of recognition for the images acquired in unconstrained or relaxed environments. Entropy based feature selection reduces time required for segmentation because of selection of optimal set of feature selection. And the feature level fusion method is selected from the range of fusion methods considering that it is expected to perform better than score level and decision level fusion.

References

[1] Support Value Based Fusion Matching using Iris, Sclera Features for Person Authentication in Unconstrained Environment. [2] Multimodal Biometric Authentication for VR/AR using EEG and Eye Tracking Vrishab Krishna Research Mentorship ProgramSanta Barbara, California, USA [3] Multi-Algorithmic Texture Feature Extraction by Fusing Iris and Sclera Features for Unconstrained Images Mrunal Pathak*, Department of CSE, KL University, Guntur, A.P., India. E-mail: mrunalkpathak@gmail.com Dr. Vinayak Bairagi, Department of E&TC AISSMS, IOIT, S.P. Pune University, Pune, Maharashtra, India. [4] Multimodal Biometric System Iris and Fingerprint Recognition Based on Fusion Technique Abdullah Ahmed, Ahmed Shamil Mustafa International Journal of Advanced Science and Technology [5] Feature Level Fusion of Iris and Sclera using Entropy Based CNN Features to Improve the Performance of Biometric Authentication [6] Robust Multimodal Biometric Authentication Integration Iris,Face and Palmprint Fenghua Wang, Jiuqiang Han School of Electronics and Information Engineering, Xi’an Jiaotong University 710049 Xi’an, China

Copyright

Copyright © 2022 Mrunal Pathak, Shweta Hol, Madhura Belsare, Shivani Pokharkar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.