Writer-Independent Offline Signature Verification Using Deep Learning

Abstract

The signature of humans is an important feature in the field of biometrics. It is used as an authentication tool especially in the banking sector because all humans have a distinct signature and each signature has its features. So human signature is used to recognize a person. There has been a fair amount of work done in the field of handwritten signature verification but still, the problem is unsolved. The main intent of signature verification is to distinguish whether the signature is genuine or forged. The signature verification can be offline or online. This is a tedious task, principally in the case of offline because the dynamic information of the signature is not available. A brief survey of various offline signature authentication methods and recent advancements in the field has been represented in this paper.

Introduction

I. INTRODUCTION

The field of biometrics is widely used for authentication purposes. There are numerous biometric authentication methods but the handwritten signature is still used in today’s life. Therefore, it is predominant to make sure that the signature done is forged or a genuine signature. This authentication problem has attracted researchers to focus on handwritten signature verification. Research in signature verification is mainly bisected into two categories: offline and online signature verification. In the case of online, the Signature is taken through input devices such as phones, iPad, tablets, and other mobile application devices, hence the dynamic information of the signature can be extracted such as the position of the pen, pressure applied, inclination, and other information. In the case of offline signature verification, the signature is acquired by scanning the document in which the signature has been done and we get a digital image [21][4].

The main obstacle in offline signature verification is the high intrapersonal variability of a person. In other words, the same person cannot sign his signature the same the second time [25]. This problem makes handwritten signatures different from other biometrics and increases the complexity of offline signature verification. To tackle this problem many methods and models were proposed but due to lack of insufficient data, many models have failed in signature verification [24]. The methods of signature verification can be dissected into two types: Writer-Dependent (WD) and Writer-Independent (WI). In the WD method, a large number of samples is taken from the user to train the model. In WD system has to be trained for every user and it depends on the number of samples collected from each individual. In reality, it’s impractical to collect a huge number of samples from an individual [4]. In the WI method, a few samples are taken from the user and there is only one model for all the writers, and a model trained for a specific set of users can be mapped to all the writers [12].

The performance of the biometric system depends on the variations namely inter-class variation and intra-class variation [13]. Intra-class variation is observed between multiple observations of an individual sample whereas inter-class variation is the variation between images that have different class labels. The main challenge of signature verification arises when there is high intra-personnel variability [22]. Compared to iris, fingerprint, and other biometrics, signature shows variation between samples. If a sample has high intra-class variability and low inter-class variability, it becomes easier for the forgers to imitate the genuine sample. This issue usually arises in skilled forgery where a person practices imitating a particular user's signature. Hence skilled forgeries resemble genuine signatures to a great extent [9].

II. PROBLEM STATEMENT

Signature verifications are a crucial task in many organizations such as banks, governments offices, corporate sectors, etc. But, due to the involvement of a large number of documents this task is now becoming a tedious job. To overcome this, this paper proposes a signature verification approach using deep neural networks. Further, It utilizes a writer-independent approach where it demands less user signature dataset to make a robust model. Also, the proposed method uses the offline signatures present in the paper for verification.

III. LITERATURE SURVEY

A. Pre-processing

Since signature verification is a pattern recognition problem, pre-processing plays a major role. The main aim of this step is to make signatures standard, this eventually increases the accuracy of the model. Various steps that are incorporated with pre-processing are:

1. Noise Removal: Due to various acquisitional methods, scanned signatures often contain noises. S. Banerjee et al. [1] proposed the Gaussian noise detection scheme. It makes use of a four-directional noise detection scheme for the removal of noise from the image. Salt and pepper noise generated during the image acquisition phase can be eliminated using the median filter by running it through a filter mask of size 3x3 or 5x5 [2,3]. Avola et al. [4] presented the removal of the background noise by first inverting the grayscale pixel followed by setting an empirical threshold value. This keeps only the signature pattern, by removing the noises.
2. Binarization: To make the feature extraction easier, black and white images are obtained from the given input color image. This converts all the random pixel values to 1 or 0. DeepOtsu [6] algorithm makes use of an iterative deep learning framework, to produce the binarized image. The output obtained from the CNN is fed iteratively into the network for fining tuning. [6] proposed an approach of generating three grayscale images from the input image based on the Laplacian zero-crossing concept. Then ordinal structure fuzzy module is used to evaluate these three images to pick the best one.
3. Thinning: To reduce the influence of thickness differences of the pen, thinning operation is performed. Morphological operations are performed to make the given signature thin [7]. The various steps involved in morphological operations for thinning and closing are discussed in [3].
4. Normalization: To get reasonable results from the models it is necessary to have all the images of the same size, this is achieved through normalization. Normalization can be performed by resizing all the images to a specific dimension such as 256´256 [3].

B. Feature extraction

Extracting the crucial features from the given image plays a critical role in discriminating between genuine and forged signatures. The extracted features can be mainly classified into two types they are global and local features. Global features extraction considers the image as a whole to generalize the entire signature. Whereas local features extraction considers the signature images as patches by computing the multiple points. Debanshu Banerjee et al. [8] investigated the extraction of features present in the signature by first converting it into the corresponding signal. A binary variant meta-heuristic method called Red Deer Algorithm is used for feature extraction. Jain. A et al. [2] proposed the method of harnessing geometrical features of the signature using an artificial neural network. It derives a total of two global features and eight local features from the given signature image.  Avola. D [4] proposed the R-SigNet and Li Liu et al. [11]  Mutual Signature DenseNet (MSDN) architecture to automatically extract the features present in the given signature. It reduces the feature space by making use of a relaxed loss based on the multi-task approach. The proposed method utilizes fewer parameters to train the model, thereby leading to smaller feature space and reduced training time. Sharif. M et al. [3] investigated the extraction of global and local features through vertical and horizontal splitting. Geometric centers are computed to extract features through vertical splitting. Similarly, geometric centers of the top half and bottom half are computed to extract features through horizontal splitting. Finally, a Genetic algorithm is used to obtain the finest set of features among the extracted.

C. Approaches for Verification

Parcham. E et al. [12] proposed the amalgamation of Capsule Neural Networks (CapsNet) and CNN called CBCapsNet for feature extraction. This has the advantage of reducing the number of layers and parameters of the network and adds the capability of detecting the spatial changes in the component. A novel training model allows the network to be trained by twin images simultaneously by utilizing only one branch of the network layer, this eventually reduces the model size. Lu. X et al. [13] presented the cut and compare network. A pair of input images are first segmented using the Spatial Transformer Network (STN). Obtained two signature regions are contrasted with the help of the Attentive Recurrent Comparator (ARC). Distances are fused by making use of an adaptive distance fusion module. It also addresses the issue of intrapersonal variability by training the network using smoothed double-margin loss. Jain. A et al. [14] investigated the usage of a shallow convolution neural network (sCNN) to acquire the features automatically present in the given signature. Features extracted from the sCNN are fed into the SoftMax classifier to distinguish the signature. The proposed sCNN has fewer parameters and network layers, hence it takes less time to train the model. Ruiz. V et al. [16] presented the usage of the Siamese neural network (SNN) for the classification of the signature. On-demand artificial signatures are produced during the training phase using the compositional method. Then the combination of generated synthetic and original signatures is used to train the SNN.

Batool. F. E et al. [17] proposed the signature classification through distance measure. The features are extracted by calculating the eight geometrical features and twenty-two Gray Level Co-occurrences Matrix (GLCM). The obtained features are fused using the high priority index feature (HPFI). To select the optimal feature skewness-kurtosis controlled PCA (SKcPCA) is used. And finally, classification is made using the support vector machine. Shivashankar. S et al. [18] investigated the usage of the Galois field operator to obtain the texture representation of the signature image. At first, the histogram is constructed, and it is normalized using the Galois field operator. Then the derived bin values are used as the features of the signature and fed into K-NN for the classification. Agrawal. P et al. [20] proposed the automatic verification of bank cheques using deep learning and image processing. OCR was used to identify the typographic character, and CNN was used to identify handwritten digits and signatures. Scale Invariant Feature Transform (SIFT) is used to obtain the best features among the obtained. Finally, SVM is used for classification.

D. Comparative analysis

Table 1 provides the comparative analysis of the Feature and Data augmentation method, Classifier, Equal Error Rate (ERR), False Rejection Rate (FRR), and False Acceptance Rate (FAR).

Table 1 Comparative analysis

IV. METHODOLOGY

In this approach, the signature of the user is classified as a genuine or forged signature by comparing it with the original signature. There will be the following section in the system.

1. Training Phase
   • Pre-processing
   • Feature extraction
   • Training the Siamese model using contrastive loss
2. Testing Phase
   • Input the test signature and the original signature to the model
   • Model output the similarity score
   • Use threshold for classification

A. System Description

Figure 1.1 illustrates the proposed system's methodology or system flow diagram. During the training phase, the training data is first Pre-processed which involves Noise removal, segmentation, and normalization. Noise removal involves the removal of noise such as salt and pepper noise. Segmentation is performed using otsu's segmentation. Normalization is performed by dividing it by the maximum pixel value. Then the inversion is performed by taking the difference from the maximum value. Then Pre-Processed data is subjected to feature extraction which involves the usage of Pre-built architecture such as ResNet, AlexNet, and Xception. Finally, the last phase of the training phase is a classification which is done by Siamese Neural Network. Siamese Neural Network is a neural network that has two or more identical subnetworks having the same weights and parameters. Parameter updating is simultaneously done across both the subnetworks that are used to learn the similarity between the inputs by comparing their features. We are using contrastive loss during training and different metrics such as Euclidean distance, cosine distance, etc to find similarity scores.

V. FUTURE SCOPE

The proposed system with some advancements can be extended for automatic verification of checks in the banking system and registration offices. The proposed system can be combined with other domains such as Blockchain for high security, IoT, etc. Future work can design a system that is a combination of online and offline verification systems. Future work may also aim to develop advanced and sophisticated models for pre-processing and feature extraction which eventually increases the accuracy of the overall system.

Conclusion

In this paper, we have presented the existing offline signature authentication methods and the advancement in the field of signature verification. Various approaches have been proposed such as CBCapsNet, STN, sCNN, Cycle-GAN, Caps-Net, and SNN, etc but still, the accuracy needs to be improved. The accuracy obtained from the existing models is not sky-high and more research on offline signature verification is required. In today’s time forging a signature can be done fluently, therefore it is necessary to build an accurate model to discriminate between forged and genuine signatures. Future work may include improving the accuracy which can be done by proper image preprocessing and a combination of existing models.

References

[1] Banerjee, S., Bandyopadhyay, A., Bag, R., & Das, A. (2020). Four-directional detection-based gaussian noise removal. In Computational Advancement in Communication Circuits and Systems (pp. 269-278). Springer, Singapore. [2] Jain, A., Singh, S. K., & Singh, K. P. (2021). Signature verification using geometrical features and artificial neural network classifier. Neural Computing and Applications, 33(12), 6999-7010. [3] Sharif, M., Khan, M. A., Faisal, M., Yasmin, M., & Fernandes, S. L. (2020). A framework for offline signature verification system: Best features selection approach. Pattern Recognition Letters, 139, 50-59. [4] Avola, D., Bigdello, M. J., Cinque, L., Fagioli, A., & Marini, M. R. (2021). R-SigNet: Reduced space writer-independent feature learning for offline writer-dependent signature verification. Pattern Recognition Letters, 150, 189-196. [5] He, S., & Schomaker, L. (2019). DeepOtsu: Document enhancement and binarization using iterative deep learning. Pattern recognition, 91, 379-390. [6] Abdulfattah, G. M., Ahmad, M. N., & Asaad, R. R. (2018). A RELIABLE BINARIZATION METHOD FOR OFFLINE SIGNATURE SYSTEM BASED ON UNIQUE SIGNER’S PROFILE. INTERNATIONAL JOURNAL OF INNOVATIVE COMPUTING INFORMATION AND CONTROL, 14(2), 573-586. [7] Jindal, U., Dalal, S., & Dahiya, N. (2018). A combine approach of preprocessing in integrated signature verification (ISV). International Journal of Engineering & Technology, 7(1.2), 155-159. [8] Banerjee, D., Chatterjee, B., Bhowal, P., Bhattacharyya, T., Malakar, S., & Sarkar, R. (2021). A new wrapper feature selection method for language-invariant offline signature verification. Expert Systems with Applications, 186, 115756. [9] Wei, W., Ke, Q., Po?ap, D., & Wo?niak, M. (2021). Spline interpolation and deep neural networks as feature extractors for signature verification purposes. IEEE Internet of Things Journal. [10] Dey, S., Dutta, A., Toledo, J. I., Ghosh, S. K., Lladós, J., & Pal, U. (2017). Signet: Convolutional siamese network for writer independent offline signature verification. arXiv preprint arXiv:1707.02131. [11] Liu, L., Huang, L., Yin, F., & Chen, Y. (2021). Offline signature verification using a region based deep metric learning network. Pattern Recognition, 118, 108009. [12] Parcham, E., Ilbeygi, M., & Amini, M. (2021). CBCapsNet: A novel writer-independent offline signature verification model using a CNN-based architecture and capsule neural networks. Expert Systems with Applications, 185, 115649. [13] Lu, X., Huang, L., & Yin, F. (2021, January). Cut and Compare: End-to-end Offline Signature Verification Network. In 2020 25th International Conference on Pattern Recognition (ICPR) (pp. 3589-3596). IEEE. [14] Jain, A., Singh, S. K., & Singh, K. P. (2020). Handwritten signature verification using shallow convolutional neural network. Multimedia Tools and Applications, 79(27), 19993-20018. [15] Yap?c?, M. M., Tekerek, A., & Topalo?lu, N. (2021). Deep learning-based data augmentation method and signature verification system for offline handwritten signature. Pattern Analysis and Applications, 24(1), 165-179. [16] Ruiz, V., Linares, I., Sanchez, A., & Velez, J. F. (2020). Off-line handwritten signature verification using compositional synthetic generation of signatures and Siamese Neural Networks. Neurocomputing, 374, 30-41. [17] Batool, F. E., Attique, M., Sharif, M., Javed, K., Nazir, M., Abbasi, A. A., ... & Riaz, N. (2020). Offline signature verification system: a novel technique of fusion of GLCM and geometric features using SVM. Multimedia Tools and Applications, 1-20. [18] Shivashankar, S., Kudari, M., & Hiremath, S. P. (2021). Offline Signature Verification Using Galois Field-Based Texture Representation. In Data Science (pp. 269-278). Springer, Singapore. [19] Li, H., Wei, P., & Hu, P. (2021). AVN: An Adversarial Variation Network Model for Handwritten Signature Verification. IEEE Transactions on Multimedia. [20] Agrawal, P., Chaudhary, D., Madaan, V., Zabrovskiy, A., Prodan, R., Kimovski, D., & Timmerer, C. (2021). Automated bank cheque verification using image processing and deep learning methods. Multimedia Tools and Applications, 80(4), 5319-5350. [21] Hafemann, Luiz G., Robert Sabourin, and Luiz S. Oliveira. \"Writer-independent feature learning for offline signature verification using deep convolutional neural networks.\" 2016 international joint conference on neural networks (IJCNN). IEEE, 2016. [22] Zhang, Zehua, Xiangqian Liu, and Yan Cui. \"Multi-phase offline signature verification system using deep convolutional generative adversarial networks.\" 2016 9th international Symposium on Computational Intelligence and Design (ISCID). Vol. 2. IEEE, 2016. [23] Zois, Elias N., Alex Alexandridis, and George Economou. \"Writer independent offline signature verification based on asymmetric pixel relations and unrelated training-testing datasets.\" Expert Systems with Applications 125 (2019): 14-32. [24] Ayan E, Ünver HM (2018) Data augmentation importance for Classification of skin lesions via deep learning. In: 2018 Electric Electronics, computer science, biomedical engineerings’ meeting (EBBT). [25] Maruyama, T. M., Oliveira, L. S., Britto, A. S., & Sabourin, R. (2020). Intrapersonal parameter optimization for offline handwritten signature augmentation. IEEE Transactions on Information Forensics and Security, 16, 1335-1350.

Copyright

Copyright © 2022 Mohammed Roshan Ahmed, Mohamed Nouman, Mohith C Shekar, Manoj L M, Natesh M. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.