Real Time Character Recognition using Convolution Neural Network

Abstract

Handwritten recognition of character (HCR) is a significant element in the current world and one of the focused fields in image processing and pattern recognition research. Handwritten recognition of character refers to the process of converting hand-written character into printed/word file character that in many applications may greatly enhance the interaction of man and machine. The styles, varied sizes and orientation angles of the current characters are tough to parse with large variances. In addition, it is hard to split cursive handwritten text as the edges cannot be clearly seen. Many ways of recognizing handwritten data are available. The proposed research is based on 5*5 convolution neural network where the performance of the system has been enhanced in terms of accuracy, precision and recall the data set. The research utilized the real time photos. The processing approaches are followed by binarization, skeletonisation, dilution, resizing, segmentation and extraction. The character characteristics are sent to CNN to train the models after preprocessing. The research achieved 92% accuracy and time delay while detecting the real time Images.

Introduction

I. INTRODUCTION

The handwriting recognition of image processing and pattern recognition is one of the most exciting and hard study sectors in recent years [1]. It makes an important contribution to advancing the automation process and enhances the interaction between man and machine in many applications. A number of research studies have focused on novel methodologies and strategies that minimize processing time while increasing accuracy of recognition [2][3]. Manual recognition is often classed in two sorts as recognition techniques online and off-line. The writing is normally optically recorded by a scanner in off-line recognition and the completed written picture is provided. In the online system, however, subsequent points are represented as two-dimensional coordinates depending on the time, and the author also has the order of the strokes available. Due to the time information provided with the previous online approaches, the handwritten characters were recognized by them as better to their off-line counterparts [4]. The neural networks were nevertheless effectively employed in offline systems to obtain comparable high accuracy of recognition [5].

Neural networks in many types of pattern recognition have lately been utilized. There are diverse manuscripts of various people, hence handwritten characters are quite difficult to distinguish. Hand-written recognition of the character is a field of pattern recognition which in recent decades has been a topic of investigation. In handwritten character recognition, neural network plays a major role. Many character acknowledgment reports in English were released however the high accuracy of recognition and the minimal training time for handwritten English characters utilizing the neural network are an outstanding challenge. An automatic handwriting identification method for the English-language character is therefore extremely important [6]. This article attempts to construct an automated, handwritten, English-language character recognition system with high accuracy of recognition and low training and classification time. Experimental results demonstrate that this document's strategy to recognition of English character gives excellent accuracy of recognition and shortest training time.

II. PROBLEM STATEMENT

Hand-written information may still be identified and extracted during the scanning process. There are a lot of reasons: the original papers can be of poor quality since the paper rapidly deteriorates; the fly has comments; signatures are nearly always unreadable (not to mention that half of the population writes "1" as if they were "7" and the other of half writes "7" as if it were "4”). Manufacturers, especially the health, pharmaceutical, insurance sector and the banking sector, consequently have a challenge in handwritten forms. A system with high accuracy of recognition and minimal validation is therefore required.

III. LITERATURE REVIEW

Sanjay Kumar et al. [7] suggested work on offline textual identification of character, especially to detect the use of neural networks by doctors and to match prescription drugs. OCR algorithm guesses and stores the character throughout the procedure. It determined that only physicians handwriting can predict, but accurate words are not detectable.

J. Pradeep et al. [8] have been presenting their work with the usage of a multi-layer feed forward network to recognize alphabetical character in offline, with a novel function extraction procedure called diagonal feature extraction. In standard format, JPG, PNG etc. are the images containing hand-written data. The hidden layers are used to activate the Log sigmoid function and the output layer is a competitive layer. The experimental findings reveal those 54 diagonal method characteristics and a precision of 98.5 percent for 69 diagonal methods are achieved with a maximum recognition precision of 97.8 percent.

In a handwritten character recognition literature study, Mansi Shah et al [9] compared the OCR with the HCR. They also have a clear analysis of existing handwritten character recognition systems. It found that the neural network is the primary choice for training and a completely automated network which recognizes all types of handwriting is impossible to construct.

A survey of manuscript identification strategies for English alphabets was provided to Manoj Sonkusare et al [10]. They explored global skew correction techniques that adjust the text line to be aligned horizontally during the picture scanning in their work. In the pre-processing step, slant correction is also carried out that regulates the tilt of the type. They agreed that a lot more effort has to be done in the field of UNHCR to provide a viable solution that is accessible to everybody.

A comprehensive methodology for offline English recognition of character, Suman Avdhesh Yadav et al. [11] have developed a database of 2600 samples from 100 writers for each character. 1041 samples were evaluated on the neural network and the rest were evaluated on the recognition model. They employed the neural network method for feedback. Routine training procedure is implemented once the network is formed and error propagation error is determined over the connection. They are well aware of the abovementioned work at 86.74 percent.

In the handwritten identification, Abdulloh Al-Mubarok and Hertog Nugroho [12] worked on a hierarchical matching of character. Handwritten alphabets are here turned into charts on the basis of their structure of the skeleton. For training purposes, the CEDAR dataset comprises of 5632 city terms, 4938 state terms and 9454 zip codes. CEDAR dataset also has an unconstrained writing, implementing and writing style. There was a 93.40% precision rate for the suggested approach, and comparable looks, forced identification and preprocessing deformation causes were present.

In a review of the recognition of hand-written English letter Nisha Sharma et al. [13] suggested many techniques to the recognition and performance of hand-written character. Different extraction methods have also been evaluated and the most striking characteristic is the 98 per cent effectiveness Fourier descriptor and employs the SVM Classifier. They found that although several methodologies exist, a lot more study is needed to provide a full software solution in this field.

A.George and V.J. Pillai[14] have suggested to use their artificial neural network to implement the VNPR system. They built a database of 40 pictures of various number plates for vehicles. It is used to transform grey images into binary images, which optimizes the inter class variation between the pixels. This study introduced a novel notion of the probabilistic neural network. You have trained 100 characters in building a database and achieved 91% accuracy with this technique. In addition, they were expanding their database for better outcomes.

IV. OBJECTIVES

1. To Study and analyze the existing techniques of Real Time Character Recognition System.
2. To design, implement and evaluate 5-layer CNN model for Real Time Character Recognition for performance and enhancement.

V. HARDWARE AND SOFTWARE REQUIRED

1. Hardware: Digital camera for capturing images, Laptop for training and testing the CNN model.
2. Software: Spyder, Anaconda, Numpy, Keras, Tensorflow, Matplotlib.

VI. METHODOLOGY

1. Image Acquisition: The required image has to be scanned with the cam-scanner in an image acquisition. Those images should have a specific format such as PNG, JPG, bnp etc.
2. Preprocessing: The preprocessing is a first procedure for the picture to be performed. Preprocessing methods can improve the input picture and prepare it in a character identification system for a following phase. The several tasks performed on the input picture are:

a. Grayscale: It is a technique of converting RGB colour picture to an image of grey intensity.

b. Binarization: It is a procedure by means of thresholds to transform a grey image into the binary picture.

c. Noise Removal:  Noise from the gadget mechanism may be random or salt & pepper noise. Various approaches are available to minimize picture noise.

d. Edge Detection: It is used to locate the limits inside the pictures. In edge detection, morphological gradient operators are utilized as they improve the intensity of a character's edge.

e. Skeletonization: It compresses all lines to one pixel thickness, saving the space in the memory and storing data from the input characters, as well as decreasing the processing time.

3. Segmentation: The character sequence/series in a picture are split into the character set. The picture processed is separated into line and segmentation of characters.

a. Line Segmentation: Each line has a weight-based algorithm that is segmented for easier treatment. The main issues of handwritten character recognition segmentation of text lines are:

• A distinctive angle of skew may be found in each text line.
• The next text line may be related to part of the text.

b. Character Segmentation: After each line has been separated, each character must now be segmented. The segmentation of the characters is an important stage as the exactness of the letters depends on the segmentation of the letters. There are several ways used to segment the character but we have provided our own algorithms to segment each character so that accuracy is readily learned within the artificial neural network.

4. Feature Extraction: Extraction of functions is called transforming the input data into the collection of functionalities. Feature removal is a particular feature of the reduction techniques of dimensionality. A high number of variables are usually analysed by a huge quantity of memory and compute capacity or an algorithm of classification which adapts to the training sample and generates badly for new samples. If the data input is too huge to be processed, the data is translated into a smaller feature set. Of course, selecting the type of feature extraction technique is vital as it is a crucial aspect for pattern detection systems performance.

The 8-neighbor approach has been employed in our work to obtain information about the border of a handwritten character. This scans the binary picture to the border. The search follows in the direction of the clock. For every foreground pixel p, the linked component containing p is the set of all foreground pixels associated with it. When you find a white pixel, it checks a fresh white pixel, etc. The trace automatically follows the border. If the initial pixel is detected, the co-ordinates of the position will be assigned to the programme indicating that the border originates. The new pixel is allocated as a new point of reference and searches are started by the eight next doors. Thus, the start point co-ordinates are modified depending on the position. The associated coordinative is saved in an array to compute Fourier Descriptors as the tracer goes across the borders of the picture. The software always checks during the boundary tracing procedure if the first boundary co-ordinates are identical to the last. Once it has been reached, this implies the whole border is traced and the border tracing operation is finished.

A. Building CNN model

CNN may basically be seen as a prolongation of classical neural network models, such multilayer perceptron (MLP). The design of a CNN model comprises specific layers for extracting features from the raw input area and a fully connected neural network model with classificator for logistical regression. The properties that are generally termed maps of these particular layers are generated and are then transformed into inputs for a fully connected neural network, which in reality is an MLP model. It can be noticed that the input for CNN is handled in two phases before processing in the classification.

Both phases include transformation and sub-sampling. The input dimensions are likewise reduced for both turnover and subsampling procedures. The transformation of a single double dimension input matrix into a smaller two-dimensional matrix or feature maps will take place throughout a transformation. At the end of the second step, each map is flattened or transformed from the two-dimensional matrix to a single dimensional matrix so that the maps are ready to be classed as mlps or neural network.

The fundamental aim of CNN converting and subsampling is to extract characteristics from raw input data. To achieve this goal, multiplications of tiny kernel arrays and certain portions of a two-dimensional input array are conducted in convolution processes. The kernel will be switched from left to right and from top to bottom over certain parts of the input matrix to create a single smaller dimension of the characteristics.

After processing, each feature map will be sub-sampled or pooled to reduce its dimensions. The function employed for sub-sampling in this study is the pooling of significant characteristics.

A dropout regularization method should also be performed in the training stage of the model to prevent overfitting and enhance the performance of the CNN model. Some neurons in CNN layers are inhibited randomly with the Bernoulli distribution using a dropout method. All the neurons in every layer of the CNN model are subsequently triggered again during the test phase.

Srivastava et al. indicated that dropped algorithms in various benchmark data sets can increase the performance of neural networks.

The maps of function are flat so that they are available for classification with the MLP or a fully connected neural network after various transformation and subsampling activities.

VII. RESULTS AND OBSERVATIONS

The findings from our experiments demonstrated a precision rate of 92.46 percent for a minimum training duration. The training target is at 0.000001 and the validation performance at the time of 92 is 0.25013. We did not use enough characteristics to recognize several pairings of alphabets such as B and E,Y and V for the ANN training course. To obtain the greatest accuracy rate, much more detailed characteristics have to be retrieved. The identification of the handwritten nature of the offline depends on the quality of the content. Future work can involve not only characters but also words, sentences and even the whole manuscript. To recognize the offline manuscript, our suggested method delivers a good result. Our suggested approach takes shorter training time when examining alternative handwriting approaches. Moreover, it is easy to apply our approach.

A confusion matrix is a table which is usually used in a number of test data, where the true values of a classification (or classification) model are known.

Based on the research topic, the evaluation indices may change. The F1 accuracy and recall mean precision, recall and harmonic are standard evaluation criteria.

VIII. FUTURE WORK

This HCR method has been widely applied. Recent technological advances in recent days have pushed farther limitations for man to get rid of obsolete equipment that has caused discomfort to his use.

This device is a keyboard in our scenario.

1. If we are not fluent in keyboard as genuine word writing
2. When any keyboard keys are broken
3. The keyboard has scripts in only one language
4. Each keyboard character has to be found in a keyboard that takes time
5. It is hard to easily integrate the keyboard to a touch-enabled handheld device.

On the other hand, we may achieve benefits such as multiple language support

a. No keyboard needed

b. Real global write style support

c. Comfortable to handle enabled devices while using the HCR programme on any device.

d. Easily documented previously handwritten records may also be uploaded to the programme

Conclusion

A system for identifying hand-written English alphabets have been suggested and developed. We conducted our experiment with numerous handwriting styles over all English alphabets. Experimental results indicate that alphabets with 92.6% on average accuracy have been identified successfully by the machines, which in some applications is considerable and acceptable. In future, this misclassification of the same patterns will improve and similar experiments may be conducted on a huge data package and on some other optimal network settings to increase the machine\'s accuracy.

References

[1] S. Mori, C.Y. Suen and K. Kamamoto, “Historical review of OCR research and development,” Proc. of IEEE, vol. 80, pp. 1029-1058, July 1992. [2] N. Arica and F. Yarman-Vural, “An Overview of Character Recognition Focused on Off-line Handwriting”, IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, Vol.31 (2), pp. 216 - 233. 2001 [3] V.K. Govindan and A.P. Shivaprasad, “Character Recognition – A review,” Pattern Recognition, Vol. 23, no. 7, pp. 671- 683, 1990. [4] R. Plamondon and S. N. Srihari, “On-line and off- line handwritten character recognition: A comprehensive survey,”IEEE. Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, no. 1, pp. 63-84, 2000. [5] U. Bhattacharya, and B. B. Chaudhuri, “Handwritten numeral databases of Indian scripts and multistage recognition. [6] Verma B.K, “Handwritten Hindi Character Recognition Using Multilayer Perceptron and Radial Basis Function Neural Network”, IEEE International Conference on Neural Network, 4, pp. 2111-2115, 1995. [7] Sanjay Kumar Sanjay Kumar, Narendra Sahu, Akash Deepu, Khushboo Gavel and Miss Rumi Ghosh, “Offline Handwriting Character Recognition (for use of medical purpose) Using Neural Network”, International Journal Of Engineering And Computer Science ISSN: 2319-7242, vol. 5, pp.18612- 18615, Oct.10, 2016. [8] J. Pradeep, E. Srinivasan and S. Himavathi, “Diagonal Based Feature Extraction For Handwritten Alphabets Recognition System Using Neural Network”, International Journal of Computer Science & Information Technology (IJCSIT), vol. 3, no. 3,Feb 2011. [9] Mansi Shah AND Gordhan B Jethava, “A Literature Review On Hand Written Character Recognition”, Indian Streams Research Journal ISSN:-2230-7850, vol. 3,issue 2, March 2013. [10] Manoj Sonkusare and Narendra Sahu, “A Survey On Handwritten Character Recognition (Hcr) Techniques For English Alphabets”, Advances in Vision Computing: An International Journal (AVC), vol. 3, no.1 , March 2016. [11] Suman Avdesh Yadav, Smitha Sharma and Shipra Ravi Kumar, “A Robust Approach For Offline English Character Recognition”, International conference on futuristic trend in computational analysis and knowledge management (ABLAZE 2015), 2015. [12] Abdulloh Al Mubarok and Hertog Nugroho, “Handwritten Character Recognition using Hierarchical Graph Matching,”, The 8th International Conference on Advanced Computer Science and Information Systems (ICACSIS) 2016. [13] Nisha Sharma, Tushar Patnaik and Bhupendra Kumar, “Recognition for Handwritten English Letters: A Review”, International Journal of Engineering and Innovative Technology (IJEIT), vol. 2, ISSUE 7, January 2013. [14] George and V.J Pillai, “VNPR system using Artificial Neural Network”, International Conference on Circuit, Power and Computing Technologies [ICCPCT], 2016.

Copyright

Copyright © 2022 Sajad Ahmad Gassi, Ravinder Pal Singh, Dr. Monika Mehra. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.