Handwritten Digit Recognition using Artificial Neural Network, Convolutional Neural Network, MNIST Dataset

Abstract

This review paper provides a comprehensive analysis of the application of Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs) for the task of handwritten digit recognition, focusing on the extensively studied MNIST dataset. The study delves into the strengths and weaknesses of both approaches, considering various aspects such as accuracy, computational efficiency, and robustness. Through an in-depth exploration of the literature and empirical evidence, this paper aims to offer valuable insights into the advancements, challenges, and future directions in the field of handwritten digit recognition.

Introduction

I. INTRODUCTION

The introduction provides a clear overview of the significance of handwritten digit recognition and the pivotal role played by the MNIST dataset in benchmarking various algorithms. It sets the stage for the reader by outlining the objectives and scope of the review.

The divided part is also known as a holistic approach for recognizing the digit without dividing it into subgroups or digits. Each word appears as a set of global features, e.g. ascender, loops, etc. Whereas the segmentation based on the approach of each of the word is divided into other subgroups either they are uniform or non-uniform or then subunits are considered independent. The Handwritten Digit processing system is domain and application-specific so it's not possible to design a generic system in which we can process all of the types of handwritten scripts and languages. There is a lot of work has been done on the European languages and Arabic (Urdu) language. Whereas the domestic languages which are Hindi, Punjabi, Bangla, Tamil, Gujarati, etc. are very less or looked into them due to their limited usage. In this paper, section II has described the basic principle of HDR which is generally followed by a detailed

II. LITERATURE REVIEW

The literature review is a notable strength of the paper, offering a comprehensive examination of the historical development of digit recognition techniques. It effectively traces the evolution from classical methods to the current dominance of neural networks. The inclusion of key contributions and seminal works in the field provides readers with a solid foundation for understanding the progression of techniques over time.

K. Gaurav, P. K. Bhatia [1] et al. investigate the various HDR pre-processing processes with a variety of photos, ranging from simple handwritten forms to papers with colored and detailed backgrounds and varying intensities. Among the preprocessing techniques addressed are skew detection and correction, contrast stretching, binarization, noise removal approaches, normalization and segmentation, and morphological processing techniques. It was determined that using only one pre-processing approach would prevent us from loading the image properly. Even with all of the aforementioned solutions in place, absolute precision in a preprocessing system may be impossible to attain. As a result, vital information is lost. The construction of a powerful segmentation algorithm is one of the method's major achievements. Slant angle, character boundaries, local maxima, and minima To optimize the search strategy for the optimal segmentation route on a grayscale image, upper and lower baselines, stroke height and breadth, ascenders, and descenders are all used. Over-segmentation is minimized using this method. Another contribution is the use of Hidden Markov Models (HMM) training not only for estimating model parameters but also for estimating various global and feature space parameters. In addition to HMM, probabilities are used to rank candidate characters and quantify shape information. By encoding a two-dimensional character picture in one dimension, the HMM shape recognizer's power is boosted.

G. Pirlo and D. Impedovo [3] developed a novel family of membership functions called Fuzzy membership functions for zoning-based classification in their work (FMFs). These FMFs could be easily modified to particular characteristics of a classification task to improve classification performance. Voronoi tessellation, a single optimizing step, is proposed for obtaining the optimal FMF, and the ideal zoning is explained by real programmed genetic algorithms. The results of the studies in the fields of HDR and character recognition reveal that the optimal FMF beats other membership functions in the literature that are based on the abstract level, ranking level, and measurement-level weighting of the models. Multiple algorithms are employed in supervised learning approaches to remove slope and slant from handwritten text. In this work, we removed slope and slant from handwritten text and normalized the size of the text images. One of the most essential elements of the recognition system was the construction of a system with high accuracy in preprocessing and recognition, both of which are based on ANNs. It provides a method for generating a handwritten Digit character from a series of strokes. A structural or form-based representation was used to depict the stroke as a string of shape components. A previously unknown stroke was discovered by comparing the string representation to the stroke database using a string-matching method. The components of a character's strokes were all determined. It provides a modified quadratic classifier-based offline handwritten numeral approach. In six popular Indian scripts, recognition (HNR) has been found. Using a multilayer perceptron, handwritten English characters were recognized. The characteristics were retrieved using boundary tracing and its related Fourier Descriptors. Examining the shape and studying its unique qualities is used to determine the character. An analysis was undertaken to identify the number of hidden layer nodes to achieve good performance in the backpropagation network. With minimum training time, handwritten English alphabets or characters were claimed to have a 94 percent recognition accuracy.

III. METHODOLOGY

The methodology section adequately outlines the architecture and training process of both Artificial Neural Networks and Convolutional Neural Networks. However, the paper could benefit from a more detailed discussion of the rationale behind the chosen hyperparameters and the selection of specific layers in the neural network architectures. Additionally, a brief overview of the data preprocessing techniques and their impact on model performance would enhance the clarity of the methodology.

1. Artificial Neural Network: A Detailed explanation of the architecture and training process of the artificial neural network employed in this study.Discussion on hyperparameter tuning and model selection.
2. Convolutional Neural Network: An in-depth exploration of the Convolutional Neural Network Architecture, focusing on its ability to capture spatial hierarchies and translational invariance in image data.Discussion on the use of convolutional layers, pooling layers, and fully connected layers.

IV. EXPERIMENTAL SETUP

The experimental setup is well-documented, covering data preprocessing, training, and evaluation procedures. The inclusion of information on normalization and augmentation techniques contributes to the reproducibility of the experiments. The paper effectively conveys the steps taken to ensure a fair comparison between ANN and CNN models, contributing to the credibility of the results.

1. Data Preprocessing: Detail on data preprocessing step, including normalization and augmentation, to enhance the model’s generalization capability.
2. Training and Evaluation: Description of the training procedure, validation strategies, and evaluation metrics used to assess the performance of both ANN and CNN models on the MNIST dataset.

V. RESULTS AND DISCUSSION

The results section presents a detailed analysis of the performance metrics for both ANN and CNN models. The comparative analysis provides valuable insights into the strengths and limitations of each approach. However, the discussion could be further strengthened by addressing potential sources of bias, such as class imbalances or model interpretability issues.

Presentation of experimental results, including accuracy, precision, recall, and F1 score for both ANN and CNN models. Comparative analysis highlighting the strengths and weaknesses of each approach.

VI. LIMITATION

One of the most difficult parts of a handwritten digit recognition system is that there are a lot of different handwriting styles which is a very personal behavior. Numbers may have different parts of stress, may be written from different angles, and may have different lengths of particular segments. Although these challenges are faced by machine learning developers, several steps such as fine-tuning the already defined models and creating state-of-the-art classification methods for predicting handwritten digits effectively by reducing computational cost, and time and also improving accuracy have been taken up already. Extensive research is also being conducted in this field to sustain it accordingly. Several issues may arise if this model is implemented on a huge scale. Recognizing handwritten digits, if used with bad intention, could ultimately lead to several issues. People may use such a technology to identify bank pins, atm pins, etc. to perform monetary felonies. On the contrary, even though issues like these may come up, measures could be taken to handle these issues effectively and sustain the use of this technology to automate many processes such as banking, address recognition, shipping systems, postal industry, etc. which will make it beneficial to use as it will ultimately have more pros than cons.

Conclusion

The conclusion effectively summarizes the key findings and their implications. The recommendations for future research directions add depth to the conclusion, encouraging further exploration in the field. However, providing a concise recapitulation of the main takeaways for readers would enhance the overall impact of the conclusion. Overall, this review paper makes a substantial contribution to the understanding of handwritten digit recognition using ANNs and CNNs. With a solid literature review and insightful analysis of experimental results, it provides a valuable resource for researchers and practitioners in the field. Addressing the aforementioned suggestions would further enhance the clarity and completeness of the paper. A summary of the findings, insights gained, and implications for the field of handwritten digit recognition. Recommendations for future research directions and improvements in model architectures. We study some Handwriting Digit Recognition and Artificial Neural network-based recognition algorithms to decide on the finest algorithm in terms of many aspects such as accuracy and performance. Different authors proposed many models and they took some criteria such as implementation time has been also taken into consideration. Random and standard datasets of handwritten digits are used to calculate the algorithms. The results show that DNN is the finest algorithm in aspects of accuracy and performance. CNN algorithm and DNN are almost equal in terms of accuracy. DNN algorithm, however, was finer than CNN and DBN in aspects of execution time. By recognizing the correct digits, the margin of errors may take place with similarities between the digits.

References

[1] Swapna Prava Ekka, “Recognition of Handwritten Digits using Proximal Support Vector Machine”, (2014), National Institute of Technology Rourkela, unpublished. [2] Viragkumar N. Jagtap, Shailendra K. Mishra, “Fast Efficient Artificial Neural Network for Handwritten Digit Recognition”, (IJCSIT) International Journal of Computer Science and Information Technologies, Vol. 5 (2), (2014), ISSN Number: 0975-9646, pp. 2302-2306. [3] Saeed AL-Mansoori, “Intelligent Handwritten Digit Recognition using Artificial Neural Network”, Journal of Engineering Research and Applications, Vol. 5, Issue 5, (Part -3) May (2015), ISSN: 2248- 9622, pp.46-51. [4] S M Shamim, Mohammad Badrul Alam Miah, Angona Sarker, Masud Rana, Abdullah Al Jobair, “Handwritten Digit Recognition using Machine Learning Algorithms”, Journal of Computer Science and Technology: DNeural & Artificial Intelligence, vol. 18, Issue 1, (2018), ISSN: 0975-4172, pp.0975-4350. [5] Md. Anwar Hossain, Md. Mohon Ali, “Recognition of Handwritten Digit using Convolutional Neural Network (CNN)”, Journal of Computer Science and Technology: D Neural & Artificial Intelligence, Vol. 19, Issue 2, (2019), ISSN: 0975-4172, pp.0975-4350. [6] Saqib Ali, Zeeshan Shaukat, Muhammad Azeem, Zareen Sakhawat, Tariq Mahmood, Khalilur Rehman, “An efficient and improved scheme for handwritten digit recognition based on the convolutional neural network\", (2019), unpublished. [7] Savita Ahlawat, Amit Choudhary, Anand Nayyar, Saurabh Singh, Byungun Yoon, “Improved Handwritten Digit Recognition Using Convolutional Neural Networks (CNN)”, (2020), unpublished. [8] Aarti Gupta, Rohit Miri, Hiral Raja, “Recognition of Automated Hand-written Digits on Document Images Making Use of Machine Learning Techniques”, Journal of Engineering and Technology Research, (2021), ISSN: 2736-576X. [9] V. Gopalakrishan, R. Arun, L. Sasikumar, K. Abhirami, “Handwritten Digit Recognition for Banking System”, Kings College of Engineering, Punalkulam, Pudukottai, Journal of Engineering Research & Technology (IJERT), (2021), ISSN: 2278-0181. [10] Ritik Dixit, Rishika Kushwah, Samay Pashine, “Handwritten Digit Recognition using Machine and Deep Learning Algorithms”, Journal of Computer Applications, Vol. 176 – No. 42, July (2020), pp.0975 – 8887.

Copyright

Copyright © 2024 Prince Singh Yadav, Sanket Singh, Anurag Tiwari, Piyush Mishra. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.