Deep Learning based Handwritten Polynomial Equation Solver

Abstract

Polynomials are algebraic expressions involving a sum of powers in one or more variables multiplied by its coefficients. If x is a variable, a0x n +a1x n-1 +a2x n-2 +....+an then it\'s a (n) powers polynomial. Human is capable to solve this type of mathematical problems. In this work, we propose a system in which machines can achieve the cognitional skills that can understand the problem by visual context. By taking an input image of Handwritten polynomial equations and simplifies the problem by generating the answer as an output. Here machine can able to solve quadratic, cubic, quartic, quantic, sextic as well as (n) powers polynomials. This proposed work can be workable in an embedded system as well as a mobile application. In this scope for recognition purposes, we use a CNN model. Robust handwritten character recognition is a tricky job in the area of image processing. Among all the problem handwritten mathematical expression recognition is one of the complicated issue in the area of computer vision research. Segmentation and classification of specific character makes the task more difficult. In this paper a group of handwritten quadratic equation as well as a single quadratic equation are considered to recognize and make a solution for those equations. Horizontal compact projection analysis and combined connected component analysis methods are used for segmentation. For classification of specific character we apply Convolutional Neural Network. Each of the correct detection, character string operation is used for the solution of the equation. The proposed workflow system automatically simplifies the Handwritten polynomial equation and has been done a really good performance. Developing an automatic equation recognizer and solver has been a desire of the researchers who worked in the field of NLP for many years.

Introduction

I. INTRODUCTION

The effectiveness of mathematics in human daily life and natural science is like a blessing of God. All places of science are     standing with the influence of mathematics. Since polynomials are used for describing various types of curves, humans use them in real-world problems like graph curves. For example, engineers use polynomials to graph the curves of roller coasters and bridges. Polynomial functions are used in economics to do cost analyses. Polynomials are mostly used to the 3D curves representation and surfaces of the objects, find the motion of a particle under the influence of gravity, forecast sales trends, develop profit margins, nursing, psychiatrists are used to determining schedules and keep records of patient progress. Nowadays, solving a handwritten mathematical equation with the help of AI is a supreme area in scientific research. Math expressions contain different sizes and two dimensional convention. The major part of this work is segmentation to the sequence of the expression and through the recognition, the process simplifies the math and shows the output to the user. From the segmented image, classification is done by using a CNN model. CNN eliminated property from the image by a sequence of actions. With the help of the CNN model after successfully recognizing each of the segmented digits and operators, we perform a string operation to calculate   those expressions.

II. METHODOLOGY

In the proposed system develops a model which recognize and evaluates the handwritten polynomial equation using  convolutional neural network. Convolutional Neural Network is used for classification, image processing and segmentation .we propose a system in which machines can achieve the cognitional skills that can understand the problem by visual context. By taking an input image of Handwritten polynomial equations and simplifies the problem by generating the answer as an output. In this approach, the main hub is on handwritten polynomial equation simplify. As there is not any proper work that can successfully handle the problem for handwritten images. The past work only focuses on the recognition level of the equation and not concentrated on multivariable identification in the equation to solve the problem. This work motivated to handle a multivariable segmentation and recognition efficiently and show the result is and every variable in the equation. After preprocessing, segmentation, and recognition of the input image, generating a string equation from that image and simplify that expression is the main target.

A. Preprocessing

Preprocessing is the method in which the input image transforms and modified to make it a suitable quality for the recognition purpose. At first, convert the original input image into grayscale image cause in the color image the identification of character is more challenging. Then removing the noise from the grayscale image makes it a more suitable quality. At last turn the image into binary inverse. This method changes all the pixel values 0-255. This will reduce our computational time for segmentation. Eliminating all the unnecessary pixel values as much as possible. Fig 2(a) and 2(b) shows the input image and preprocessed image respectively.

B. Line Segmentation

In the input image, there will be a multiline of characters. Thus, by applying text line segmentation on the image that can be identified is there multiple handwritten equation exits or not. In the input image between two lines have a minimum horizontal  gap approximately 5-pixel size. By iterate through horizontally in the image, if we find the pixel value is 255 (white pixel), that means it’s a text. The white pixel value is considered as a text. If the sum of a horizontal row is 0 that   means it’s a black row and it is considered as a gap between two lines. Line segmentation massively used in many   languages [11] and earn a great achievement. Fig 3 shows the line detection of the preprocessed image.

C. Character Segmentation

Segmentation of a character is a way to extract individual images of the character from the line image [12]. As the preprocessed image have only binary pixel values, it is easy to find the character indexes. Every two characters have a minimum gap between them. To find a character we calculate the summation of all pixels from each column in the image. If the resulting sum of each column is less than or equal to five (Black pixel value 0) then it is called a gap and suspect as a character. So, the idea is in the y-axis connected white pixel is considered as a separate character shown in fig 4. After finding the gaps between each character it is easy to separate the images using this method [1]. To remove the unnecessary horizontal gaps from the images, calculate the first-pixel value index and the last pixel value index in the column from the image, and copy all the pixels between them. Finally, resize the separated image into pixel size of (28×28). Fig 5 shows the segmented characters.

D. Dataset Preparation

For training, this proposed model used MathNet [13] dataset for collecting the handwritten arithmetic symbols, MNIST [14] for handwritten digits and EMNIST [15] for handwritten alphabets which used as a variable in polynomial equations. We prepared a dataset containing 21 classes, each class contains 800-900 image data in total 16,000 binary images. Which include 0-9 numerals, 8 variables, and 3 mathematical symbols and operator. For better computational purposes all images converted into the 28×28-pixel size and kept in Comma-separated values (CSV) format. In fig 6 shows some dataset sample images.

IV. FUTURE SCOPE

In future days the main focus will be to try to raise the precision level and build a segmentation system that can successfully  segment   two connected digits, and also increase the performance level of the dataset.

Conclusion

In this presence, handwritten polynomial equation simplification has been described. To simplify the math, the main task don in the feature extraction from the image and recognition with the help of the CNN model. If the CNN model classifies correctly all of the segmented images then this will be generated the correct list of equations. Which will be better for the simplification part. This is a successful representation of the state of the art.

References

REFERENCES [1] Jalal Uddin Mahmud, Mohammed Feroz Raihan and Chowdhury Mofizur Rahman, “A Complete OCR System for Continuous Bangla Characters\", IEEE TENCON-2003: Proceedings of the Conference on Convergent Technologies for the Asia Pacific, 2003. [2] Fuad Hasan, Shifat Nayme Shuvo et al, “Bangla Continuous Handwriting Character and Digit Recognition Using CNN”, 7th international Conference on Innovations in Computer Science & Engineering (ICICSE 2019), vol 103, springer, Singapore. [3] Gharde, S. S., Baviskar, P. V., & Adhiya, K. P. (2013) Identification of Handwritten Simple Mathematical Equation Based on SVM and Projection Histogram. International Journal of Soft Computing and Engineering (IJSCE),3(2),425 [4] Francisco Álvaro and Joan Andreu Sánchez, “Comparing Several Techniques for Offline Recognition of Printed Mathematical Symbols”, IEEE,ICPR 2010. [5] Christopher Malon, Masakazu Suzuki, and Seiichi Uchida, “Support Vector Machines for Mathematical Symbol Recognition”, Technical Report. [6] Fotini Simistira, Vassilis Katsouros and George Carayannis, “A Template Matching Distance for Recognition of On-Line Mathematical Symbols”, Institute for Language and Speech Processing of Athena - Research and Innovation Center in ICKT, Athens, Greece. [7] Dipak D. Bage et. al,“A new approach for recognizing offline handwritten mathematical symbols using character geometry”, International Journal of Innovative Research in Science, Engineering and Technology Vol. 2, Issue 7, July 2013. [8] Catherine Lu Karanveer Mohan.‘ Recognition of Online Handwritten Mathematical Expressions Using Convolutional Neural Networks’ cs231n project report stanford 2015. [9] Sanjay S. Gharde, Vidya A. Nemade and K. P. Adhiya,“Evaluation of Feature Extraction and Classification Techniques on Special Symbols”, IJSER, Volume 3, Issue 4, 201 [10] Ahmad-Montaser Awal ,Harold Mouchère, Christian ViardGaudin.‘Towards Handwritten Mathematical Expression Recognition’ 2009 10th International Conference on Document Analysis and Recognition.

Copyright

Copyright © 2022 Vedavathi K, A Swapna. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.