Smart Essay Grading

Abstract

With the onset of the technological revolution, the efficiency of the current manual systems has been improved drastically and the accuracy of the products produced are also increasing. The most affected of the fields from this technological change is Education. The combination of Education with technology has been coined as a new term of edtech. Assessment in the Education system plays a significant role in judging student performance. Humans are becoming more interested in using automated tools. Consequently, in the last few years, the use of automatic assessment methods in the education system and student response evaluation has increased significantly. There is currently no adequate evaluation mechanism for grading essays and short responses; the computer-based evaluation system only works for multiple-choice questions. For the past few decades, many researchers have been working on automated essay grading and short answer scoring, but evaluating an essay while taking into account all the criteria, such as the content\'s relevance to the prompt, the development of ideas, cohesion, and coherence, is still a difficult task. We examined the limits of the most recent studies and research trends while studying the Artificial Intelligence and Machine Learning approaches used to assess computerized essay grading. In this project, we have studied different uses of Machine Learning and how we can improve the efficiency of the Essay Scoring using AI and other algorithms. In this project we also aim to highlight the problems faced by teachers and the possible solution which could be designed to overcome the problem.

Introduction

I. INTRODUCTION                                                                    

With the onset of the technological revolution, the efficiency of the current manual systems has been improved drastically and the accuracy of the products produced are also increasing.

The most affected of the fields from this technological change is Education. The combination of Education with technology has been coined as a new term of edtech.

One of the primary evaluation factors that teachers use to assess student achievement is essays. Due to the subjectivity of essay evaluation, teachers indicate that it takes a considerable amount of time.

Because of subjective nature of the essay variation in grades usually occurs. Solution to such problem is automatic essay evaluation.

The workload of teachers will be lessened, and there will be less variety in grades due to human variables, thanks to computerized essay evaluation.

An Essay Scoring system takes as input an essay written for a given prompt, and then assigns a numeric score to the essay reflecting its quality, based on its content, grammar, and organization, also reduces effort.

A. Aim

The primary aim of Essay Scoring system is to provide immediate scoring and diagnostic feedback for the students’ writings in order to motivate them to improve their writing proficiency on the topic.

Also Smart essay grading uses machine learning and artificial intelligence in psychometrics to speed up the tedious task of grading essays.

B. Motivation

One of the difficulties of grading essays is the subjectivity, or at least the perceived subjectivity, of the grading process. Many researchers claim that the subjective nature of essay assessment leads to variation in grades awarded by different human assessors, which is perceived by students as a great source of unfairness.

The main motivation is to get unbiased result and also to lessen the teachers’ effort of marking too much essays.

C. Objectives

With the onset of the technological revolution, India has immensely grown in the technological sector and we are trying to make maximum of our tasks online as it is easier to track as well as manage things digitally.

In this technological era, our education institutions have also shifted many of their tasks online which in turn has resulted in increased efficiency of work. The purpose of the Smart Essay Scoring system is, In comparison to human raters, the SES system is able to evaluate a vast number of essays effectively. And to get Reliable and High-Quality Result with unbiased result.

II. LITERATURE SURVEY

As per the literature review the present system of university management is done manually rather than using technology. This has some following pros and cons –

Table 1 – Pros and Cons of Present System

Thus, though the pros and cons weigh out to be equal. The cons are quite critical and can cause troubled situations in longer run.

Thus, the technology we plan to implement will reduce time, human error, the human effort and will make more accurate future predication thus reducing time, effort and cost while implementing the system.

The following table presents our basic findings from each paper which provides a direction to our idea.

Table 2 – Literature Survey

III. SYSTEM ARCHITECTURE

Our main objective via this project is to bring ease in essay evaluation. Our motive is to get reliable and high-quality result. The idea behind our project is to get unbiased results.

Our project is divided into 3 parts-

1. Login Section: The login section is the first page that appears when a user opens our website. The user has to login here to proceed further. If the login procedure failed due to some reason or login verification failed, the process is rolled back.
2. Input Section: After Login verification, the user is directed to next page, i.e., the page where the user gives input. In this section, the user gives the input in two ways. One way is the typed way, where the user types the essay in the text area given. And the second way is where the user drops or uploads the file.
3. Output Section: After the evaluation of the user’s I/P (essay), the result is shown there with spelling mistakes checked, grammatical mistakes, plagiarism checked etc.

IV. PROBLEM SOLUTION AND DATA FLOW

1. The user opens the website. Then, the user logs in.
2. There is a text box on the home page. Then, on that text box, one must write the essay that they wanted to get evaluated or even they can drag and drop image file.
3. After this step, user clicks on the ‘Evaluate’ button.
4. And at last, the user gets the evaluated result with its score that includes checked spellings, grammatical mistakes etc.

V. ALGORITHMS

A list of some of the basic algorithms applied in the project are as follows –

1. LSTM Algorithm: Long short-term memory (LSTM) is an artificial neural network used in the fields of artificial intelligence and deep learning. Unlike standard feedforward neural networks, LSTM has feedback connections. Such a recurrent neural network (RNN) can process not only single data points (such as images), but also entire sequences of data. An RNN using LSTM units can be trained in a supervised fashion on a set of training sequences, using an optimization algorithm like gradient descent combined with backpropagation through time to compute the gradients needed during the optimization process
2. Bidirectional LSTM: In bidirectional, our input flows in two directions, making a bi-lstm different from the regular LSTM. With the regular LSTM, we can make input flow in one direction, either backwards or forward. It involves duplicating the first recurrent layer in the network so that there are now two layers side-by-side, then providing the input sequence as-is as input to the first layer and providing a reversed copy of the input sequence to the second.

VI. SCOPE

The scope of our project is –

1. Student: The students would be able to get their essays evaluated, quickly and in an efficient way. This application will help them to get their mistakes checked like spelling errors, grammatical mistakes etc.
2. Universities/ Educational Organizations: This Website will help universities in reducing manual checking because in manual checking, it become complicated. The system will also give university more uniformity.
3. Future Scope Of Project: This application could be integrated and used by several testing institutions to meet their needs for essay grading. The model used could be trained with an increasing number of input essays to further improve its accuracy. The model could also be trained on giving a score on specific criteria of essay grading such as relevancy, linguistic and reasoning ability of the author. Research could be conducted on making the model faster. This technology could also be extended for use with languages other than the English language, effectively rendering it useful on a worldwide level.

VII. UNIQUE FEATURES

1. Simplicity-We have kept the product as simple as possible by maintain a simple and easy to use UI/UX design to simple and efficient backend algorithms. This not only decreases the server load and runtime but also makes the application usable for all the age groups.
2. Automatic essay scoring can evaluate many essays in an effective way as compared to human raters.
3. The SES system is steadier and more unbiased, which means, results will be accurate without any malpractices or cheating involved.
4. Use of google API: The use of google API is one the unique features here. Because of this feature, the uploaded file by user is processed, so that the essay can be extracted.
5. Plagiarism Check: Plagiarism checks are useful not only for keeping you honest, but they can also help improve your writing by showing where there are still issues with your content. You can see how much of your writing is original and what was copied by using a plagiarism tool.
6. Easy to operate on any operating point. This feature is useful for all the users using different operating systems.
7. Usage of long-short term memory for training of model. LSTMs were developed to deal with the vanishing gradient problem that can be encountered when training traditional RNNs. Relative insensitivity to gap length is an advantage of LSTM over RNNs

VIII. ACKNOWLEDGEMENTS

This project and the research behind it would not have been possible without the exceptional support of our project guide, Prof. Subhash A Nalawade - HOD Information Technology. His enthusiasm, knowledge and attention to detail have been an inspiration and kept our work on track.

We are also grateful for the insightful comments offered by Prof. Sonali Patil – Project Coordinator Information Technology and Dr. L. K. Wadhawa Principal - Dr. DY Patil Institute of Technology for his constant support and motivation. The generosity and expertise of one and all have improved this study in innumerable ways and saved us from many errors.

Conclusion

This project has proposed real time solution for evaluation of the essays in an efficient way. So, we have successfully implemented our project where the user will pass the input and will get the evaluated result with checked spellings, vocabulary, grammatical mistakes etc. We have solved the problem of – 1) Manual evaluation complications. 2) Possibility Of biased results. 3) Assisting to Students preparing for competitive exams like TOEFL, GRE etc. Our solutions were – a) Implementation of integrated smart essay scoring. b) Ease in evaluation of long and difficult essays. c) Spell checks, Grammatical error check, Essay length, Overall scoring.

References

[1] Landauer, T. K., Holtz, P. W, & Laham, D. (1998). Introduction to Latent Semantic Analysis. Discourse Processes, 25, 259–284. [2] Chung, G. K. W. K., & O’Neil, H. F., Jr. (1997). Methodological approaches to online scoring of essays. (ERIC Document Reproduction Service No. ED 4?8 ?0?), 39pp [3] Rudner, L.M. & Liang, T. (2002). Automated essay scoring using Bayes’ theorem. journal of Technology, Learning, and Assessment, ? (2) [4] Burstein, J., Kukich, K., Wolff, S., Lu, C., Chodorow, M., Braden-Harder, L., et al. (?998, August). Automated scoring using a hybrid feature identification technique. Proceedings of the Annual Meeting of the Association of Computational Linguistics, Montreal, Canada. [5] Porter, M.F., 1980, An algorithm for suffix stripping, Program, 14 (3), ?30–?37. Reprinted in Sparck Jones, Karen, and Peter Willet (?997). Readings in Information Retrieval, San Francisco: Morgan Kaufmann. [6] Siddhartha Ghosh, Sameen S Fatima, (2007), A Web Based English to Bengali Text Converter, will be presented in The 3rd Indian International Conference on Artificial Intelligence (IICAI-07), Pune, India, December 17-19, 2007. [7] Valenti, S., Cucchiarelli, A., & Panti, M. (2002). Computer based assessment systems evaluation via the ISO9126 quality model. Journal of Information Technology Education, 1 (3), 157-175. [8] Yigal Attali & Jill Burstein, Automated Essay Scoring with e-rater V.2, The Journal of Technology, Learning and Assessment, Vol 4, no. 3, Feb\'06 [9] Ajay HB, Tillett PI, Page EB (1973) Analysis of essays by computer (AEC-II) (No. 8-0102). Washington, DC: U.S. Department of Health, Education, and Welfare, Office of Education, National Center for Educational Research and Development.

Copyright

Copyright © 2023 Prof. S. A Nalawade, Rishabh Raj Shukla, Prachi Gurjar, Harsh Kumar, Harshvardhan Chavan. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.