Improving College Assistance with the Help of Richer Human Computer Interaction and Speech Recognition

Abstract

Chat bots are intelligent systems that interpret and react to users questions in their native language. In a conversation, the chat bot reacts in the same way as a human would. It functions as a virtual assistant, and its accuracy is assessed by determining a correlation between user questions and chat bot responses. For a better user experience, the implemented Chat bot has two modes: text mode and audio mode. It provides an interactive approach of answering through voice messages when in audio mode. There is a long line at the inquiry window during the Institute\'s Academic Admission procedure. Even more challenging is the situation for parents who live in various cities, states, and nations. The purpose of the system is to give students and parents a place to ask questions and get answers using simple English language text messages or audio commands. Instead of queueing at an information desk to ask questions about the admission process students and parents will collaborate with the bot. The proposed application is easily available to students and responds to user any time.

Introduction

I. INTRODUCTION

Students who opting for engineering after completing their 12th or diploma had a lot of issues prior to enrolling. Students and their parents are concerned about a variety of admissions-related issues. When it comes to choosing a  suitable engineering institution, students are often  perplexed. They choose institutions based on a variety of factors, including prices and the previous year's admissions As a result, some students send me e-mails or telephone me. As a result, there is an unnecessarily large gathering of people waiting to be questioned. The admissions department frequently has trouble answering students' persistent questions. In addition, the college admissions department  required additional. people and funds to respond to all of the inquiries.

The installed chatbot will answer users' questions, supply them with the information they need, increase the quality of service time, and make customers happy by offering clever solutions. It also boosts productivity by offering 24-hour support, reducing wait times at the help desk, and reducing human effort. Students can use their laptops or smartphones to communicate with the chatbot on the web. Students ask a variety of questions on admissions details in plain language, and both can respond with correct replies. Customers can simply access the proposed application, and it responds to users at any moment. In order to improve the quality of service, the chatbot not only responds, but also self-learns and improves.

II. RELATED WORK

Chat bots have gotten a lot of attention in the last decade in practically every online area, including e-commerce platforms, travel sites, government sites, college websites, company websites, online ticket booking sites, and so on. Bayu Setiaji and Ferry Wahyu Wibowo[2] created a chatbot utilizing relational databases and pattern matching. They've also employed spell check normalization after testing spelling to improve accuracy.

Authors Karthick Sowndarajan, R John Victor, and S Manikandan use natural language processing in their article [1] to create an android-based chat application written in Java. After preprocessing, they used stemming to remove terms that appeared at the beginning and end of the sentence. Special characters and numerals are removed during preprocessing. Following stemming, the leftover words are matched with keywords stored in a dictionary.

Dungeon Lee, Kyo-Joong Oh, and Ho-Jin Choi [5] created and built an artificial intelligence-based chatbot (dubbed "college bot") for a college website that answers all of the student's questions. To make it more engaging, the authors used a Facebook-like user interface.

Borah B., Pathak D., Sarmah P., Som B., and Nandi S. conducted a comprehensive study on text-based chat bots. Their main focus is on conversation bots that use artificial intelligence and chat bots that use natural language processing. They've nicely outlined the issues with implementing text-based chatbots for various types of applications.

AM Rahman, Abdullah Al Mamun, and Alma Islam go over the several promising chatbot application domains as well as the general architecture of interactive bots. Their work focuses on a variety of current APIs that can be used to build chat bots.

Chabot, sometimes known as an interactive agent, is a piece of software that communicates with humans through written or audio means. Chat bots are intelligent and developed in such a way that they can act as a human's talking companion. These types of smart bots are used in a variety of sectors, including information collecting and customer service. While some chatbots use NLP (Natural Language Processing), simple chat bot systems scan keywords inside input and respond with a similar pattern or matching term.

For the school system, Guruswami Hiremath created a text-based chatbot [10]. This paper's preprocessing, tokenization, and pattern matching are quite similar to those employed by Bayu Setiaji and Ferry Wahyu Wibowo [2]. For template matching, author Guruswami Hiremath used machine learning and natural language processing (NLP).

However, none of the above-mentioned bots were able to bridge the gap between how a real human would engage and how a bot would answer to the same user queries, resulting in a negative user experience. Natural Language Understanding is a critical feature of chat bots that allows them to recognise a human error during a conversion and attempt to repair it while extracting meaning from the wrong phrases. Thus, it has been sought to leverage these strategies to provide consumers with a nice experience when engaging with bots.

III. ARCHITECTURE

The process of chatbot design determines how a chatbot interacts with its users. The overall interaction, the chatbot personality, and the questions that users will ask are all defined by the chatbot creator. Designers employ dedicated chatbot design tools to speed up the process by allowing team collaboration, immediate preview, and video export. User testing is an essential component in chatbot development.

A. BOT Client

The GUI allows the user to interact with the system. He/She will type the question or doubt in the supplied text field. Following the submission of the enquiry, the request will be handled via the chat interface.

B. Conversational Interfaces

Conversational interfaces are platforms that can converse with each other as if they were real people. Graphical user interfaces (GUI) are commonly used to give orders to computers. After that, the computer reads the command and performs the desired action. In the case of Conversational interfaces, instead of issuing commands or utilizing a GUI, the user can communicate with the computer in their own language. This is made feasible through the use of Natural Language Processing (NLP), which allows the computer to comprehend the meaning of the user's input and perform the task accordingly.

Because of the ambiguous nature of languages, it is difficult for computers to always understand the correct meaning of human input, which is referred to as Natural Language Understanding (NLU).

Conversational interfaces can understand the correct meaning of a query that involves spelling errors, incorrect grammar, and so on using Natural Language Understanding. If a user inquires, "What is the Computer Engineering intake?" for example. When the user queries, "Who is HOD?" the search will continue in the context of Computer Engineering and deliver the proper answer.

C. Service Gateway

1. Dialogue Definition

A Dialog Definition is a computer system that is designed to communicate with a human.

The following are the phases of a dialogue system:

The user asks questions in plain English using the system microphone. The input is transformed to plain text using the Automatic Speech Recognizer (ASR).

The transformed text is analyzed using a natural language understanding unit (NLU).

The semantic information must be analyzed by the dialogue manager. The dialogue manager keeps track of the dialog's history and current state.

The task managers are familiar with a certain task domain that the dialogue manager contacts.

Dialog manager uses an output generator to generate output.

Finally, using an output renderer, the output is generated and shown as a response to the user.

2. Knowledge Extraction

Knowledge extraction is created using a structured relational database or XML and documents.

3. Backend Connectors and Database

MongoDB, a cross-platform document oriented database, is used in the backend. MongoDB is a NoSQL (non-relational) database. It makes use of schemata and JSON-like documents.

4. Representational

State Transfer (REST API) Requesting systems can access and manipulate textual representations of web resources using a uniform and specified set of stateless actions using RESTful web services.

IV. PROPOSED SYSTEM

A. Classification of Questions

User-submitted questions via web application can be classified into two categories: FAQ questions and transactional questions.

Frequently Requested Questions [FAQ] are questions and answers that are frequently asked in a specific environment. BOT is capable of answering these types of inquiries without the need for further inquiry. For instance, where is the college located? The answer to this question is the college's address, and there is no need for further clarification or questions.

Conversational Questions: The Abstract Question Descriptor is used to fill the blanks in conversational questions. If a slot isn't filled, the BOT may ask a fallback question or a clarifying question.

B. Dialog Definition File (DDF)

Global Flags are used to define Dialog behaviour in the Dialog Definition File (DDF). It consists of many tasks that BOT is required to complete. Some tasks are set aside and used just for specific objectives.

1. Current Task is Being Cancelled: The user has stated that he or she wishes to cancel the task.
2. Help Task: Assisting the user if there are multiple failed attempts at answering the ITO.
3. Task to Complete Before Leaving: This task will acquire final confirmation from the user before logout if the user says (bye, exit, or bye bye).

C. Flags to Define Dialog Definition

1. startTaskName - This specifies the first task that BOT will perform. The initial task may not be "start," but after it is completed, BOT will return to the start task.
2. The BOT language is specified by globalLanguage. It must be written in ISO language code.
3. SODA (System Of Dialogue Act) — Each utterance will be classified using the Max Entropy Classifier to determine whether the user is seeking information, providing information, or issuing a defined command (e.g. switch ON, Switch OFF)
4. allowSwitchTasks–if true, sub-dialogues from other tasks will be allowed; if false, only sub-dialogues from the same task will be executed. (For example, in the trip.xmlDDF use case, the user can only answer questions related to getTripInformation and cannot go to the getWeatherInformationtask.)
5. allowOverAnswering- if true, the user is permitted to supply more information than that requested (but at least the current question).
6. allowCorrection-if true, the user can amend the answer to a previously asked question.useIntentEngine–if true, the intent will be determined using an internal intent engine that has been trained separately (see appendix -3), otherwise BOW (Bag Of Words) will be used as stated in the selector> section of the dialogue definition file.
7. failureAttempts–is an integer number that represents the number of attempts a user will be allowed to answer a specific ITO in a row.
8. 
   If he or she does not respond, he or she will be referred to "helpTask."

D. DDF-Task Definition

As an intent, a task definition includes a unique task name. After removing stopwords, the purpose is mapped to ll potential utterances using a bag of words model.

The positions in Information Transfer Entities (ITOs) are filled based on AQD (Abstract Question Descriptor). If the slot is not filled and the needed flag is true, the BOT may ask the user a backup question. If the user does not supply the proper answer, the clarify question is used.

E. DDF-Action

The REST API is supported by DDF-action. The API result will always be saved in the result variable, which can be accessed using #result

V. USER INTERFACE

The process of building user interfaces for BOT in order to increase user usability and experience is known as user interface design (UI). The goal of user interface design is to make interaction with users as simple and efficient as possible.

BOT can be interacted with and questions can be asked in three distinct ways.

A. Keyboard Interface

By using the keyboard to enter text, the user can ask questions. To submit a query to BOT, the user types their query into the supplied textbox and clicks the submit button.

B. Card Interface

The card interface allows the user to provide input to the BOT by selecting values from the cards. The card interface allows the user to select a specific card and submit a question.

C. Speech Recognition

BOT can distinguish words and sentences in spoken language and convert them to a machine-readable representation thanks to speech recognition. When a user speaks, speech recognition is used.

Gives input through microphone of system, such as voice commands (e.g. "What is address of college") as shown in figure 4. The bot reply query by providing address of location.

Figure 5 is a snippet of a query of a user enquiring about courses and intake. In General a variety of course are offered by a college so bot provides card interface to user to avoid the ambiguity of fees of different courses. After that user can provide extra inputs by selected course type and department from dropdown option to or by typing message to bot. This is a transactional type of question where bot require more information to reply back to the user. Therefore bot provide a card interface for more user details. So Chabot is offering to choose a type of course

D. Voice Commands

1. Manual Mode: In manual mode, the user must engage with the BOT for each voice command. The user will issue a command, and the BOT will respond and then stop. If the user wishes to ask a new inquiry, he must push the voice command button once more.
2. Auto Mode: Auto mode looks and feels like a real dialogue interface. When auto mode is on user can have a back-to- back conversation with BOT like he is speaking  to a    real human being  . The Bot Uses text-to-speech to answer the question of the users query.

3. Windows

4. Python 3.3 + (required)

5. Flask module

VI. HARDWARE AND SOFTWARE REQUIREMENTS

Below steps describe the deployment of Chat BOT on windows machine.

Required configuration of Windows machine:

1. Laptop/PC
2. Processor – Intel Core i3
3. RAM 8GB Min
4. 64 bit processor
5. Speaker
6. 512 GB SSD / 1 TB HDD

Following software are required for implementation of chat bot:

a. My SQL server

b. ChatterBot

To test the chatbot's accuracy, the identical query was rephrased with alternative wordings and the addition of various exceptional characters and irrelevant terms.

VII. FUTURE SCOPE

Chat bots are a constantly growing area of study in the field of computer science. If there are any flaws in input owing to human spoken language, such as a grammatical or context issue, bots may still be unable to understand what is being asked.

The usage of powerful Natural Language Processing [NLP] techniques can improve misinterpretation of any commands, which may be due to misspelling. The BOT can be improved till it passes the Turing test.

The accuracy of BOT in detecting a specific query for various settings in which the user is asking can be increased, giving the user a more genuine feeling of speaking with a human counsellor. This bot can be created in a local language and multilingual at a later time

Conclusion

Any college or university can deploy a chat bot on their website to allow external stakeholders to ask questions at any time. And it would be quite beneficial to ease the admission process, as this chatbot can answer questions about state-by-state cutoffs, categories-by-category cutoffs, gender-by-gender cutoffs, and shift-by-shift cutoffs, among other things. Users can give feedback by tapping the like and dislike buttons in response to the responses to their queries. This information is saved in the backend and may be examined by the administrator to analyze the types of queries asked and, if necessary, enhance the answer As shown in figure 6, user is trying to interact with bot with speech command mode so voice option is highlighted in output screen shot. User will ask question and speech recognition model of Chabot will try to extract information via audio received from users.

References

[1] Bayu Setiaji ; Ferry Wahyu Wibowo “ Chatbot Using a Knowledge in Database: Human-to-Machine Conversation Modeling.” In IEEE 2016 7th International Conference on Intelligent Systems, Modelling and Simulation (ISMS) [2] DOI: 10.1109/ISMS.2016.53 [3] Naz Albayrak ;” An overview of artificial intelligence based chatbots and an example chatbot application.” In IEEE 2018 26th Signal Processing and Communications Applications Conference (SIU),DOI: 10.1109/SIU.2018.8404430 [4] Chaitrali S. Kulkarni, et.al. “ BANK CHAT BOT – An Intelligent Assistant System Using NLP and Machine Learning.” In International esearch Journal of Engineering and Technlgy (IRJET) e:ISSN:2395-0056 [5] Dungeon Lee, Kyo-Joong Oh, Ho-Jin Choi. “The ChatBot Feels You – “ A Counseling Service Using Emotional Response Generation.” In 2017 IEEE International Conference on Big Data and Smart Computing (BigComp), DOI: 10.1109/BIGCOMP.2017.7881752 [6] A. Argal, S. Gupta, A. Modi, P. Pandey, S. Shim and C. Choo, \"Intelligent travel chatbot for predictive recommendation in echo platform,\" 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, 2018, pp. 176-183. [7] Borah B., Pathak D., Sarmah P., Som B., Nandi S. (2019) Survey of Textbased Chatbot in Perspective of Recent Technologies. In: Mandal J., Mukhopadhyay S., Dutta P., Dasgupta K. (eds) Computational Intelligence, Communications, and Business Analytics. CICBA 2018. Communications in Computer and Information Science, vol 1031. Springer, Singapore [8] A M Rahman ; Abdullah Al Mamun ,et.al,” Programming challenges of chatbot: Current and future prospective.” 2017 IEEE Region 10 Humanitarian Technology Conference (R10 -HTC) [9] Ashay Argal ,et.al “ Intelligent travel chatbot for predictive recommendation in echo platform.” In 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC) [10] Guruswami Hiremath, et.al,” Chatbot for education system.” In International Journal of Advance Research, Ideas and Innovations in Technology, ISSN: 2454-132X

Copyright

Copyright © 2022 Anshu Varma, Vishakha Upadhye , Sayli Hulji, Dr. Shreya N Patankar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.