Ijraset Journal For Research in Applied Science and Engineering Technology
Authors: Dhruv Patel, NIhal Shetty, Paarth Kapasi, Ishaan Kangriwala
DOI Link: https://doi.org/10.22214/ijraset.2023.51324
Certificate: View Certificate
Chatbots are computer programs that use text or voice-based interfaces to replicate human conversation. They are often used to automate mundane processes, provide customer support, or aid in the retrieval of information. Chatbots are built with a number of strategies that enable them to interpret and respond to user inputs in a more human-like manner. They can be employed in a variety of industries, including e-commerce, healthcare, and banking. We have analysed and compared numerous chatbot strategies in this report to establish the optimal way for our own chatbot project. We reviewed twenty-six papers on chatbot development and assessed the advantages and disadvantages of various strategies. Natural language processing techniques, such as tokenization and named entity recognition, have been shown in our research to be critical for interpreting user inputs. We also discovered that dialogue management methods, such as rule-based and machine learning-based approaches, have an important influence in influencing discussion flow. Furthermore, we discovered that natural language generation techniques, such as template-based and neural network-based methods, are critical in generating effective chatbot responses. We also investigated various services on the market in order to create a functional chatbot for our college. We also emphasized the various applications of chatbots as well as the current hurdles in the industry. Based on these findings, we chose a technique for our own chatbot project that employs advanced natural language processing and machine learning techniques to create more human-like conversations and improve overall user experience.
I. INTRODUCTION
A. Background on Chatbots
Chatbots are computer programmes that replicate human-to-human interaction, typically over the Internet. They were first developed in the 1960s and have evolved greatly since then due to technological improvements and growing Internet usage. Chatbots have become an essential element of the customer care sector, thanks to the rise of messaging apps and the growing popularity of virtual assistants. They are capable of handling a wide range of responsibilities, including answering frequently asked inquiries, assisting customers with transactions, and offering support. They also serve an important role in automating tasks that are monotonous and time-intensive, allowing human agents to focus on more complicated concerns. Chatbots are now integrated into a variety of platforms and can be accessed via websites, smartphone apps, and messaging apps.
B. Purpose Of The Research Paper
The goal of this study is to look into the viability and usefulness of utilising a chatbot on the college (NMIMS MPSTME, Mumbai) website with Microsoft Azure. The primary goal of the project is to create, test, and improve a chatbot that will provide students and other stakeholders of the college with immediate access to information regarding the college and its resources. The chatbot will be programmed to interpret and react to users' natural language queries in a conversational manner. The chatbot will be pitted against two other types of chatbots: rule-based chatbots and Rasa NLU chatbots. The rule-based chatbot responds to user queries using a set of pre-defined rules, whereas the Rasa NLU chatbot understands and responds to user inputs using machine learning techniques. The comparison of the aforementioned chatbots will provide insight into the merits and shortcomings of each method, and will help to shape the construction of the most effective chatbot for the college website.
The study's findings will add to the increasing body of information about chatbots and their applicability in a variety of fields, including education. The research findings will help the college make educated decisions about using chatbots and similar technology to improve its services and communication with students and other stakeholders.
II. CHATBOT SERVICES IN THE CURRENT MARKET
In the current market, there are various distinct types of chatbot services, each with its own set of features and capabilities. Among the most prominent chatbot services are:
To summarize, there are several chatbot services accessible on the market today to meet diverse corporate goals and requirements. Some services are platform-based, while others are cloud-based, AI-based, customized, or sector-specific.
III. LITERATURE REVIEW
A. Overview Of College Inquiry Chatbots
College enquiry chatbots, which are often linked into the college's website or accessible via messaging apps, are intended to provide quick and convenient answers to common questions. College inquiries chatbots can answer a wide range of questions, such as entrance requirements, campus facilities, and educational offerings. They also provide helpful perspectives about student life and college culture, assisting prospective students in making educated educational decisions. College enquiry chatbots can dramatically improve the overall experience of potential students and help institutions better manage their outreach efforts by automating redundant and laborious chores.
According to Hrushikesh Koundinya K, Vaishnavi Putnala, Ajay Krishna Palakurthi, and Dr. Ashok Kumar K [1,] a chatbot for college management system that uses Artificial Intelligence (AI) algorithms can provide accurate answers, and users won't have to go to the college or its official website for their queries. Suyash Awasthi, Anupriya Purwar, Dhananjay Batra, and Prof. Prakash Devale [2] created the SDA bot, which has reduced the time it takes to search, navigate, and scroll on webpages. Their bot not only assisted end-users in obtaining necessary college information, but it also quickly resolved queries. A chatbot, as opposed to a shift-based personal assistance team, will additionally offer continuous support and availability. Because of computational limitations, the amount of tasks related to data processing that have to be finished in a brief amount of time, and an unintegrated information system, management departments are entering the same data multiple times across multiple distinct databases in which they own all students' data to be accessed inevitably, which increases the need for a Chatbot application, according to Herry Derajad Wijaya, Wawan Gunawan, Reza Avrizal, and Sutan M Arif [3].
As stated by Walaa Hassan, Shereen elBohy, Mina Rafik, Ahmed Ashraf, Sherif Gorgui, Michael Emil, and KarimAli [4], users expect the chatbot to think and respond like a human and to perform on par, if not better, than a human assistance team, so AI in chatbot development is required. Bots created with the generative model, according to Khang Nhut Lam, Nam Nhat Le, and Jugal Kalita [5,] are capable of responding to questions that aren't present in the dataset used for training, but the responses may be in incorrect syntax or misspelling; while bots developed with the retrieval-based model are limited to responding to queries in the training dataset with correct spelling and grammar.
B. Overview of Azure
Microsoft Azure [6] is a cloud computing system that offers an extensive collection of tools and services for developing, deploying, and operating chatbots. It has a variety of features tailored exclusively for chatbot creation, including as natural language processing (NLP) and dialogue management [7], which makes it a suitable platform for developing conversational AI applications. Azure also provides a number of pre-built templates and connectors [8] with other services such as Microsoft Power Virtual Agents, allowing developers to quickly create sophisticated chatbots.
Furthermore, Azure provides scalability, security, and reliability, ensuring that chatbots can handle increased traffic while maintaining consistent performance [9].
Because of its powerful and flexible capabilities, Azure has become a popular choice for chatbot development among businesses and organisations looking to improve their customer engagement and customer service operations.
C. Comparison Of Azure, Rule-Based And Rasa Chatbots
Microsoft Azure is a cloud-based platform that offers a variety of tools and services for developing, deploying, and operating chatbots. Azure creates chatbots with sophisticated conversation skills by combining NLP and dialogue management. In their comparative study of chatbot techniques, Amit Patil, K Marimuthu, Nagaraja Rao A, and R Niranchana state, [10] "Azure Bot Services provide an integrated environment in which we can deploy, build, test, and connect with different channels that interact naturally wherever your users are interacting."
Rule-based chatbots are chatbots which respond to specific inputs using predetermined rules. These chatbots have poor natural language understanding and can only react to particular, pre-programmed questions. They are, nevertheless, relatively simple to construct and can handle rudimentary interactions. Rule-based chatbots, according to Fikadu Wayesa [11], are intended for simple queries and could fail when posed more complicated topics since they cannot produce their own responses.
Rasa chatbots are constructed with the open-source Rasa framework, that emphasizes natural language processing and dialogue management. Rasa chatbots are extremely adaptable, allowing developers to design chatbots with extensive conversational skills, such as understanding and responding to difficult queries. Yurio Windiatmoko, Ahmad Fathan Hidayatullah, and Ridho Rahmadi created a chatbot utilising the Rasa Framework [12] that provided the greatest efficiency in terms of intent and entity classes, as well as appropriate responses from the chatbot model's dialogue policy.
The decision regarding these chatbots will be determined by the project's specific requirements and the required level of sophistication.
D. Existing Chatbot Technology
We also examined the existing Chatbot on the college's website. It is a rudimentary, rule-based chatbot that cannot engage in conversation with the user. It just offers links to the college website, offering little to no assistance to the consumer. It has relatively few features, as shown in the screenshots below:
E. The examination of Chatbot Technologies
We have studied the implementations and outcomes of chatbot models presented in sixteen papers from 2019 to the present. A number of ways were used; some used well-known procedures, while others devised unique strategies to accomplish their goals. We have summarised their work and given its features and drawbacks in the sections below. Table I. W. Mahanan, J. Thanyaphongphat, S. Sawadsitang, and S. Sangamuang [1] used ML and NLP foundations to build a Thai-language collegiate chatbot. To improve its performance, the chatbot must learn from its users. The suggested chatbot was created specifically to answer questions about the Digital Industry Integration (DII) curriculum at Chiang Mai University in Thailand.
According to M. M. Khan [2], the most difficult challenge in the field of natural language processing (NLP) and natural language understanding (NLU) right now is to create a chatbot that can provide an experience that imitates real human sales agents, bringing together two sectors, e-commerce sales and AI, to create an immersive model that can perform synergy. To enhance customer help and increase sales, an ecommerce sales chatbot was created. For natural language understanding, the system employs machine learning.
A. Ait-Mlouk and L. Jiang [3] created a chatbot (KBot) which solves some of their issues and can compete with existing linked data chatbots in terms of performance. Furthermore, they process one of the largest study databases in social science (the myPersonality corpus6), which has been gathered from over 6 million Facebook volunteers. The data was anonymised and sampled for distribution to registered scholars worldwide.
The Xatkit framework, developed by G. Daniel, J. Cabot, L. Deruelle, and M. Derras[4], tackles those issues by offering a set of Domain Specific Languages (DSL) for defining chatbots (and other types of bots such as voicebots and bots in general) in a platform-independent manner. Xatkit also includes a runtime engine that can seamlessly deploy the application and handle the required conversation logic across the platforms. Xatkit is completely open source, which means that all of its features are accessible online.
According to B. Chempavathy, S. N. Prabhu, D. R. Varshitha, Vinita, and Y. Lokeswari [5], the main purpose of developing their proposed Chabot is to deal with academic activities such as admission inquiry, scholarship details, fee details, details of the documents necessary to attach to the timetable of each department, and so on, as the system allows the user to clear up their queries in less time and effort.
Ranoliya, Bhavika, Raghuwanshi, Nidhi, and Sanjay Singh [6] Based on a collection of commonly requested questions, AIML and Latent Semantic Analysis were utilised to create a chatbot that provides an efficient and right response to every enquiry. (LSA). Any university can use this chatbot to react interactively to frequently requested queries from students.
S. Keyner, V. Savenkov, and S. Vakulenko [8] expect open data access to increase public awareness and citizen participation. Their chatbot is a piece of software that provides a conversational interface.
Shingte, Kshitija, Chaudari, Anuja, Patil, Aditee, Chaudhari, Anushree, and Desai, Sharmistha [9] propose a system that aims to reduce the work of the admission process department by providing the required information to the students or parents, as well as the department's workload by attempting to answer all of the students' queries.
Gawade and Harshala [10] want to build an AI-powered virtual assistant that can answer any college-related question. This will function as a machine for College Oriented Intelligence. This virtual machine will reply to student inquiries about college-related topics.
V. Prathyusha, G. L. Sri, G. Meenakshi, and Y. K. Chakravarti [11] offer a system that uses in-built AI with an effective Graphical User Interface (GUI) tool to react properly to user inquiries. The chatbot analyses the user's questions using a cognitive service built by Microsoft called Language Understanding Intelligent System (LUIS). It is integrated with the Skype application, which can be downloaded and installed on the user's smartphone via the Google Play store.
A college inquiry chatbot project is proposed by R. Parkar, Y. Payare, K. Mithari, J. Nambiar, and J. Gupta [12]. The project handles the user's queries in question-based format to deliver the requested response as a message. It eliminates the time-consuming strategy of visiting colleges and gathering required material to meet the requirements by giving assistance 24 hours a day, seven days a week.
Daswani, Mohinish, Desai, Kavina, Patel, Mili, Vani, Reeya, and Eirinaki, Magdalini [13] created CollegeBot, a conversational AI agent that uses natural language processing and machine learning to help visitors of a university's website find information related to their queries. Multiple ways for domain-specific and non-domain-specific query resolution were offered for the system. The authors also build a proof-of-concept prototype of the suggested system to show how colleges can use it.
Shanmugam, Dr Raju, Jena, Soumya, and Gaur, Vishvaketan [14] focus on the process of web communication automation and generate the relevant answer to the user inquiry utilising AIML and LSA (Latent Semantic Analysis). If the relevant answer to the user inquiry is not found, the system will ping the administrator.
K, Ashok, and Ajay Krishna Palakurthi [15] look into how advances in artificial intelligence-based conversation technology can be used to improve a variety of services, notably the development of chatbots as a medium for information exchange. They create and deploy a chatbot using AIML, which is an XML format. The WordNet method is used by the developers to discover keywords and context triggers. A database is used to identify and display replies to similar requests.
Sounding board, an NLU/NLP-based conversational agent, is designed, implemented, and evaluated by Hao Fang, Hao Cheng, Maarten Sap, Elizabeth Clark, Ari Holtzman, Yejin Choi, Noah A. Smith, and Mari Ostendorf [16]. It demonstrates that it is possible to create an agent capable of long-term communication when supported by rich information and expertise. This bot also uses speech synthesis markup language for voice recognition. (SSML). They can also deduce personality attributes based on the length of the chat.
Table I
Literature Review
Paper Name |
Methods |
Inference |
Weaknesses |
College Agent: The Machine Learning Chatbot for College Tasks (2022) [1] |
NLP, AIML |
Most of the basic questions about the DII project are answered correctly, as shown as a "correct" tag in the paper. |
The training size of the dataset is small, which could've hindered the performance. |
Development of an e-commerce Sales Chatbot (2022) [2] |
NLP, NLU, SVM |
It is a single page application which serves all the front-end login and the views, to provide a smooth experience. |
There was no measurement of accuracy, or any details about the dataset mentioned on the paper, which is why the model may only sound good on paper. |
KBot: A Knowledge Graph Based Chatbot for Natural Language Understanding Over Linked Data (2020) [3] |
NLP, NLU, NER, SVM |
Usability analysis shows that the proposed KBot has improved the end-to-end user experience. Kbot performs better than some of the online available state-of-the-art chatbots compared in this paper |
The F1-scores used as evaluation metrics for some of the classes were quite low. |
Xatkit: A Multimodal Low-Code Chatbot Development Framework (2020) [4] |
XatkitML, NLP |
The Xatkit Modeling Language packages decouple the different dimensions of a chatbot definition, facilitating the reuse of each dimension across several chatbots. |
Security and Access-Control is another important aspect as users must be able to query (or not) certain aspects of data depending on their profile." |
AI based Chatbots using Deep Neural Networks in Education (2022) [5] |
NLP, LSTM, NLTK, Naïve Bayes, SVM |
The proposed model is helpful for students with their queries regarding academics, placements, sports, scholarships, or the college itself. Not only the students, but parents and staff can also present their queries here. |
The model is trained on a dataset consisting of question-answer pairs regarding college inquiries taken from an open-source website which is not enough data for the model. |
Chatbot for university related FAQs (2017) [6] |
NLU, NLP |
This paper presents the chatbot for the educational sector, where they can ask queries regarding college admission, about college information and other things related to academics. |
"Lack of inclusion and changing patterns and templates for general client queries using AIML and right response are given often utilizing LSA." |
A Literature Survey of Recent Advances in Chatbots (2022) [7] |
AIML, NLP |
The proposed methodology helped analyze the state-of-the-art language models, applications, datasets used, and evaluation frameworks as well as underlining current challenges and limitations, as well as gaps in the literature. |
AI chatbots are still unable of simulating human speech, despite technical advances. This is owing to a flawed approach to conversation modelling and a lack of accessible domain-specific data. In addition, Information Retrieval chatbots lack a learned AI model. |
Open Data Chatbot (2019) [8] |
AIML, Naïve Bayes |
The given design approach is not confined to the use case in question and may be applied to other domains, such as cultural heritage or e-commerce. |
Conversational data must be labeled in order for machine learning techniques to be used to train dialogue models. |
Research Paper on Chatbot Development for Educational Institute (2021) [9] |
Tensorflow, NLTK |
The AI based Chatbot system can be used by colleges and businesses. General user queries such as, questions related to the Enquiry process, course details, eligibility criteria description and Admission are answered using the chatbot. |
It does not include speech-based questions and responses for people who cannot read and type their queries. |
College Enquiry Chatbot (2020) [10] |
IR, NER, LSTM, CNN |
The result can be displayed in the form of images and card format or in text format. |
The final system outlook is missing and hence it becomes difficult to evaluate the success of the same. |
Information Acquisition Chatbot System using LUIS (2021) [11] |
KNN, NLP |
The system provides round-the-clock services. The output is provided in a dynamic format. |
In spite of having an exceptional design architecture, the researchers have failed to either detail or develop other functionalities in the chatbot. |
AI And Web-Based Interactive College Enquiry (2021) [12] |
LSTM, RNN, NLU, NLP |
The chatbot offers suggestions that assist the user to ask the right questions. The FAQ button allows users to ask a commonly asked question. Identification of intent in terms of the sentence is required to take confidence rate into account. |
The system is also tested only for a fixed set of hyper-parameters like learning rate, batch size and no. of epochs, which limits the possibility of achieving better predictions by tweaking and testing the said hyper-parameters. |
CollegeBot: A Conversational AI Approach to Help Students Navigate College (2020) [13] |
NLP, NER |
The researchers have practically implemented the proposed system and verified the accuracy of the model. The proposed bot can maintain context throughout a user session. |
The implemented system with the semantic similarity model is slower than the seq2seq model. |
College Chatbot using Natural Language Processing for Student Queries (2020) [14] |
NLP, AIML, LSA |
The paper provides a substantial amount of flow and data diagrams to give a good understanding of the system. |
The paper also provides evidence of practical implementation of the system. There was no measurement of accuracy of the model mentioned on the paper, hence the effectiveness of the chatbot cannot be properly evaluated. |
Smart College Chatbot Using ML and Python (2020) [15] |
Wordnet, NLTK, AIML |
As the system is trained using machine learning techniques, it can undergo rigorous training to improve accuracy substantially. |
As the chatbot is implemented using AIML XML, it has its inherent drawbacks. |
Sounding Board: A User-Centric and Content-Driven Social Chatbot (2018) [16] |
AWS Lambda, NLU, NLP, NLG, DM |
The sounding board system is one of the few systems we reviewed that is practically implemented, tested, and rated by professionals. It won the Amazon Alexa prize. |
The paper gives minimal information about the algorithms used in the system, although they are named and described, their detailed explanation and working is missing. |
IV. METHODOLOGY
A. Description Of The Development Of The College Inquiry Chatbot Using Azure
The project, titled MPSTME bot after the college for whom it was developed, aims to assist students, teachers, and prospective workers, as well as students and their families, with quick answers to their inquiries via the college website. The MPSTME Bot must be able to engage in friendly conversations, respond with course and faculty information, provide a link to the academic calendar, answer frequently asked questions, determine the student's attendance based on database information, and provide department timings, addresses, contacts, and event information. The chatbot will be built using Microsoft Azure bot service and Microsoft cognitive services such as Text Analytics, LUIS, and QnAMaker.
The vast majority of present-day chatbots do not have empathy and are incapable of adapting to situations that are not scripted. To address these difficulties, the MPSTME Bot augments existing chatbot deployment with sentiment analysis and active learning. Although successfully classifying the user's question as good, negative, or neutral, the algorithm was not entirely successful in providing the chatbot with empathy. Because the algorithm necessitates more severe training data in order to handle all off-script questions. Active learning, on the other hand, enhances chatbot performance for such queries since it accurately interprets the user's requests, asks clarifying questions, and then retrains the system to produce the desired response.
B. Description Of The Rule-Based And Rasa Chatbots Used For Comparison
In this paper, a rule-based chatbot will be created and compared with the other two chatbots created using Microsoft Azure or the Rasa framework. The goal of this comparison is to highlight how these more complex strategies outperform rule-based chatbots. This research promises to provide knowledge about the optimal technique for designing chatbots that deliver nuanced and personalised interactions by emphasising the drawbacks of rule-based chatbots.
Similarly, a Rasa Framework-based chatbot has been implemented. The Rasa chatbot will offer correct responses, but the steep learning curve, tedious development, and unintegrated system show the need for a different solution, in this case, a Microsoft Azure-based Chatbot.
C. Comparison Criteria
An assortment of evaluation criteria is required to be utilised to measure the performance and effectiveness of different chatbot models when comparing them. Our project's evaluation criteria for comparing chatbots comprise of accuracy, natural language understanding (NLU), personalisation, scalability, ease of use, and security. When comparing different chatbot models, it is important to evaluate a variety of parameters such as accuracy, natural language understanding, personalisation, scalability, ease of use, and security. It is feasible to compare chatbots in a systematic and objective manner using a set of well-defined assessment criteria, and to establish the optimal technique for constructing effective and user-friendly chatbots.
V. PROPOSED SYSTEM DESIGN
Based on the review of literature, we determined the scope of our project, decided that a Conversational AI chatbot would be implemented for the project to provide a more genuine experience to the userbase (students, teachers, and prospective students), and proposed a system design for the MPSTME Enquiry Chatbot that included the integration of multiple technologies. This method makes use of powerful Natural Language Processing and Machine Learning methods in order to create more human-like interactions and enhance the overall user experience.
The goal is to create a chatbot capable of understanding and responding to a wide range of user inputs while also providing precise and pertinent responses. The solution will employ ASP.NET Core for the back end and C# for API development. The front-end of the project will be built using HTML and CSS. Microsoft Azure offers Cognitive Service, Bot Service, and Web Apps. For Natural Language Processing, cognitive services include LUIS and QnAMaker. Language Understanding (LUIS) is a technique for defining and distinguishing different types of intent. In addition, the entities in the user query will be extracted using pre-defined criteria. To train the model, LUIS requires a variety of utterances to be supplied into it. After training and testing the model with these utterances, it may be released for usage in the chatbot.
To design the bot processes for each recognised purpose, we use the Microsoft Bot Framework Composer. Bot Framework Composer is an interactive editing canvas for creating chatbots. To build the bot application on Azure, the Azure Bot Service is used. Web Apps are also utilised to host the program on Microsoft Azure. To store and retrieve data, the system will use Microsoft SQL Server and a Relational Database Management System (RDBMS).
The MPSTME Conversational AI Platform will be divided into features by our development method. Basic functionalities will be built and deployed utilising Microsoft Azure services, with more capabilities gradually added over time. Weekly sprints are used to develop and test these features.
Following the completion of a sprint, the developed features will be tested and bug checked. Following this check, the Project Mentor will be presented with a channel build for approval. Following acceptance, the produced feature will be integrated with the master build in the project repository before being deployed via Microsoft Azure services.
The MPSTME Enquiry Chatbot's system architecture incorporates many technologies. This system employs ASP.NET Core for the back end and API development, as well as HTML and CSS for the front end. Microsoft Azure offers Cognitive Service, Bot Service, and Web Apps.
For Natural Language Processing, cognitive services include LUIS and QnAMaker. To build the bot application on Azure, the Azure Bot Service is used. Web Apps are also utilised to host the application on Microsoft Azure. The system makes use of Microsoft SQL Server, a relational database management system, to store and retrieve data.
VI. IMPLEMENTATION
Microsoft Azure's LUIS was used to generate the intents. LUIS aids in the identification of user intents by analysing user input and categorising it. Based on the project's requirements, various intents such as Department Information, Faculty, Contact, Attendance, and others were established. These intents were built using the LUIS web interface and trained on sample data to increase the chatbot's accuracy. A number of features were added to make the chatbot more interactive and engaging. Among these features are:
Course Information – Users can request information about the various courses provided by the college.
Attendance – Registered students can review their attendance information.
Faculty Information – Provides information about the many faculty members who teach at MPSTME.
Department Information – Users can inquire about various departments within the college.
FAQs – The chatbot may respond to frequently asked inquiries about the college.
To provide a visually beautiful and engaging user interface for the chatbot, Adaptive Cards were employed. Adaptive Cards are a novel approach to exchange information in a conversational setting; they can display rich, user-friendly controls within the dialogue. These were used in the project to display information on Faculties, Departments, Attendance, and so on. The cards also have the ability to apply for courses and scholarships. The chatbot was linked to the college website via API calls. The API calls were done utilising the Microsoft Azure API Management service, which offers APIs with a safe and scalable environment. A mock frontend was created in HTML and CSS to demonstrate the functionality of the College Enquiry Chatbot. The frontend was coupled with the chatbot using Microsoft Azure's Bot Framework Web Chat component. The dummy frontend provides an easy-to-use interface for interacting with the chatbot.
VII. COMPARATIVE ANALYSIS
Table II compares three chatbots created to help students with college-related questions at the NMIMS MPSTME college in Mumbai. The table contains a set of inquiries related to department information, faculty information, contact information, a broad overview, logged-in students' attendance information, and placement information. The table has three columns that represent each of the chatbots, and each column has a checkmark or a cross to indicate if the chatbot can effectively react to the queries or not.
A comparison of the chatbots demonstrates that the Rule-based bot properly answered the majority of inquiries about department information, faculty information, contact information, and general descriptions. However, it was unable to provide reliable responses to attendance and placement questions. The RASA-NLU chatbot, on the other hand, was able to answer most inquiries about attendance and placement but struggled to answer questions about general description and faculty information. The Azure chatbot was the most well-rounded of the three chatbots, as it answered the majority of questions correctly across all categories. However, it battled with a few issues affecting placement and attendance data.
Overall, the comparison study indicates each chatbot's strengths and limitations, which can be utilised to impact future development and advancement of chatbots for college question help. Furthermore, it emphasises the importance of building chatbots that can deliver accurate and comprehensive information across all categories in order to create a great user experience.
VIII. RESULTS AND ANALYSIS
In this study, we used Microsoft Azure's Language Understanding Intelligent Service to build a chatbot for MPSTME College Mumbai. (LUIS). The chatbot was created to answer queries about the college's departments, staff, courses, infrastructure, placements, fees, and scholarships. The chatbot reached an astounding 95% accuracy, making it a very dependable tool for providing correct and timely solutions to student concerns. We additionally implemented two more chatbots for comparison purposes: an elementary rule-based chatbot and a RASA-NLU based chatbot. The simple rule-based chatbot was 65% accurate, whereas the RASA-NLU-based chatbot was 70% accurate.
The durability of the LUIS platform, which is built to handle complicated natural language queries, can be linked to the outstanding precision of the Azure-based chatbot. The chatbot's training data was carefully handpicked to contain an extensive variety of queries commonly asked by students. The chatbot was also linked to the college's database, allowing it to present students with correct and up-to-date information.
In contrast, the basic rule-based chatbot depended on a set of established rules to answer queries. While this strategy can be useful for simple inquiries, it fails to answer more complicated questions that demand a better grasp of natural language. The RASA-NLU-based chatbot outperformed the rule-based chatbot but fell short of the accuracy of the Azure-based chatbot.
The Azure-based chatbot's high accuracy illustrates the promise of deploying powerful AI-based chatbot systems to provide speedy and precise responses to student enquiries. To achieve high accuracy in chatbot systems, this study emphasises the need of using complex natural language processing algorithms and training data curation.
This research report outlines the development and execution of a virtual assistant at Mumbai\'s MPSTME college. The chatbot was built with Azure\'s LUIS and achieved a 95% accuracy rate, rendering it a highly dependable and efficient tool for college students, teachers, and other stakeholders looking for information. We also created an elementary rule-based chatbot and a RASA-NLU chatbot for comparison reasons, with accuracies of 65% and 70%, respectively. The comparison shows that the Azure-based chatbot is not just more accurate, but also more successful at processing complex requests and providing relevant information. Even if the current chatbot implementation has attained a high degree of accuracy, there are still areas for improvement and growth. Here are some suggestions for future work: 1) Transitioning from LUIS to CLU: While LUIS is a valuable tool for designing chatbots, its scalability and customisation are limited. Moving to Microsoft\'s Cognitive Language Understanding (CLU) could give the chatbot more development and expansion options. 2) Adding more features: The current chatbot provides basic college-related information. It can, however, be modified to include features such as event reminders, personalised notifications, and course recommendations. 3) Database expansion: The accuracy of the chatbot is dependent on the data it has access to. By expanding its database, the chatbot will eventually be able to provide more thorough and accurate information. 4) Integrating the virtual assistant with the official website of the college: By connecting the chatbot to the college\'s official website, you can provide a more streamlined and effective user experience. Students and employees can access the chatbot right on the website, avoiding the need to navigate to other pages. The implementation of an Azure-based chatbot for the MPSTME college in Mumbai has resulted in good precision and efficacy. There is still opportunity for improvement and expansion, and we hope that our future work ideas will stimulate further growth and adoption of chatbot technology in educational institutions.
[1] K, Ashok. (2020). Smart College Chatbot using ML and Python. 10.1109/ICSCAN49426.2020.9262426. [2] Awasthi, Suyash. (2021). Web Based College Chatbot - SDABot. International Journal for Research in Applied Science and Engineering Technology. 9. 853-856. 10.22214/ijraset.2021.37505. [3] Wijaya, Herry & Gunawan, Wawan & Avrizal, Reza & Arif, Sutan. (2020). DESIGNING CHATBOT FOR COLLEGE INFORMATION MANAGEMENT. IJISCS (International Journal of Information System and Computer Science). 4. 8. 10.56327/ijiscs.v4i1.82. [4] El Ashmawi, Walaa & Elbohy, Shereen & Rafik, Mina & Ashraf, Ahmed & Gorgui, Sherif & Emil, Michael & Ali, Karim. (2023). An Interactive Chatbot for College Enquiry. 2. 20-28. [5] Lam, Khang & Le, Nam & Kalita, Jugal. (2022). Building a Chatbot on a Closed Domain using RASA. [6] Microsoft Azure documentation, available at https://learn.microsoft.com/en-gb/azure/?product=popular [7] Overview on Dialogue Management, https://www.sciencedirect.com/topics/computer-science/dialog-management. [8] Azure integrations: https://learn.microsoft.com/en-us/azure/bot-service/bot-service-manage-channels?view=azure-bot-service-4.0&culture=en-us&country=us [9] Azure security: https://azure.microsoft.com/en-us/explore/security/ [10] Patil, Amit & Karuppiah, Marimuthu & A, Nagaraja & Niranchana, R.. (2017). Comparative study of cloud platforms to develop a Chatbot. International Journal of Engineering & Technology. 6. 57. 10.14419/ijet.v6i3.7628. [11] Wayesa, Fikadu. (2021). Design and Implementation of a Rule Based Afaan Oromoo Conversational Chatbots. [12] Windiatmoko, Yurio & Hidayatullah, Ahmad Fathan & Rahmadi, Ridho. (2020). Developing FB Chatbot Based on Deep Learning Using RASA Framework for University Enquiries. [13] W. Mahanan, J. Thanyaphongphat, S. Sawadsitang and S. Sangamuang, \"College Agent: The Machine Learning Chatbot for College Tasks,\" 2022 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI DAMT & NCON), 2022, pp. 329-332, doi: 10.1109/ECTIDAMTNCON53731.2022.9720420. [14] M. M. Khan, \"Development of An e-commerce Sales Chatbot,\" 2020 IEEE 17th International Conference on Smart Communities: Improving Quality of Life Using ICT, IoT and AI (HONET), 2020, pp. 173-176, doi: 10.1109/HONET50430.2020.9322667. [15] A. Ait-Mlouk and L. Jiang, \"KBot: A Knowledge Graph Based ChatBot for Natural Language Understanding Over Linked Data,\" in IEEE Access, vol. 8, pp. 149220-149230, 2020, doi: 10.1109/ACCESS.2020.3016142. [16] G. Daniel, J. Cabot, L. Deruelle and M. Derras, \"Xatkit: A Multimodal Low-Code Chatbot Development Framework,\" in IEEE Access, vol. 8, pp. 15332-15346, 2020, doi: 10.1109/ACCESS.2020.2966919. [17] B. Chempavathy., S. N. Prabhu, D. R. Varshitha, Vinita and Y. Lokeswari, \"AI based Chatbots using Deep Neural Networks in Education,\" 2022 Second International Conference on Artificial Intelligence and Smart Energy (ICAIS), 2022, pp. 124-130, doi: 10.1109/ICAIS53314.2022.9742771. [18] Ranoliya, Bhavika & Raghuwanshi, Nidhi & Singh, Sanjay. (2017). Chatbot for university related FAQs. 1525-1530. 10.1109/ICACCI.2017.8126057. [19] Caldarini, G.; Jaf, S.; McGarry, K. A Literature Survey of Recent Advances in Chatbots. Information 2022, 13, 41. https://doi.org/10.3390/info13010041 [20] The Semantic Web - 16th International Conference, ESWC 2019, Portoroz, Slovenia, June 2-6, 2019. https://doi.org/10.48550/arXiv.1909.03653 [21] Shingte, Kshitija and Chaudhari, Anuja and Patil, Aditee and Chaudhari, Anushree and Desai, Sharmishta, Chatbot Development for Educational Institute (June 6, 2021). [22] Gawade, Harshala. (2020). College Enquiry Chat-Bot System. International Journal of Engineering Research and. V9. 10.17577/IJERTV9IS090396. [23] V. Prathyusha, G. L. Sri, G. Meenakshi and Y. K. Chakravarti, \"Information Acquisition Chatbot System using LUIS,\" 2021 6th International Conference on Inventive Computation Technologies (ICICT), 2021, pp. 1025-1029, doi: 10.1109/ICICT50816.2021.9358531. [24] R. Parkar, Y. Payare, K. Mithari, J. Nambiar and J. Gupta, \"AI And Web-Based Interactive College Enquiry Chatbot,\" 2021 13th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), 2021, pp. 1-5, doi: 10.1109/ECAI52376.2021.9515065. [25] Daswani, Mohinish & Desai, Kavina & Patel, Mili & Vani, Reeya & Eirinaki, Magdalini. (2020). CollegeBot: A Conversational AI Approach to Help Students Navigate College. 10.1007/978-3-030-60117-1_4. [26] Shanmugam, Dr Raju & Jena, Soumya & Gaur, Vishvaketan. (2020). College Information Chat-Bot System Based on Natural Language Processing. Xi\'an Dianzi Keji Daxue Xuebao/Journal of Xidian University. 14. 825-831. 10.37896/jxu14.5/086. [27] K, Ashok. (2020). Smart College Chatbot using ML and Python. 10.1109/ICSCAN49426.2020.9262426. [28] Hao Fang, Hao Cheng, Maarten Sap, Elizabeth Clark, Ari Holtzman, Yejin Choi, Noah A. Smith, and Mari Ostendorf. 2018. Sounding Board: A User-Centric and Content-Driven Social Chatbot. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 96–100, New Orleans, Louisiana. Association for Computational Linguistics.
Copyright © 2023 Dhruv Patel, NIhal Shetty, Paarth Kapasi, Ishaan Kangriwala. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Paper Id : IJRASET51324
Publish Date : 2023-04-30
ISSN : 2321-9653
Publisher Name : IJRASET
DOI Link : Click Here