Comparative Study on Chatbot Frameworks

Abstract

Chatbots are gaining popularity in different sectors of our day-to-day life from virtual assistants on our mobile devices, and on different social media platforms to customer service agents on different websites, organizations, and institutes. Due to the popularity of chatbots there exist multiple ’frameworks’ of varying characteristics for developing Chatbots. Hence, selecting the right ’framework’ for chatbot development is a problem, and the outcome of a study to better understand these frameworks can mitigate that problem to some extent. Studying every intricate detail of all of the existing chatbot development frameworks in one single study is not feasible. In this, two of the most popular chatbot development frameworks (Microsoft Bot and Rasa) to study, and some topics closely related to chatbot development were taken into consideration for comparison. The goal was to compare these two frameworks and recommend which may be a better solution for chatbot development. Documentations of each framework were taken into account and two identical case study chatbots were implemented using each framework to study these frameworks regarding those related topics.

Introduction

I. INTRODUCTION

According to the Researcher and consulting services for businesses, and organizations in the IT sector, have a high demand for deploying Chatbots. A Chatbot can carry out many functions as users interact with the Chatbot. This would save time and a lot of effort instead of waiting for a response from humans. The Chatbot would provide instant information anytime and from anywhere, this would also avoid the involvement of manpower and make the Chatbot available 24*7 and solve the queries. Chatbot is a computer program that humans will interact with in natural language and includes artificial intelligence techniques such as NLP (Natural language processing) that make the chatbot more interactive and reliable. The term "ChatterBot" was coined by Michael Mauldin (creator of the first Verbot) in 1994 to describe conversational programs.

Chatbots are much more demanding in the current day. The emergence of social media platforms, e-commerce sites, sophisticated mobile devices, and widespread adoption of all of these aspects by people across the globe have opened up different use cases for Chatbots. The rapid development of different technologies closely related to chatbots such as artificial intelligence, natural language processing, etc. This has also motivated developers to develop and integrate chatbots into their projects. Chatbots are not completely new technology. Different studies show the growth of chatbots billions of people are willing to use chatbots and they use messaging apps frequently.

The growth of chatbots has led to the development of different types of chatbot development frameworks. There are numerous Chatbot development frameworks (e.g. RASA, Dialog-flow, IBM Watson) out there. However, very little work exists to understand these tools. The outcome of studying these tools may help chatbot developers to gain better insight into them and pick the right one for their projects.

The purpose of this is to study two such tools Microsoft Bot and Rasa and attempt a comparison between them by using. The reasons to select these two frameworks are: these two are very popular Chatbot development frameworks. The comparison has been done through a case study by implementing identical chatbots: one using RASA and one using Microsoft Bot and also studying the documentation of those two frameworks. Chatbot development frameworks are currently in use by many Chatbot developers.

II. TYPES OF CHATBOTS

Fig1, Based on the working of Chatbots can be divided into 2 types: Rule-based Chatbots and AI-powered Chatbots. Rule-based Chatbots work when particular commands have been given or the user types in certain things. AI-powered chatbots are more sophisticated and hence much smarter than rule-based Chatbots. AI-powered Chatbots utilize machine learning. This type of bot can predict and behave accordingly based on previous interactions.

Based on the accessibility of knowledge chatbots can be divided into 2 major categories: Open Domain and Closed Domain. Open-domain bots can talk about general topics and respond accordingly. Closed-domain bots have been defined as bots that specialize in one area of knowledge in which it will provide a reply based on the area it is been trained. For example, a bot that can report day-to-day updates for employees. It will not be able to answer if asked about books or foods.

Based on what type of service is provided bots can be categorized into three categories: Interpersonal, Intrapersonal, and Interagent. Interpersonal bots are those that are not supposed to act as the user’s companion, but rather an information provider. For example, a bot that can book flight tickets or a place in a restaurant. Intrapersonal bots are those that are supposed to be the user’s companion. They reside in the user’s messenger or slack channel, can manage a calendar for the user, and store the user’s information. The last type: interagent bots are bots that can communicate with other bots and will be prevalent in IOT dominant ecosystems, where 6 one bot may act as the service handler of other bots or manage communication between bots and so on.

III. LITERATURE SURVEY

The world is leaping into a future where everything will be automated. Most of the tasks could be completed without the intervention of human beings. Heading in the right direction, chatbots have taken the world by storm. A chatbot is an automated query response system that deals with end users to answer their queries and eliminates customer service needs. There wouldn’t be any delay in providing services if a chatbot handles all the queries systematically.

Artificial Machine Intelligence is a very complicated topic. It involves creating machines that are capable of simulating knowledge. This paper examines some of the latest AI patterns and activities and then provides an alternative theory of change in some of the popular and widely accepted postulates of today. Based on basic A.I. (Artificial Intelligence) structuring and working for this, System-Chatbots are made (or chatterbots). The paper shows that A. I am ever improving. As of now, there isn’t enough information on A.I. However this paper provides a new concept that addresses machine intelligence and sheds light on the potential of intelligent systems.

Technological advances in artificial intelligence and machine learning have led to the growth in the development of chatbots. Chatbots are virtual agents which mimic a user to give a response to the user’s queries. Chatbots have gained a lot of popularity in the business as they can automate customer service and reduce the efforts of humans. Chatbot frameworks are prebuilt bot engines that help users to reduce development time. In this paper, an overview and comparison of various chatbot frameworks is presented along with this it also helps the users to identify how to select one of the chatbot frameworks for their business. Chatbot frameworks are compared based on the features it provides to end users.

Nowadays, many people are using smartphones with many new applications i.e., technology is growing day by day. A chatbot has information stored in its database to identify the sentences and make a decision itself as a response to answer a given question. The college inquiry chatbot will be built using an algorithm that analyses queries and understands the user’s message. This chatbot is implemented using RASA. Rasa is an open-source framework for building AI bots that consists of two components: Rasa NLU and Rasa core. Rasa core is the component that handles the dialog engine for the framework and helps in creating more complex chatbots with customization.

Rasa’s NLU helps the developers with the technology and the tools necessary for capturing and understanding user input and determining the intent and entities.

In the era of chatbots, besides imitating humans they can also perform complex tasks like booking tickets for movies, etc. Out of various implementations, RASA is an open-source implementation for NLU and DIET models. It can interact with the database, API, conversational flow, and interactive learning with reinforcement Neural networks. In this study, various features of the rasa core are studied and up to much extent it can perform complex tasks.

IV. PROPOSED WORK

In this, we have discussed Chatbot Framework that is used to build Chatbots. The chatbot frameworks which we identify are Dialog-flow, IBM Watson, Rasa, Microsoft Bot, and Wit.ai, All these platforms are powered by machine learning algorithms. Chatbot frameworks help to build, connect, publish, and manage chatbots that are smart and interactive to give the best user experience. It helps to create new chatbots with ease.

1. Microsoft bot: Microsoft Bot Framework and Azure Bot Service are a collection of libraries, tools, and services that let you build, test, deploy, and manage intelligent bots. It offers a quick and easy way to start building bots, providing a conversational interface for new and existing apps or services.

The Bot Framework includes a modular and extensible SDK for building bots and connecting to AI services. It lets you create new experiences and reach billions of users connected through different channels such as Skype, and Facebook Messenger.

Key features:

a. It uses intents and entities.

b. Provides Active learning.         

c. JSON for easy use.

d. It gives HTTP endpoint for integration.

2. Rasa: Rasa is an open-source machine learning framework for building AI assistants and chatbots. To work with Rasa mostly you don't need any programming language experience. Rasa is a Python framework that helps us to build any kind of Chatbot easily. It is based on NLU (Natural Language Processing) which offers the possibility to understand what the user wants.

Key features:

a. It is open-source

b. User can customize models for training data.

c. It can be deployed on-premises

3. Dialog-flow: Dialog-flow is a Google-owned framework that enables users to develop human-computer interaction technologies that can support Natural Language Processing (NLP). It lets you make Digital Programs that interact with end users through natural languages.

Key features:

a. Using Intents and Contexts.

b. The modeling of large and complex systems.

c. Ability to handle the code proactively and decreases server-side coding.

d. It offers one-click integration with several platforms such as Twitter, and Facebook.

4. IBM Watson: IBM Watson is AI for business. Watson helps organizations predict future outcomes, automate complex processes, and optimize employees' time. It is a data analytics processor that uses natural language processing, a technology that analyzes human speech for meaning and syntax.

IBM Watson performs analytics on vast repositories of data that it processes to answer human-posed questions, often in a fraction of a second.

Key features:

a. Channel integration

b. Skill routing

c. Provide a specific and customized solution

5. Wit.ai: Wit.ai works on stories as a key concept to model the behavior of Chatbot. Every story gives an example of all possible conversations. In this bot the developers teach the Wit.ai using examples and when a user writes about a similar type of object. It will be able to process the request and get the extract entities and apply developer logic to the scenario. It is based on the webhook integration in which the information bot sends a command into the web service and gets the results from it.

Key features:

d. The story concept is very useful and powerful.

e. Branches lead to better control of the conversation also conditions on the actions.

Comparing the Microsoft bot framework and the rasa framework. Here are the parameters to be considered between the Microsoft bot framework and the Rasa framework

• Integration capabilities: Both frameworks have a wide range of integrations, but MS Bot Framework has a stronger focus on Microsoft services.
• Performance of Natural Language Understanding: Intent classification accuracy, Entity classification accuracy. Define a dataset, train, and evaluate models.
• Cost: Microsoft Bot has cost for specific services and features and Rasa is free to use for core functionality
• Conversational flow management: Microsoft Bot provides a dialog management system and creates conversation flows using the Bot Framework SDK and Rasa provides a dialog management system that can define conversation flow using the Rasa Story format.
• Open source: Rasa Framework is open-source, while MS Bot Framework is not.
• Community support: Both frameworks have active communities, but Rasa Framework has a more robust open-source community
• Documentation: Microsoft Bot Framework has extensive documentation and tutorials, while Rasa Framework's documentation may require more technical expertise.
• Authentication / Authorization: Microsoft Bot  Azure Active Directory(AAD) provides role-based access control (RBAC) features and Rasa custom action-based authentication.
• Comparison of Deployment: Microsoft Bot deployed in Azure and RASA deployed locally.
• Customization: Microsoft Bot Azure Functions perform custom tasks such as sending emails, or retrieving data from a database as Rasa Integration performs tasks with external services and Custom connectors.
• Architecture: How do the architecture and design of the Microsoft bot framework and the Rasa framework differ and are those differences affect the development and deployment of Chatbots?

V. IMPLEMENTATION

A. Integrations Capabilities

In Fig 2, Fig 3, Fig 3.1, Fig 4, and Fig 4.1, the Microsoft bot is integrated with external services such as a database, Language Understanding (LUIS), Conversational language understanding (CLU), Question & Answer marker (QnAMaker) and the Testing is on Bot Emulator

C. Cost

Microsoft bot has both types as free tier and pay-as-you-go so here is the list of the services according to the cost-type  Rasa bot has a free tier but now they have said that the Rasa community will not be able to support free services as there will be prices for certain services

D. Conversational flow management

Microsoft Bot provides a dialog management system and creates conversation flows using the Bot Framework SDK  Rasa provides a dialog management system that can define conversation flow using the Rasa Story format.

E. Open-source

Microsoft bot framework is proprietary  rasa is open-source Rasa is an open-source machine-learning framework for automated text and voice-based conversations. Understand messages, hold conversations, and connect to messaging channels and APIs.

F. Community support

Microsoft frameworks have active communities Rasa Framework has a more robust open-source community

G. Documentation

Microsoft Bot Framework has extensive documentation and tutorials. Rasa Framework's documentation may require more technical expertise.

H. Authentication / Authorization:

Microsoft bot framework supports Azure active directory (ADD) Rasa framework supports Role-based access control (RBAC)

Conclusion

Chatbots are becoming popular in different mediums (e.g. social media platforms, virtual assistants in mobile devices, and business websites), and due to this popularity, a set of chatbot development frameworks and platforms have emerged. This study was an attempt to compare two major chatbot development frameworks: Microsoft Bot and Rasa. For the comparison, two identical chatbots were developed using each framework and a set of factors were taken into consideration. The comparison was done based on those factors, using the implemented chatbots and existing documentation of these frameworks. According to the comparison of this study, considering those factors, the Microsoft Bot framework has more integration capabilities than RASA, and the implemented prototype chatbots were very simplistic. Only a few variants of conversations were implemented. The performance of NLU was better in Microsoft Bot than in Rasa. The user experience of each chatbot was not tested thoroughly either. The factors that were taken into account for comparison are very generic. In future works, for more effective comparisons, a more narrowed-down approach over one or two areas is suggested.

References

[1] Pal, Angad, J. Rishi, and K. Saritha. \"Cross Domain Answering FAQ Chatbot.\" In 2022 International Conference on Advanced Computing Technologies and Applications (ICACTA), pp. 1-4. IEEE, 2022 [2] Suhel, Sasha Fathima, Vinod Kumar Shukla, Sonali Vyas, and Ved Prakash Mishra. \"Conversation to automation in banking through chatbot using artificial machine intelligence language.\" In 2020 8th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions)(ICRITO), pp. 611-618. IEEE, 2020. [3] Shah, S., and Seema Shah. \"A comparison of various chatbot frameworks.\" J. Multi-Criteria Decis. Anal 6 (2019): 375-383. [4] Ingale, Nikita, Tushar Anand Jha, Ritin Dixit, and V.K. Borate. \"College inquiry Chatbot using rasa.\" International Journal of Scientific Research in Science and Technology 5 (2020). [5] Sharma, Rakesh Kumar, and Manoj Joshi. \"An analytical study and review of open source chatbot framework, RASA.\" Int. J. Eng. Res 9, no. 06 (2020): 1011-1014. [6] Mohammed, Moataz, and Mostafa M. Aref. \"Chatbot system architecture.\" arXiv preprint arXiv:2201.06348 (2022). [7] Cahn, Jack. \"CHATBOT: Architecture, design, & development.\" University of Pennsylvania School of Engineering and Applied Science Department of Computer and Information Science (2017). [8] Lokman, Abbas Saliimi, and Mohamed Ariff Ameedeen. \"Modern chatbot systems: A technical review.\" In Proceedings of the Future Technologies Conference (FTC) 2018: Volume 2, pp. 1012-1023. Springer International Publishing, 2019. [9] Clarizia, Fabio, Francesco Colace, Marco Lombardi, Francesco Pascale, and Domenico Santaniello. \"Chatbot: An education support system for student.\" In Cyberspace Safety and Security: 10th International Symposium, CSS 2018, Amalfi, Italy, October 29–31, 2018, Proceedings 10, pp. 291-302. Springer International Publishing, 2018. [10] Arteaga, David, Juan Arenas, Freddy Paz, Manuel Tupia, and Mariuxi Bruzza. \"Design of information system architecture for the recommendation of tourist sites in the city of Manta, Ecuador through a Chatbot.\" In 2019 14th Iberian Conference on information systems and Technologies (CISTI), pp. 1-6. IEEE, 2019. [11] Ngai, Eric WT, Maggie CM Lee, Mei Luo, Patrick SL Chan, and Tenglu Liang. \"An intelligent knowledge-based chatbot for customer service.\" Electronic Commerce Research and Applications 50 (2021): 101098. [12] [Online]. Available: https://www.ibm.com/watson [13] [Online]. Available: https://dialogflow.com/ [14] [Online]. Available: https://www.luis.ai [15] [Online]. Available: https://rasa.com/

Copyright

Copyright © 2023 Vidya G, Nagaraj Krishna Vernekar, Ganraj Kelkar . This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.