SAMPURNA- A Multi-Functional Medical Assistant Application Framework

Abstract

In recent times, the need for the advancement of medical facilities has been majorly realised and a lot of efforts have been made in the same direction. Healthcare is vital to leading a healthy and prosperous life. However, there are still certain arenas where there is a need for improvement so that people can benefit the most. The COVID-19 pandemic made everyone realise that there is nothing more important than their own health and family. During the peak times when India was trembling with the COVID waves, in spite of the ceaseless efforts from the doctors, there were still many people who remained unattended and eventually perished because of the lack of awareness and aid facilities. Had there been a system through which they could have gotten the aid on time, they wouldn’t have died a helpless death. Here we bring a system which has the capability of solving such issues for one and all. This Multi-Functional Medical Assistant Framework, as the name suggests, would provide the patient or the person in need with medical aid. The Chatbot will provide the exact medical composition that would suite their metabolism in an orchestrated and expedited manner. In this paper we have provided the design, the internal functioning, and the implementation of our application. General Terms Natural Processing Language Pattern Matching Artificial Intelligence Medicine.

Introduction

I. INTRODUCTION

"Sampurna", as the name suggests, is a Multipurpose Progressive Web Application which will serve a gamut of facilities to the user in the field of medicine. We are providing a common platform where a patient can visit a page and seek whatever they are looking for. This consists of five modules, namely Speech to Text, Text to Speech, Image to Text translators, a medical assistant chatbot, and an automatic prescription generator. The translators will act as catalysts in using the chatbot and medical prescription modules to expedite the result. The chatbot and the medical prescription generator will be the main features of this project.

A. Medical Chatbot

As a type of conversational agent or computer software, a chatbot is meant to support intelligent discussion via audio and/or written techniques. We can also say that, it is a type of artificial intelligence (AI) that can engage in conversation with a human. that is meant to support intelligent dialogue through auditory or written means.

A chatbot is a system that is capable of executing linguistic communication with humans.

The chatbot provides an interface between the user and the system with smart features.[1] With the increasing usage of machines and the need to provide natural language interfaces, the need for these types of conversational agents has grown.[5]

If implemented correctly, a chatbot's knowledge base should have a collection of pre-defined templates that match user inputs and provide the appropriate responses.[6]

This Medical Chatbot will enable user to consult a sensible medical expertise by sitting at their homes. All they need to do is to provide relevant information in the bot and then answer some direct questions, and the bot will come up with the most appropriate result

B. Medical Prescription Generator

The Medical Prescription generator will help in generating E-Prescription for the patients. This will provide an easy and convenient access of the medical prescription.

This will further end the problem of comprehending the doctor’s writing which in general case is not legible at times.

Having an E-Prescription will also be making it available in the system in the long run so that whenever a patient wishes to get the old records, they can easily fetch it. The prescription will be available for download in various formats such as PNG, JPEG, PDF.

B. Three Tier Application

This will be a 3-Tier Application following all the industry standards, this will be based on multiple databases at the backend using both Oracle 19c as well as SQL Server 2019.

This will use JPA (Java Persistence API) and Hibernate dependencies with the help of which the database tables will be automatically created using Object Relational Mapping.

The middle communication layer will we comprising of multiple Restful APIs which will be developed using both Spring boot as well as .Net Core separately. With the help of rest templates, application will be communicating among pages via URI (Uniform Resource Identifier).

Also, the user credentials will be end-to-end encrypted using JWT (JSON Web Token). The application’s front-end will be developed using ReactJS or Angular, through which the clients will interact with the application, whenever the client will request for the available services, the request will then be forwarded to the back-end of the application using Rest API call.

II. ORGANIZATION OF PAPER

In this paper, topic ‘4’ deals with the related work in this domain that has been previously done, the topic ‘5’  contains the workflow of the entire project comprising of the flowchart, DFD 0 and DFD 1 diagrams, the next topic ‘6’ will brief us about the methodology of the entire application, topic ‘7’ comprises of the literature surveys of some of the related research papers that we found relevant, the next topic ‘8’ covers the conclusion of the project and finally the last topic ‘9’ enlists the references of various research papers used in order to complete this application.

III. RELATED WORK

In recent years, a number of medical Chatbot prototypes have been created that attempt to advise the person with medical consultations after extracting the illness information from the user's input messages.

There's a model for a comprehensive medical Chatbot that's reinforced mistreatment in a related work called "Pharmabot: A Pediatric Generic Medicine Consultant Chatbot" by Bien Michael B Francisco, Benilda Eleonor V. Comendador, Sharleen M. T. Nacion, Jefferson S. Medenilla & Timothy Bryce E. Serac using Visual C# with MS Access. [8].

But this does not provide doctor’s availability which is an addition in our case, along with a list of salt combinations for a particular ailment.

Our application also generates an E-Prescription which makes it altogether a unique combination.

There are several translators available in the market. Ours is not any different, the basic functionality will be the same as others in the market, the major difference between our three translators and the ones available in the market is that our translators are integrated with our chat bot and medical prescription generator in the backend.

So, when a user uses either of the translator services from our app, it will redirect the result to the chat bot or the medical prescription page as desired by the user.

For example: The doctor while prescribing the medicines to the patients, instead of writing it manually on a prescription, can use our Speech-to-Text translator for putting the spoken information as text to an E-prescription.

This will make it more legible, convenient further; the e-prescription can also be maintained in the long run and most importantly it promotes avoidance of using paper which will lead to saving trees.

IV. WORK-FLOW

V. METHEDOLOGY

This will be a three-tier application. First the user will have to either sign in or login as per there previous visit. The user will then be directed to the home page. In the home page, the user will get the five modules displayed in the Nav bar.

A. Login

The customers who already have an account in our application can directly login to our app, by entering their registered email id and the corresponding password, at the backend we will match the records and if valid the user will get the access of the application.

B. Sign-Up

This will for our new customers, who do not have an account in our app. They will need to enter their name, email id, password and mobile number. We will create their account in the backend and then they will get the access.

The First Three modules comprise of three different Translators. Speech to Text, Text to Speech and Image to Text. These translators will act as a catalyst in the implementation of Chat bot as well as Medical Prescription Generator with the respective functions.

The Next Module is a Medical Assistant Chat bot. This chat bot will act as a medical consultant which will provide the user with a combination of relevant medicines as well as an available doctor in the desired domain.

This will help the user in getting necessary information conveniently from their comforts of their homes. The chat bot is built on Artificial Intelligence or NLP technology, which enables the chatbot to work smartly and learn, analyse and improve from inputs.

The Last Module will be the Medical Prescription Generator. This will provide an easy and convenient access of the medical prescription.

This will further end the problem of comprehending the doctor’s writing which in general case is not legible at times. Having an E-Prescription will also be making it available in the system in the long run so that whenever a patient wishes to get the old records, they can easily fetch it.

VI . LITERATURE SURVEY

A. A Medical Chatbot

Natural language processing (NLP) is critical to the success of medical chat-bots since it enables users to communicate their health concerns. The chat-Bot allows the user to ask any personal health care question without having to be present at the hospital in person. With the help of the Google API, it is possible to convert audio files into text files and vice versa.

B. Octopus: A Multi Agent Chatbot

Octopus, a new and enhanced Sinhala chatbot, was presented in this study along with its design and implementation.

To design and construct the Octopus, MaSMT, a framework for developing multi-agent systems, was used. In addition, there are eight subsystems.

C. E-JAVA Chatbot: A Post-Pandemic Alternative to the Traditional Online Tutor

In the future, this e-JAVA will be expanded to include all programming language topics, but for now, it has just a limited amount of information and scope.

D. A Conversational Tool: Chatbots

In this research, we studied how Chatbots are built and how they may be used in a variety of industries. There have been comparisons with other Chatbots as well. For all intents and purposes, the chatbot must be basic, user-friendly, understandable, and condensed in its knowledge. There is still need for development in the design of Chatbots despite the recent appearance of commercial solutions.

E. Chatbots: Are they Really Useful?

To sum up our findings, we found that while technology can enhance learning experiences in some ways (such as through multimedia presentations or computer algebra systems), it cannot take the place of a skilled educator. Instead of trying to replace or accurately mimic human speech, chatbot designers should aim to create tools that make people's jobs easier while also allowing them to converse with computers in natural language.

As the Colby (1999) points out, “We need not take human-human conversation as the gold standard for conversational exchanges. If one had a perfect simulation of a human conversant, then it would be human-human conversation and not human computer conversation with its sometimes odd but pertinent properties.”

F. Extracting Chatbot Intelligence from Online Forums.

Now, all retrieved <thread-title, reply> is fed straight into the knowledge base of the chatbot. However, there is still potential for improvement.

Generalizing the conversation templates by grouping comparable subjects and responses, for example, can increase the coherence of the subsequent chat responses and help us comprehend the style of each reply.

G. Speaker C# & SQL databases in the Visual Studio are used to create an independent voice recognition application for the Paite language.

Paite language is translated into text using a Voice Recognition System created and tested in this project. The system is built on the Microsoft Visual Studio Platform, with C# serving as the programming language of choice.

The Speech Recognition Engine included into Visual Studio is quick and precise. Due to the additional terms used to test the system's confusability, the accuracy obtained as a consequence is dependable and legitimate.

H. Image Text Extraction And Text Summarization

We want to use unsupervised extraction summarization methods to discover extractive summaries from provided images. We've been able to effectively extract picture summaries using two separate algorithms: Text rank and TF-IDF.

Text may be extracted from photos using Tesseract and OpenCV, as well. There is still work to be done to accurately identify the results of this study's pipeline between two separate applications, however this research was able to create an effective summary.

I. RNNs Using Multi-Modal Attentional Hierarchy for Abstractive Synthesis of Text and Image

The text-image summarization challenge proposed in this research summarizes and concludes and align both text and pictures at the same time. Text-image summarization is a relatively new field of study, and the majority of past research has focused on synthesizing texts alone. This hierarchical encoder-decoder model is used to suggest a multi-modal attentional mechanism that concurrently pays attention to original phrases, pictures, and captions.

Begin with the RNN model and present a multimodal beam search strategy that uses the bigram overlap of produced sentences and the picture captions to score beamed images.

J. Conversion Of Written Text Into Spoken Language

Readability threshold of less than two percent and an average processing time of fewer than 3 minutes for A4 size paper size can be achieved by the use of a Text-to-Speech (TTS) device. People may use this gadget, which does not require an internet connection, on their own. We can speed up the editing process for books and websites by using this strategy.

K. History, Development, and use of Conversational Interfaces (CUIs)

Chatbots are referred to as Natural Dialog Systems (NDS) or simply chatbots in this research study.

Its goal is to compile important data or information that will serve as the foundation for future work in the area of chatbot.

L. The bot framework of Microsoft

Building sophisticated, enterprise-grade bots that have ownership and management of your data is presented in this reference.

Organizes vital data that is essential for further researches in the area of chatbots.

M. Youper, an AI-powered emotional health aide. (2020)

In terms of mental health, this website is an excellent resource. Their purpose is to make mental health treatment available to everyone one person at a time by bringing together physicians, therapist, engineers, and mission-driven professionals.

Conclusion

This will prove to be an all-in-one answer to all of the medical problems that have been plaguing the country. The user will be entertained by the chatbot, and the chatbot will also give the user with the appropriate treatment for any condition that the user may be suffering from. It will provide a list of medicines along with there salt compositions to the user. Also, it will help the patient to get to the nearest medical pharmacy or home delivery of the medicines to their entered address from that pharmacy. The medical prescription generator will ease the access plus understanding of the hand written prescription assigned by the doctors. It would also help in depleting the use of paper thereby it will be an attempt of saving the trees. The translators can be used for the medical purpose as well as for non-medical/personal purposes. This will increase the number customers using our application as it will not be restricted to only patients using our application. We will provide a separate button to get the translated result which can be fetched in various formats depending upon the services used by the person.

References

[1] Hettige, B. and Karunananda, A.S., 2015. Octopus: a multi agent chatbot. [2] Sisca, J., 2021. Penerapan Algoritma Elias Delta Code Untuk Kompresi File Video Pada Aplikasi Video Downloader. Resolusi: Rekayasa Teknik Informatika dan Informasi, 1(4), pp.254-264. [3] Daud, S.H.M., Teo, N.H.I. and Zain, N.H.M., 2020. Ejava chatbot for learning programming language: A postpandemic alternative virtual tutor. International Journal, 8(7), pp.3290-3298. [4] Dahiya, M., 2017. A tool of conversation: Chatbot. International Journal of Computer Sciences and Engineering, 5(5), pp.158-161 [5] Shawar, B.A. and Atwell, E., 2007, January. Chatbots: are they really useful?. In Ldv forum (Vol. 22, No. 1, pp. 29-49). [6] Huang, J., Zhou, M. and Yang, D., 2007, January. Extracting Chatbot Knowledge from Online Discussion Forums. In IJCAI (Vol. 7, pp. 423-428). [7] Legoh, K., 2019, March. Speaker Independent Speech Recognition System for Paite Language using C# and Sql database in Visual Studio. In 2019 2nd International Conference on Innovations in Electronics, Signal Processing and Communication (IESC) (pp. 34-38). IEEE. [8] Addya, S., 2020. Extractive text summarization of image extracted text (Doctoral dissertation, Dublin, National College of Ireland). [9] Chen, J. and Zhuge, H., 2018. Abstractive text-image summarization using multi-modal attentional hierarchical rnn. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (pp. 4046- 4056). [10] Venkateswarlu, S., Kamesh, D.B.K., Sastry, J.K.R. and Rani, R., 2016. Text to Speech Conversion Review of Chatbots Design Techniques, Nahdatul Akma Ahmad, Mohamad Hafiz Che Hamid, Azaliza Zainal, Muhammad Fairuz Abd Rauf and Zuraidy Adnan Faculty of Communication, Visual Art & Computing, Universiti Selangor, Malaysia. [11] Dialogflow. (2019). [12] Multi-Agent Communication System with Chatbots ZOJAN MEMON*, AKHTAR HUSSAIN JALBANI*, MOHSIN SHAIKH*, RAFIA NAZ MEMON* AND AHMED ALI*. [13] Microsoft bot framework. (2019). [14] 4 reasons for enterprise chatbot failure and how to overcome them using a multi-bot approach. (2020). https://servisbot.com/four-reasons-for-enterprise-chatbot-failure/. (Retrieved 30 September 2020). [15] 5 core considerations for choosing your chatbot | by Ruth Zive | chatbots magazine. (2020). [16] Zhou, H., Huang, M., Zhang, T., Zhu, X., & Liu, B. (2017). Emotional chatting machine: emotional conversation generation with internal and external memory [17] A, S., & John, D. (2015). Survey on chatbot design techniques in speech conversation systems. International Journal of Advanced Computer Science and Applications. [18] About ChatterBot—ChatterBot 1.0.2 documentation. (2019). [19] AbuShawar, B., & Atwell, E. (2015). ALICE Chatbot: Trials and outputs. Computation y Sistemas, 19. [20] Ada: Your health companion. (2020). [21] Chatbot | definition of chatbot in English by Lexico Dictionaries, (2019). Lexico Dictionaries | English. [22] #Chatbotfail: 4 chatbot customer experience fails (and how to avoid them). (2020). [23] How to reduce risk in deploying conversational ai solutions. (2020). [24] Youper—emotional health assistant powered by ai. (2020). [25] Subway bot. (2020). ChatbotGuide.org. [26] Chatbots: History, technology, and applications Eleni Adamopoulou ?, Lefteris Moussiades Department of Computer Science, International Hellenic University, Agios Loukas, 65404 Kavala, Greece.

Copyright

Copyright © 2022 Abhinav Jain, Aditya Mukherjee, Krista Chaudhary. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.