Speech or Text to Indian Sign Language Convertor

Abstract

Sign language display is the software of converting the text/speech to animated sign language to meet the demands of the special needs population. The goal of this is to make the communication A more comfortable and to improve both health and productivity .As technology has improved, the usage of computer systems has increased, which means we can implement better ways to meet the needs that can impact the health of special needs population . So, this project could help people improve their communication. Thus, they improve their mental health .With the help of Blender, Python, SQL, and NLP, We have tried to design a process to detect text and subsequently the live speech , which can converted to animated sign language in real time. For the entire process, I used Blender for animation and video processing and SQL and NLP to determine the different words and text to by training the dataset and for text to animation conversion .Therefore, this project could to meet the needs of many people .The number of different countries use different sign language even same countries that are multilingual use a different system which meets the social and cultural standards of their countries. As such there is a wide disparity in many countries for digital sign language But, Software such as our could be helpful to bridge the gap between such discrepancies and be useful not just for acting as a tool for communication but also as tool use to teach . In conclusion , it is very cost effective and widely applicable .

Introduction

I. INTRODUCTION

The term “Sign Language” is known to be rooted in the Linguistics, which itself is known to have a long history. There is a sizeable population with hearing impairments, even in India there are only a handful certified interpreters

A. Purpose of the Study  

Sign language is the basis of communication for the DHH community. This includes the various types of sign languages like deaf sign language, Auxiliary sign language, and signed mode of sign language. There are various types of sign languages some of them include American sign language (ASL), British sign language (BSL) and in some cases, neighbouring countries have similar sign languages like the Indo-Pakistani sign language used by India, Pakistan, and Bangladesh. These act as a tool to bridge the daily interaction for accessing the information. Sign languages are used by the deaf and hard-of-hearing population but recently they are also used by people on the autism spectrum for improving their communication. Gesture recognition is an important topic due to the fact that segmenting a foreground object from a cluttered background is a challenging problem.

B. The Objective of the Project

The objective of this project is to convert English text language to Indian sign language using natural language processing to enhance the communication capabilities of people with hearing disabilities. Sign language is primarily used by the DHH population for communication. Though the DHH residents in India are over 50 million the population the certified sign language interpreters are less than 300 and the DHH population estimate is also a rough estimate as they generally get classified under disabilities without a separate demarcation. Even in the current landscape in countries like Russia, India, and Iran the lack of available resources for the DHH is blatantly apparent. For the sign language as the grammar and rules change depending on the input language and the sign language as one is spoken and the other audio-visual aided, then the translation complexity increases. The users of the system are not just for daily communication but can be adapted to be implemented in the education field as well. As The Indian Sign language (ISL) is not taught in school and the teaching staff is not yet thoroughly adapted to the difference in the teaching methodology. To fill the gap between the people or for people can learn sign language .There are specified models used for the ISL translations and they are not formally taught in the education and the general population that can understand the sign language is minuscule. So the program is useful for the kids as the DHH population that learn sign language at a younger age shows improved communication skills and the formalization of the digital sign language once introduced in the education field can later be extended to support the DHH in other fields such as transportation, health or service sectors wherein it can address their needs better.

The other drawback of the digitalization of the sign language is that they are not extensively applicable in real-time as most of them make of the word to word or letter-to-letter transcriptions, it is not able to keep up with the speech this can slowly be tackled by making use of the developing database that has a direct translation for the sentences starting with the commonly used sentences that are used in day-to-day like that which is useful to ask for basic communication requirements .The other concern with the digitalization for ISL is that India is a multilingual country, each state has differences in language or the vast difference in diction, this can also be addressed in the future update as this is applied in other fields, for now, it limited to usage in general English communication and to as an introduction towards the introducing the sign language in formal education as the DHH population dropping out of formal education is very high and this can be used to address whether it is lack of language communication.

 II. EXISTING SYSTEM

The systems that are currently used focus mainly only on letter-to-letter translations and can only support American sign language(ASL) It may be difficult for teaching a population to use a more popular existing sign language like ASL, BSL, etc as the difference in the everyday communication is effected the same thing may be called the difference in the everyday communication is effected the same thing may be called using a completely different thing and other differences restrict using this as an alternative. There are many sign languages divided based on the language or even for the same language like in the case of ASL and  BSL, so although there is work in this sector it cannot be applied in countries such as Russia as transcribing from sign to text and then converting that to another language causes a lot more confusion than communication. There is a lack of proper and effective audio-visual support for oral communication. There may be a wide DHH population but the systematic use of the sign language is not existing in any sector even in education the number of certified sign language interpreters is less than 1000, and this tool has a wide audience it has to cater to. There are very few systems working on ISL as the main piece of work is done on western systems like BSL, ASL. The core architecture used maybe 

1. Direct translation: Translation on direct basis with the help of a person in between as a mediator output may or may not be accurate.
2. System translated: System needs a lot of training the models and the required dataset are not easy to collect
3. Transfer-based architecture: In this, the grammar is applied so that a proper translation can happen if and when the text needs to be converted from one language to another.

III. PROPOSED METHODOLOGY

The proposed system is useful for dealing with the daily communication problems faced by the DHH population. This project makes use of the video that are more effective ways of communicating as they convey the verbal and non-verbal ques used in them.

Our proposed System translates the input into Indian Sign Language using methodologies.

1. Natural Language Processing: In the translation process we encounter many unnecessary words like ‘is’,’are’,’was’,etc. , are filler words they do not help in the translation process so we remove them from the sentence.
2. Root Words: The words may be in plural form ,adjective form or any other form .The system removes or converts the words into root words by elimination. The roots words are very effective for bridging the void that could happen when transcribing between fundamentally different languages
3. Database: This is also one of the main import part of the project. It is used to store all of the videos that are fetched that are to be fetched to the users. It makes the most data used for the project and is maintained by the admin. The database can later can be optimized such that there is direct translations for the common words and sentences so that when they occur, the project can fetch the direct  translations as it is faster and in some cases the word by word transcription becomes a bit slow to keep up with the real time conversations

This way the resources that can be made use of at any point of time and reach to a wider audience as it can easily be used to promote the sign language usage in the classes and address the communication needs as currently the number of the institutions that make use of sign language for the formal education is very scarce in various countries and these resources can be made use in other ways like in case of the children that come under the autism spectrum are taught sign language as it can help them in developing the communication.

This project is to make the digitalized ISL as a resource more widely used.

As we can see in the use case diagram the user there are two main part the NLP and the database. The Admin will update the database to provide more support such that in the future more regional languages are used and the more direct translations for the sentences are readily available.

IV. SYSTEM ARCHITECTURE AND IMPLEMENTATION

Conclusion

Sign language is very extensively used and vital in conveying information with regard to various social settings. Lack of communication can be a hindrance be it in terms of education, health, or traveling. It removes the communication-barrier between a normal hearing person and DHH Here, this model is aimed as more of a general approach, which will help them not become left out mainly in terms of education, the lack of the resources for ISL is one of the reasons that the usage of ASL is being more and more widespread. The proposed work can be used to convert the given text or speech into an animation or provide a catered video meeting the user requirements. Various types of modifications and improvements can be made as and when the ISL Dictionary grows, including extending support for regional language translations.

References

[1] Amit Kumar Shinde and Ramesh Khagalkar “sign language to text and vice versa recognition using computer vision in Marathi” International journal of computer Application (0975-8887) National conference on advanced on computing (NCAC 2015).Jyoti Budhwar, Sukhdip Singh, 2021, Sentiment Analysis based Method for Amazon Product Reviews, INTERNATIONAL JOURNAL OF ENGINEERING RESEARCH & TECHNOLOGY (IJERT) ICACT – 2021 (Volume 09 – Issue 08) [2] Neha Poddar, Shrushti Rao, Shruti Sawant, Vrushali Somavanshi, Prof.Sumita Chandak \"Study of Sign Language Translation using Gesture Recognition\" International Journal of Advanced Research in Computer and Communication Engineering Vol. 4, Issue 2, February 201Gupta, Bhumika & Negi, Monika & Vishwakarma, Kanika & Rawat, Goldi & Badhani, Priyanka. (2017). Study of Twitter Sentiment Analysis using Machine Learning Algorithms on Python. International Journal of Computer Applications. 165. 29-34. 10.5120/ijca2017914022. [3] Deaf Mute Communication Interpreter Anbarasi Rajamohan, Hemavathy R., Dhanalakshmi M. (Department of B.M.E., Sri Sivasubramania Nadar College of Engineering, Chennai, Tamil Nadu, India.)Internationalhttps://towardsdatascience.com/represent-your-geospatial-data-using- folium-c2a0d8c

Copyright

Copyright © 2022 N. Pranay, N. Ganesh, G. Suresh. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.