Sign Language to Speech Conversion

Abstract

A huge population in India alone is of the dumb and deaf people. So the system is working on a glove based device which will be used for conversion of sign language (ASL) to speech .The basic system consists of a two parts; sign language recognition and conversion to text and further to speech. The sign language glove consist of a simple hand gloves fitted with flex sensors which is being used for the monitoring the amount of bend on the fingers. Flex means bend, this is the sensors that change the resistance depending on the amount of bend on the sensor. Data from the sensors is send to the Control unit which is the Arduino Nano the analog signals from the sensors are digitally converted and compared with the stored value for the recognition of sign and then displayed as a text on the 16x2 LCD. Further the text output is wirelessly transmitted to a cellular phone or a PC which consist of a test to speech conversion software.

Introduction

I. INTRODUCTION
Sign language is a natural way of communication between normal and dumb people. Sign language is mostly dependent on hand gesture recognition. It is sometimes not easy for normal people to recognize the signs properly and understand what they want to say. So the intension of the gloves is to make the life style of the dumb and deaf people easy. The gloves translates the hand gestures to text and further speech so that the normal people can read the recognized gesture and hear to the voice and understand what that person wants to tell, which will make the communication more efficient. The system consists of both physical and non-physical communication. Sign language differ from country to country it is not universally same. America developed American Sign Language (ASL); British developed British Sign Language and so on. Most of the countries follow the American Sign Language and our system is also based on the same. The gloves convert the specific gestures to text then to speech using Arduino as heart of the system. The flex sensors are used in the system which is attached onto the gloves which convert the gesture into resistance which is further converted to the text through Arduino nano. The flex sensors come from flexible sensors family, which are flexible enough. The output of the sensors is processed on Arduino nano to get text as an output displayed on LCD. Further that text is sent via Bluetooth module to mobile phones/computers. Further that data is converted to speech via text to speech conversion software.

ata is converted to speech via text to speech conversion software.

II. LITERATURE SURVEY

Thomas Pryor and NavidAzodi are UG students who made the Gloves that translate sign language into text and speech known as signaloud. They had won Lemelson-MIT student price for this project. Which inspired me to make a system of my own which can be used for public welfare[1].The 1st Hand Talk glove was designed by Ryan Patterson in the year 2001. This model had limitations that a computer or a laptop was always required for its functioning which made it less portable. n 1620, Juan Pablo Bonet published R, education of letters and art for teaching mute people to speak which is considered as the first modern treatise of sign language phonetics, setting out a method of oral education for deaf people and a manual alphabet [4].

Interactive Acclerometric Glove For Hearing Impaired: Kuldeep Singh Rajput, Shashank Deshpande, UmaMudenagudi: The main aim is to set an interface between the Hearing Impaired people and normal person to improve the communication efficiency so that they can communicate handily with others.[5] Hand Gesture Recognition System :Swapnil D. Badgujar , GourabTalukdar ,OmkarGondhalekar, Feb. 2014. Implemented by real time gesture recognition a user can control a computer by doing a decided gesture in front of a video camera which is linked to the computer. [5]

III. PROBLEM STATEMENT

There are over 70 Million deaf people worldwide, and most of them rely on sign language to communicate.However, the majority of the hearing population cannot understand signLanguage, leading to communication barriers and exclusion. Our solution aims to bridge this gap and create a more inclusive society.This can be solved by , the development of the Sign Language to speech Conversion .

IV. PROPOSED METHODOLOGY

The overall functioning of the System is explained through the block diagram shown in figure. It represents the general order and hierarchy of various working blocks of the project. The person wears the glove which has flex sensors, contact sensors and accelerometer stitched to it and makes the gesture according to the American Sign Language (ASL). Arduino Nano is used to gather signals from the flex sensors and accelerometer placed on the glove. Then the processed output is send over the LCD to display the text output and also via a Bluetooth link to an Android Smartphone or a Personal Computer consisting of test to speech software (application) and speech output is obtained.

VI. FUTURE SCOPE

Enhanced accuracy: The accuracy rate of the most recent state-of-the-art technology for translating sign language using smart gloves is about 98%. Nonetheless, there is always potential for improvement, and upcoming developments in artificial intelligence and machine learning may contribute to enhancing the precision of sign language identification Miniaturization: At the moment, smart gloves are very large and can be difficult to wear for extended periods of time. Upcoming developments in wearable technology and material science may result in the creation of smaller, lighter smart gloves that are more comfortable to wear. Increased language support: Although American Sign Language (ASL) can be translated using current technology, there are numerous additional sign languages in use throughout the world. Future technological developments might result in the creation of smart gloves that can understand and translate a variety of sign languages. Connectivity with other devices: Smart gloves that can translate sign language could be connected to other gadgets like tablets or smartphones for easier accessibility and communication. Real-time feedback: Future smart gloves could translate sign language as well as give the user immediate feedback on how accurately they are signing. This could help the user become more fluent in the language.

VII. ACKNOWLEDGEMENT

I take this opportunity to express my sincere appreciation for the cooperation given by Prof.S. G. GIRAM, Principal of AISSMS'S POLYTECHNIC, Pune and need a special mention for all the help extended by him, constant inspiration and encouragement to make my project a memorable experience. I am thankful to our

 H. O. D. of E&TC Department, Prof. V.S.Gaikwadfor her time to time support and valuable guidance. I am deeply indebted to my internal guide Mrs. M.T.Dangat, for completion of this project for which she/he has guided and helped me going out of the way.. I am thankful to all teachers and professors of our department for sharing with me, valuable knowledge on their respective fields. I would also thank my fellow classmates and friends for their support and timely suggestions. I would also like to thank library staff and laboratory staff for providing me cordial support and necessary facilities, which were of great help for preparing the project report. Thanks to all!!!

Conclusion

Those who are blind, deaf, or dumb can converse with one another using this method. Those who are dumb communicate in a language that is difficult for blind and illiterate people to grasp. To aid the deaf, the sign language is also translated into written form. These words are displayed on a computer screen. Those who are deaf must be healed. We’ll utilizing it to watch how the blind and deaf move their hands. The system translates hand motions into text, which is subsequently translated into voice. A provision has been included to the text system so that people can still read and comprehend what the other person isattempting to say even if they are unable to hear the sound emitted owing to those limitations.

References

[1] Sign language recognisation using sensor gloves Mehdi,S.A FAST-Nat.univ.of Computer & Emerging Sci.Lahore. [2] K. Hanumanthu, K. HarshitSreevastcha, N. Manohar, G.Soumya Reddy. ”Sign Language To Speech Conversion”. [3] AmbikaGujrati, Kartigya Singh, Khushboo, LovikaSoral,Mrs.Ambikapathy.”Hand-talk Gloves with flex sensor”. [4] Heena Joshi, Shweta Bhati,KOmal Sharma, VandanaMatai.”Detection of Finger MotionUsing Flex Sensors for ,Assisting Speech Impaired”. [5] NishaKawale, PradnyaKaspate, HruchikaVanjari, Prof.PrachiSarod”Implementation Paper On Sign Language, Using Flex Sensor”. [6] Mr. Jose L. Hernandez-Rebollar” A New Instrumented Approach For Translating American Sign Language Into Sound And Text” [7] Ms. Maryam Rehman” American Sign Language Translation through Sensory Glove; SignSpeak”

Copyright

Copyright © 2023 Prof. M.T. Dangat, Rudra Chandgude, Pravin Kushwaha, Mohammed Champeli, Prathamesh Pardeshi. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.