EchoGlove: Assistive Smart Glove for Gesture-Based Communication and Health Monitoring

Abstract

By creating a multifunctional smart glove, this paper offers a novel way to help people who are both deaf and mute. The suggested system combines an ESP32 microcontroller, which includes an integrated Bluetooth module, a speaker for voice output, a pulse sensor for health monitoring, an ultrasonic sensor for obstacle detection, and a flex sensor-based hand gesture recognition system. Using a speaker, the smart glove converts preset hand movements into spoken words, facilitating effective communication between users. The ultrasonic sensor aids in navigation by identifying surrounding objects, while the pulse sensor tracks heart rate and sounds an alarm if an irregular heart rate is detected. This affordable, wearable, wireless assistive technology seeks to improve safety and communication for those with speech impairments.

Introduction

I. INTRODUCTION

Although effective communication is crucial, people with speech and hearing impairments encounter several obstacles. By combining speech output, gesture recognition, health monitoring, and navigation support into a single wearable gadget, the suggested Smart Glove seeks to close this gap. Users can converse more readily thanks to an ESP32 microcontroller that interprets hand motions detected by flex sensors and plays the appropriate spoken words through a speaker. The glove also uses an ultrasonic sensor to help detect obstacles and a pulse sensor to track heart rate. For real-time display and health tracking, the gadget establishes a wireless connection with a mobile application. The goal of this project is to give people with speech and hearing impairments a multipurpose, reasonably priced assistive technology that will improve their freedom, safety, and communication.

II. LITERATURE REVIEW

Sanjay S¹, Dineshkumar S², Suresh M³, and Vasanthakumar S? investigated the creation of a smart glove for the deaf-mute. Flex sensors and accelerometers were used in their study's system, which used an Arduino UNO for processing and a web-based user interface to show text output. The necessity for more standalone and portable solutions was brought to light by this strategy. [1]

Sharma A¹ and Iyer M² focused on wearable gesture detection gloves for mute people. Their study focused on bridging the communication gap for individuals with speech impairments by converting hand gestures into text-based communication. According to the study, the glove shows promise as a means of improving communication. [2]

Suresh S¹ and Kumar D² implemented an IoT-based smart support glove for people with disabilities. Their research highlighted how the Internet of Things (IoT) facilitates real-time data transfer and remote monitoring in assistive technologies. This method made clear how important it is for these gloves to have designs that are both user-friendly and energy efficient. [3]

Patel A¹ and Verma R² created a smart glove designed as a wearable device for people with disabilities, with a particular emphasis on gesture detection and real-time feedback. Their research explored the potential of integrating advanced sensors and technology to interpret hand gestures accurately. The study highlighted the importance of enhancing energy efficiency, as well as improving gesture detection accuracy, to ensure that the glove functions seamlessly as an assistive technology. [4]

Rajiv Gandhi Institute of Technology, Mumbai unveiled the "Multi-Purpose Smart Glove," a device intended to help those with disabilities. Along with capabilities like obstacle detection and health monitoring, this glove converts sign language movements into audible speech. According to the study, integrating several features into a single glove might greatly increase user autonomy. [5]

Kadam K¹, Telange S², Yadav K³, and Vishwakarma A? developed a smart glove for mute people that converts hand gestures into audible speech. The system made use of a microprocessor, wireless communication module, and flex sensors. Notwithstanding its efficacy, the study pointed out its shortcomings in real-time response and gesture vocabulary. [6]

SOLUTION PROCEDURE

The suggested smart glove solution is a complete assistive technology made to help those who have hearing and speech problems. It provides an accessible communication tool by fusing several technologies into a single wearable device. Five flex sensors are incorporated into the glove along the fingers, recognising pre-programmed hand movements that correlate to everyday words and phrases. An audio message is played through a speaker built into the glove when a user makes a gesture, which is processed by the ESP32 microcontroller. The gesture data is simultaneously sent to a smartphone application using Bluetooth, enabling dual communication via the app's visual text and voice features.

With the addition of a pulse sensor, the glove puts user safety above communication. The user's heart rate is continuously monitored by this sensor, which looks for any irregularities. The device will sound an alarm if the user's heart rate deviates from the typical range, guaranteeing that they get help right away if needed. This tool is very helpful for people who need regular monitoring due to underlying health issues. The dual-purpose glove improves safety and quality of life by fusing communication tools with real-time health monitoring.The glove's ultrasonic sensor, which helps users navigate their environment by identifying obstructions, is another important feature. The sensor alerts the user when an object is within a crucial distance, enabling them to confidently navigate through new areas and prevent collisions. This function for detecting obstacles is particularly helpful in busy or unfamiliar settings where mobility may be limited. This smart glove promotes independence, safety, and accessibility for people with speech and hearing impairments by combining gesture recognition, speech output, health monitoring, and environmental awareness into a comprehensive, wireless, and reasonably priced assistive solution.

Fig. 1 Block diagram of smart hand glove system

Fig 1: An ultrasonic sensor recognises impediments, a pulse sensor tracks heart rate, and flex sensors detect hand motions in the wearable assistance system depicted in the block diagram. With the help of a dedicated supply, a microcontroller processes these inputs in accordance with preset instructions and outputs the results via an application for user interaction and a speaker for audible feedback. Through the translation of motions into speech or app-based communication, this configuration probably helps people with disabilities.

Fig. 2 Flowchart of smart hand glove system

The flow chart shows a smart hand glove system with flex sensors that record finger movements, a pulse sensor that tracks heart rate, and an ultrasonic sensor that identifies obstructions. An ESP32 microcontroller processes these inputs in accordance with preset instructions, producing outputs through an application interface and a speaker. It's likely that this approach helps people with disabilities communicate and navigate.

IV. ANDROID APPLICATION

By establishing a wireless connection with a mobile application via Bluetooth, the glove allows for smooth communication between the device and the user's tablet or smartphone. The application instantly shows the gesture and its associated meaning in real time when the glove detects a particular gesture, giving the message a visual representation. This function makes sure that even those who are not familiar with sign language may grasp the message being sent. The software may also save and monitor health parameters, such heart rate data, that are gathered from the pulse sensor. This function enhances the user's general well-being by enabling continuous health monitoring and enabling the app to produce reports or alarms in the event of aberrant data.

Fig. 3 Application for real-time gesture and health data transmission

An application created for smooth communication with the smart glove's Bluetooth module is shown in Figure 3. It receives commands from the flex sensors and uses a pulse sensor to monitor and show the user's heart rate via Bluetooth. Real-time tracking and interaction are made possible by this feature, which guarantees effective data flow between the glove and the user for improved usability and performance.

V. FLEX SENSOR OUTPUT AND CORRESPONDING INSTRUCTIONS

Specific outputs that match predetermined instructions are detected by the flex sensor. The message will either be shown on an Android application over Bluetooth or spoken out loud through a connected speaker when a specific output is activated. For instance, if the sensor picks up the phrase "I need rest," the system will react by speaking the message or displaying it on the app that is connected. Every flex sensor output is interpreted by the system and converted into a meaningful message.

Requests such as "Open the door" or "Call emergency" are instantly acknowledged by the system thanks to these instructions, which facilitate smooth communication. The flex sensor's Bluetooth connectivity allows for real-time answers, which makes it a useful tool for people to express their wants with simple hand motions. The system can respond quickly thanks to the specified outputs, offering a user-friendly interface that guarantees efficiency and accessibility.

VI. FUTURE SCOPE

1. AI-driven speech synthesis: Including AI-driven speech synthesis will allow for more contextually aware and natural-sounding voice outputs for lively discussions.
2. Increased gesture vocabulary: The glove's gesture recognition capabilities can be enhanced through machine learning, enabling a wider variety of expressions and more individualised communication.
3. Text-to-speech conversion: In order to facilitate faster communication, future app development may incorporate real-time text-to-speech conversion, which would enable instant audible voice from text inputs.
4. Cloud health data storage: Remote monitoring and individualised medical insights for improved long-term care are made possible by storing health metrics in the cloud.
5. Increased battery life and sensor accuracy: These improvements will guarantee dependable operation, precise gesture recognition, and health monitoring over time.

Conclusion

For those who struggle with speech and hearing, the suggested smart glove provides an innovative solution that improves safety and communication. It enhances users\' independence by integrating speech output, gesture recognition, health monitoring, and obstacle detection. Real-time audio and video communication is possible through the glove\'s seamless integration with a smartphone app. Additionally, it monitors vital indicators, adding another layer of security. Users will be able to live more freely and confidently thanks to future advancements including cloud storage, a wider gesture vocabulary, and AI integration. For those who depend on assistive communication devices, this discovery has the potential to revolutionise daily life.

References

[1] Sanjay, S., Dineshkumar, S., Suresh, M., & Vasanthakumar, S. - Development of Smart Glove for Deaf-mute People, NVEO – Natural Volatiles & Essential Oils Journal, Volume 8, Issue 4, 2021. [2] Sharma, A., & Iyer, M. - Wearable Gesture Detection Glove for Mute People, IEEE Xplore, 2021. [3] Suresh, S., & Kumar, D. - Implementation of IoT Based Smart Assistance Glove for Disabled Person, International Journal of Research and Analytical Reviews, Volume 9, Issue 2, 2022. [4] Patel, A., & Verma, R. - Smart Glove: A Wearable Device for Disabled Person, Journal of Emerging Technologies and Innovative Research, Volume 9, Issue 4, 2022. [5] Rajiv Gandhi Institute of Technology, Mumbai - Multi-Purpose Smart Glove for Differently Abled Community People, Journal of Emerging Technologies and Innovative Research, Volume 10, Issue 4, 2023. [6] Kadam, K., Telange, S., Yadav, K., & Vishwakarma, A. - Helping Hand: A Glove for Mute People, International Journal for Research Trends and Innovation, Volume 8, Issue 4, 2023. [7] Sambath, M., & Suresh, M. - Development of an IoT-Based Smart Glove for Disabled Individuals, International Journal of Electrical and Electronics Engineering, Volume 12, Issue 3, 2023. [8] Singh, R., & Yadav, A. - Smart Glove with Gesture Recognition for Disabled People, International Journal of Advanced Research in Electronics, Communication and Instrumentation, Volume 8, Issue 2, 2023. [9] Rajendran, G., & Kumar, S. - Gesture-Controlled Smart Glove for Speech-Impaired People, Journal of Biomedical Engineering Research, Volume 9, Issue 4, 2024. [10] Sivasubramanian, P., & Sathish, R. - Design and Implementation of a Smart Glove for Hearing and Speech Impaired Individuals, Journal of Assistive Technology and Robotics, Volume 6, Issue 5, 2023. [11] Gupta, V., & Sharma, D. - Wearable Smart Glove with Real-Time Assistance for Differently Abled People, International Journal of Robotics and Automation, Volume 15, Issue 2, 2023.

Copyright

Copyright © 2025 Vijay A, Lochan Pingalay S, Mohanakrishnan. N, Rohan Raghavendra. G, Shalini Agnes Mary A. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.