Wheelchair Controlled by Speech and Vision

Abstract

The voice assistant module and eye tracking are both used in the smart wheelchair designed for individuals with disabilities. A module for image processing, a wheelchair-controlled voice associate module, and restricted by appliances are among the two modules found in the smart wheelchair. The wheelchair-mounted camera that can take pictures and handle those images makes up the picture processing module. To obtain pupil movement in the 2D bearing, the acquired image was put into a Raspberry Pi microcontroller process using Open CV. The student is then remotely moved to the wheelchair management module. Additionally, the procedures are controlled by the movement of the eyes. Technology such as speech recognition offers a means for people to communicate with wheelchairs. Thus, employing this technology to manage a wheelchair makes it simple to overcome the challenges that people confront. This can be done by employing a smart phone or other device with a smart assistant as an interface between a person and a wheelchair.

Introduction

I. INTRODUCTION

This Wheel chairs are currently a necessity for the elderly and those with disabilities. Most electrical wheelchairs are controlled using a joystick control system that is readily available. The low resistance to the boisterous setting might occasionally divert framework and cause framework to react incorrectly, as in the case of a person with complete loss of motion due to amyotrophic lateral sclerosis. The users of a brain-controlled wheelchair can function without stretching the order when using electroencephalogram signals, although the organisation method is in some ways unorganised. . The goal of this project is to construct a wheelchair model that can be operated using voice or eye movements. It will be challenging for patients with Parkinson's disease to make use of these structures. In a voice-controlled wheelchair, patients provide orders like "forward," "right," "left," and "stop," and the system recognises them and sends an order to move the wheelchair as needed. The framework is divided into three sections: the image handling, wheelchair control, and machine control components. The image Its processing module includes newly added and updated picture handling software for cameras. The image will be delivered to the Arduino controller, which will prepare it and use Open CV to ascertain the 2-D orientation. The 2-D phases introduced on the electronic wheelchair are controlled by the wheel chair module using the pupil movement organise as the cursor on the LCD.

II. LITERATURE SURVEY

The research challenge, the objectives, and the technique for assessing the suggested solution for this project have all been identified through a review of the literature.

A. "Powered wheelchair controlled by Eye-Tracking technology" from 2006.

A "Powered Wheelchair controlled by Eye Tracking technology" was created in 2006. In order to determine the direction for the wheelchair movement, they used eyewear for tracking the students that are connected to a controller. This technique allows for continuous eye tracking independent of head movement. The user's pupil will appear at a fixed location on the screen.

B.  "Towards an Eye-Activated and Brain-Activated Wheelchair" [2011].

A group of engineers published a report in 2011 outlining the shortcomings of the current eye-controlled electric propelled wheelchair. This paper has raised notice of the possibility that users who are trying to steer their wheelchair by staring at a screen may not be aware of their current location. By include the video stream in the LCD, this wheelchair flaw can be fixed. In order to accommodate the webcam positioned in front of the wheelchair user, this design is implemented.

C.  Voice Controlled Wheelchair Using Arduino, 2016

For those with physical disabilities, the proposed design provides voice command detection. Disability prevents an individual from being independent and forces them to rely on others for daily tasks. Instead of using buttons or gestures, the voice- controlled wheelchair has been built depending on user instructions. Because the wheelchair contains a Node Mcu microcontroller with a built-in wifi module, we can operate it using the user's commands. Users provide the instructions using their smartphones. The instructions that are obtained from the microprocessor regulate how the wheelchair's motors move. With the aid of a buzzer, it alerts and informs users when an obstruction is discovered.

III. PROPOSED SYSTEM

A few highlights were included in the suggested plan for the eye-controlled wheelchair, which had similar development potential and ease of use to a wheelchair powered by a regular control stick. The most notable aspect of this structure is its purpose: all movements are produced only by eye movements and with the assistance of a speech partner. Our wheelchair's setup includes a PC-mounted exhibition. The LCD setup and the eye tracker are mounted atop an armrest. The user will be able to choose the direction they want to move in using the LCD, from forward to in reverse developments anywhere, turn left and right, and develop the wheelchair using voice assistance. The wheelchair-facing camera's live stream is also included in the interface displays to prevent even little visual impairment when typing commands. Directions can be typed only when looking at the screen, preventing any coincidental events when looking about. At this moment, whether the screen is switched on or off, the person's look is coordinated outside of it. The purpose of using the instruction-based transformation is to manage the wheelchair's wheels so that it can demonstrate projects that are more advanced than usual and conventional ones. . It would spark interest in developing day-to-day activities in society by leveraging this new change associated to the conventional framework to detach daily existence.

IV. METHODOLOGY

The framework starts by examining eye movements and then sends signals to the Arduino controller to render an advanced image that depends on OpenCV to infer the pupil's 2-D movement direction. In order to control some devices and activities, such as wheelchairs, the student's movement bearing is also used as the LCD pointer control. The Arduino controller chip for the core microcontroller system is connected to the camera and voice assistance module.

A. Detection of Eyeballs

The camera-taken image of the eye is transferred to the Arduino controller for processing and pupil position detection. The wheelchair's movement is controlled based on the pupil position. Introducing the Arduino, which OpenCV uses for computational image processing. The recorded image is first transformed to binary images, and only then are the images doubled using a simple threshold level. Hough roundabout identification is done to determine the pupil's area. The citation point shifts to the pupil location

B. Processing of Images

PC software using OpenCV is used to simulate the eye movement tracking framework for this execution of the framework. Viola- Jones and Haar Cascade Classifier calculations will be used for face and eye detection. It is used to identify eyeballs, whether they are open or closed. When the eyes were closed at this moment, the framework tried to notify and demonstrate information and sound. Following confirmation, the system is replaced with an Arduino controller board serving as the framework's primary processor. Through the critical consideration for identifying the driver's condition when they are rotating and recognising their eyes. These are currently many consistent tactics that could facilitate the discovery and following of diverse items as picture handling fields continue to advance. The often-used methods for locating objects include Pack of Word models, Histogram-of-arranged angles (HOG), Deformable Parts, Exemplar models, and Viola-Jones. The Viola-Jones approach is a common tool for object recognition. The main characteristic of this approach that makes it so well-known is its ability to prepare slowly but detect swiftly. As a result of the current use of har premise feature channels, augmentations are being avoided. Within a detection window, recognition happens.

C. OpenCV

OpenCV is a free, open-source computer vision programme with integrated libraries. It is employed in commercial products for utilising and monitoring the equipment, and it is worked for a standard basis in computer applications. It makes organisation and editing or modifying code simpler.

D. Voice Assistant

The wheelchair responds to voice commands when they are given by the user. The prerequisite is Wi-Fi or internet access. In order to drive the wheelchair in multiple directions, such as left, right, advance, or halt, the microcontroller first checks for accurate information before giving the motors explicit instructions.

V. FUTURE ENHANCEMENT

This technology can be further enhanced by integrating a gearbox to regulate the wheelchair's speed as well as infrared sensors (IR) to identify obstacles. For the purpose of hearing spoken commands in a noisy environment, noise filters can be implemented. You can use gestures as well. For battery charging and power management, a solar panel can be installed.

VI. ACKNOWLEDGMENT

The authors would like to thank the editors and the unidentified reviewers for their insightful comments that significantly improved the quality of this paper. We would especially want to thank our guide for her unwavering support and encouragement as we wrote this paper.

Additionally grateful are our B M S Educational Trustees, Principal, Head of the Department of CSE's Teaching Faculty, and Non-Teaching Faculty of the B M S Institute of Technology & Management in Yelahanka, Bengaluru.

Conclusion

Most of the smart wheelchairs that were taken into consideration for the survey have yet not hit the market. The suggested system is applicable to hospitals for people with disabilities. Because it is economical, it will benefit the disabled. Since no attendant is required to operate the gadget, it increases the disabled person\'s self-assurance and independence. Additionally, it can be produced in a commercially viable manner.

References

[1] Romil Yash Jain “Study of Implementation of Voice Controlled Wheelchair(2016). [2] Gneo, M., Severini, G., Conforto, S., Schmid, M., & D\' Alessio, T. (2011). Towards a Brain-Activated and Eye- Controlled Wheelchair. International Journal of Bioelectro magnetism, 44-45. [3] Chiu, c.-c., & Yeh, M.-S. (2006) Powered Wheelchair controlled by eye-tracking system. Optica App/icata, 401-412. [4] Pai, S., Ayare, S., & Kapadia, R. (2012). Eye Controlled wheelchair. international Journal of Scientific & Engeering. [5] Abhilash Kodi and Pasha, “EEG – Controlled Wheelchair for ALS patients,” (2013).M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989. [6] K. Arai, R. Mardiyanto, “Eyes Based Electric Wheel Chair Control System,” [7] Dutta Debotosh Bhattacharjee. \"Face detection using generic eye template matching.

Copyright

Copyright © 2022 Hemanth K, Amith P, Anantha Nandan BR, Akshay Aravind, Mrs. Smitha P. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.