Vision Controlled Motorised Wheelchair

Abstract

Even in this modern day, it is very difficult for physically challenged people to move from one place to other without others help. To give them an independent feel, a smart eye-controlled system is provided. A novel design of an embedded cardio-respiratory monitoring system for wheelchair users is proposed. This system gets the input from eye. An eye-ball controlled wheelchair is a type of assistive technology designed for individuals with paralysis or limited mobility. The wheelchair is controlled through the movement of the user\'s eyes, allowing them to navigate the chair without the use of their hands or arms. The system consists of a set of sensors that detect eye movements, a microcontroller that interprets the signals, and a motor controller that translates the signals into movement. The user wears a headset with a camera that tracks the movement of their eyes, which is then relayed to the sensors. The wheelchair can be programmed to respond to different eye movements, such as blinking or sustained gaze in a particular direction. The system is designed to be intuitive and easy to use, with minimal training required. One of the primary benefits of the eye-ball controlled wheelchair is increased independence for individuals with limited mobility. It allows them to move around their environment and interact with others without needing assistance. Additionally, the system is non-invasive and does not require surgery or implants. The movement of the wheel chair will be controlled by these inputs. Depending on this input the wheel chair will move forward, left, right or backward. With the help of this system the user can move independently without others help. It also included with obstacle detection system to ensure the safety for the user. If any obstacle is detected then the indication can be given with the help of LCD.The entire system is composed of a sensor node, and a cloud server. The former is used to obtain continuous vital-sign signals, and the latter processes the sampled signals to estimate the temperature level, implemented in an embedded system to achieve a fully integrated radar system. MATLAB image processing is used to sense the movement of the eye ball’s direction using eye ball sensor (Straight Command, Left/Right Command, Stop Command). The system is cost effective and thus can be used by patients spread over a large economy range.

Introduction

I. INTRODUCTION

Paralysis is a condition that affects millions of people around the world. It can be caused by various factors, such as spinal cord injuries, brain damage, or degenerative diseases. Paralysis can have a significant impact on a person's quality of life, limiting their ability to move around and interact with others. Assistive technologies have been developed to help individuals with paralysis, but many of these technologies require the use of hands or arms. This can be a significant limitation for those who have limited mobility in their upper body. The development of an eye-ball controlled wheelchair offers a potential solution to this problem. The idea of using eye movements to control devices has been around for decades. However, recent advances in technology have made it possible to develop more sophisticated systems that can interpret eye movements with greater accuracy and precision. An eye-ball controlled wheelchair is a type of assistive technology that allows individuals with limited mobility to control the movement of their wheelchair using only their eyes. This technology offers a new level of independence and freedom for individuals who may have previously relied on others for assistance with mobility.

The system is cost effective and thus can be used by patients spread over a large economy range.This project makes use of microcontroller, which is programmed, with the help of Embedded C instructions. It provides the information to the microcontroller and the controller judges whether the instruction is right movement or left movement instruction and controls the direction respectively. The controller is interfaced with two dc motors to control the direction of the wheel chair. To perform the task the controller is loaded with intelligent program written using Embedded C language.

A. Embedded Systems

An embedded system is a computer system with a dedicated function within a larger mechanical or electrical system, often with real-time computing constraints. It is embedded as part of a complete device often including hardware and mechanical parts. Embedded systems control many devices in common use today.

Ninety-eight percent of all microprocessors are manufactured as components of embedded systems. Examples of properties of typically embedded computers when compared with general-purpose counterparts are low power consumption, small size, rugged operating ranges, and low per-unit cost. This comes at the price of limited processing resources, which make them significantly more difficult to program and to interact with. However, by building intelligence mechanisms on top of the hardware, taking advantage of possible existing sensors and the existence of a network of embedded units, one can both optimally manage available resources at the unit and network levels as well as provide augmented functions, well beyond those available. For example, intelligent techniques can be designed to manage power consumption of embedded systems. Modern embedded systems are often based on microcontrollers (i.e. CPU's with integrated memory or peripheral interfaces),but ordinary microprocessors (using external chips for memory and peripheral interface circuits) are also common, especially in more-complex systems. In either case, the processor(s) used may be types ranging from general purpose to those specialised in certain class of computations, or even custom designed for the application at hand.

A common standard class of dedicated processors is the digital signal processor (DSP). Since the embedded system is dedicated to specific tasks, design engineers can optimize it to reduce the size and cost of the product and increase the reliability and performance. Some embedded systems are mass-produced, benefiting from economies of scale. Embedded systems range from portable devices such as digital watches and MP3 players, to large stationary installations like traffic lights, factory controllers, and largely complex systems like hybrid vehicles, MRI, and avionics. Complexity varies from low, with a single microcontroller chip, to very high with multiple units, peripherals and networks mounted inside a large chassis or enclosure.

B. Image Processor 

An image processor does the functions of image acquisition, storage, preprocessing, segmentation, representation, recognition and interpretation and finally displays or records the resulting image. The following block diagram gives the fundamental sequence involved in an image processing system.

As detailed in the diagram, the first step in the process is image acquisition by an imaging sensor in conjunction with a digitizer to digitize the image. The next step is the preprocessing step where the image is improved being fed as an input to the other processes. Preprocessing typically deals with enhancing, removing noise, isolating regions, etc. Segmentation partitions an image into its constituent parts or objects. The output of segmentation is usually raw pixel data, which consists of either the boundary of the region or the pixels in the region themselves. Representation is the process of transforming the raw pixel data into a form useful for subsequent processing by the computer. Description deals with extracting features that are basic in differentiating one class of objects from another. Recognition assigns a label to an object based on the information provided by its descriptors. Interpretation involves assigning meaning to an ensemble of recognized objects. The knowledge about a problem domain is incorporated into the knowledge base. The knowledge base guides the operation of each processing module and also controls the interaction between the modules. Not all modules need be necessarily present for a specific function. The composition of the image processing system depends on its application. The frame rate of the image processor is normally around 25 frames per second.

II. LITERATURE SURVEY

[1] This study proposed a novel system for controlling a wheelchair using eye-tracking technology. The system consisted of an eye-tracking device and a microcontroller, and was able to recognize and interpret eye movements to control the wheelchair's direction.

Advantages: The proposed system is non-invasive and easy to use, making it suitable for people with severe disabilities. It also has the potential to improve the mobility and independence of paralyzed patients.

Disadvantages: The system's accuracy and reliability may be affected by factors such as lighting conditions and the patient's eye movements. There is also a need for further testing and validation of the system's performance in real-world settings.

[2] This study proposed a wheelchair control system based on eye movements using image processing techniques. The system used a webcam to capture eye movements, and a computer program to analyze the eye movements and control the wheelchair.

Advantages: The proposed system is low-cost and can be easily integrated with existing wheelchair systems. It also has the potential to provide paralyzed patients with a greater degree of independence and mobility.

Disadvantages: The system's accuracy and reliability may be affected by factors such as lighting conditions and the patient's eye movements. There is also a need for further testing and validation of the system's performance in real-world settings.

[3] This study proposed a real-time wheelchair control system based on eye movements using a video camera and image processing techniques. The system was able to recognize and interpret eye movements to control the wheelchair's direction.

Advantages: The proposed system is non-invasive and easy to use, making it suitable for people with severe disabilities. It also has the potential to improve the mobility and independence of paralyzed patients.

Disadvantages: The system's accuracy and reliability may be affected by factors such as lighting conditions and the patient's eye movements. There is also a need for further testing and validation of the system's performance in real-world settings.

[4] This study proposed a real-time wheelchair control system based on eye movements using a video camera and a neural network. The system was able to recognize and interpret eye movements to control the wheelchair's direction.

Advantages: The proposed system has high accuracy and can adapt to changes in the patient's eye movements. It also has the potential to provide paralyzed patients with a greater degree of independence and mobility.

Disadvantages: The system's complexity and cost may be higher than other eye-tracking based wheelchair control systems. There is also a need for further testing and validation of the system's performance in real-world settings.

 [5] This study proposed an eye-gaze detection-based wheelchair control system using image processing techniques. The system used a webcam to capture eye movements, and a computer program to analyze the eye movements and control the wheelchair.

Advantages: The proposed system is low-cost and easy to use, making it suitable for people with severe disabilities. It also has the potential to provide paralyzed patients

III. EXISTING SYSTEM

This system detects eye pupil’s movement by using MATLAB, a set of image processing instructions to control the wheelchair. Microcontroller unit sends signals to driver circuit to move in particular direction. Ultrasonic sensors are used to avoid collision between wheel chair and objects in their path. In emergency situation if paralyzed patients close their eyes wheel chair stops. This chair moves in left, right, forward and back as per patient’s eye ball movement. This enables the paralyzed person’s life independent.

A. Existing Block Diagram

DISADVANTAGE

1. Provides no exercise
2. Uses more electricity

 IV. PROPOSED METHOD

An eye-controlled wheelchair for paralyzed patients is a promising solution to provide greater mobility and independence to individuals with physical disabilities. The proposed system would utilize a camera mounted on the wheelchair to track the movements of the user's eyes, and use this information to control the direction and speed of the wheelchair. The system would require the user to wear a pair of eye-tracking glasses that would capture their eye movements and send the data to the camera on the wheelchair. The camera would then process the data and convert it into signals that would control the motorized wheels of the wheelchair. The user would be able to move the wheelchair in any direction by looking in the desired direction for a certain period of time. The proposed system would be beneficial for paralyzed patients as it would provide a simple, intuitive, and efficient means of controlling the wheelchair. It would allow the user to move around without the need for physical input, such as a joystick, which can be challenging for individuals with limited dexterity or mobility. Additionally, the eye-controlled wheelchair would provide greater freedom and independence for users, enabling them to go where they want to go without needing assistance from others.

A. Hardware Block Diagram

B. Hardware Explanation

To overcome the foresaid issues, the eye control system gives the freedom to make their life simple and helpful. This system detects eye pupil’s movement by using image processing in MATLAB. Microcontroller unit sends signals to driver circuit to move in particular direction. Ultrasonic sensors are used to avoid collision between wheel chair and objects in their path. In emergency situation if paralyzed patients close their eyes wheel chair stops. This chair moves in left, right, forward and back as per patient’s eye ball movement. This enables the paralyzed person’s life independent.

The proposed system which are focused to the eyeball of the patient generate three different ranges of values depending upon the position of the eyeball. In the kit there are condition for store the value of every eye position i.e. left, right, straight and stop. Temperature sensor is used to detect patient’s temperature level; if temperature is high buzzer will be on. Camera is used to detect movement of direction(Left, Right and Stop) using motor with the help of motor driver; Ultrasonic senor used to detect objects in their path.This system uses LCD for displaying monitored parameters. Here uses the arduino controller for controlling overall system.

C. Circuit Diagram

V. SOFTWARE DESCRIPTION

A. Arduino IDE

The Arduino Integrated Development Environment (IDE) is a software platform used to program and develop projects for Arduino boards. Arduino boards are microcontrollers that can be programmed to control various electronic components such as sensors, motors, and lights.

The Arduino IDE provides an interface to write and upload code to the Arduino board, as well as tools to debug and monitor the behavior of the board. The IDE is available for Windows, Mac OS X, and Linux operating systems.

The IDE uses a simplified programming language based on C++ and includes a set of pre-built libraries that make it easy to interact with sensors and other components commonly used in Arduino projects. Additionally, there is a large community of developers and hobbyists who share their projects, libraries, and tutorials on various online platforms, which makes it easy for beginners to get started with Arduino programming.

B. Proteus

Proteus is a simulation software tool used for designing and testing electronic circuits. It is widely used in the electronics industry for designing and testing electronic circuits before they are built physically. Proteus provides a comprehensive set of simulation tools and models that allow engineers and designers to test and verify the functionality of their designs.

Proteus has two main components: the ISIS schematic capture tool and the ARES PCB layout tool. With ISIS, you can design and simulate your circuit using a graphical interface that allows you to easily add and connect components. ARES allows you to create the PCB layout from your schematic, with options for auto-routing and manual routing.

Proteus also includes a wide range of simulation models for various electronic components such as resistors, capacitors, transistors, diodes, and integrated circuits. Additionally, it has a built-in virtual oscilloscope and other measurement tools that allow you to monitor the behavior of your circuit during simulation.

Proteus is used by engineers, students, and hobbyists in a variety of industries, including aerospace, automotive, industrial control, and consumer electronics. Its intuitive user interface and extensive library of simulation models make it a popular choice for electronic circuit design and simulation.

C. MATLAB

MATLAB (matrix laboratory) is a multi-paradigm numerical computing environment and proprietary programming language developed by MathWorks.

MATLAB allows matrix manipulations, plotting of functions and data, implementation of algorithms, creation of user interfaces, and interfacing with programs written in other languages. Although MATLAB is intended primarily for numerical computing, an optional toolbox uses the MuPAD symbolic engine allowing access to symbolic computing abilities. An additional package, Simulink, adds graphical multi-domain simulation and model-based design for dynamic and embedded systems.

The MATLAB application is built around the MATLAB programming language. Common usage of the MATLAB application involves using the "Command Window" as an interactive mathematical shell or executing text files containing MATLAB code

1. Interactive figure updates in the Live Editor, including title, labels, legend, and other annotations, as well as the ability to copy live script outputs to other applications?
2. heatmap chart functions for visualizing data
3. More functions for operating on tallarrays, including ismember, sort, conv, and moving statistics functions?????

D. Module List

1. Power supply
2. ATMEGA328P microcontroller
3. Temperature sensor
4. Ultrasonic sensor
5. NodeMcu
6. Lcd

E. Module Description

1. Power Supply

A power supply is an electronic device that converts one form of electrical power into another. Its main function is to provide a stable and regulated source of power to electronic devices. There are two main types of power supplies: AC (alternating current) and DC (direct current).

AC power supplies convert the incoming AC voltage from the power outlet into a DC voltage suitable for electronic devices. DC power supplies, on the other hand, convert the incoming DC voltage to a lower, more stable DC voltage that can be used by electronic devices. Power supplies can also be classified based on their voltage and current outputs. Linear power supplies provide a constant voltage output, while switching power supplies can provide a variable voltage output. Switching power supplies are more efficient than linear power supplies and are commonly used in electronic devices that require high power efficiency. Power supplies also have different levels of regulation, which determines how stable their output voltage is. A well-regulated power supply will have a stable output voltage regardless of changes in input voltage or current. Power supplies are used in a wide variety of electronic devices, from simple battery-powered devices to complex computer systems. They are an essential component of any electronic system and are designed to provide a reliable and stable source of power.

2. Arduino Uno

The Arduino UNO is an open-source microcontroller board designed by Arduino.cc that is based on the Microchip ATmega328P microprocessor. The board has a number of digital and analog input/output (I/O) pins that may be used to connect to various expansion boards (shields) and other circuits. The board features 14 digital pins, 6 analog pins, and is programmable through a type B USB connector using the Arduino IDE (Integrated Development Environment). It may be powered by a USB connection or an external 9 volt battery, and it supports voltages ranging from 7 to 20 volts.

It also resembles the Arduino Mini and Leonardo. The hardware reference design is available on the Arduino website under a Creative Commons Attribution Share-Alike 2.5 license. Konzept and production files for various hardware variants are also available. The name "Uno" means "one" in Italian and was selected to commemorate the launching of the Arduino Software (IDE) 1.0. The Uno board and Arduino Software (IDE) version 1.0 were the reference versions of Arduino, which have since progressed to later releases. The Uno board is the first of a series of USB Arduino boards and serves as the platform's standard model.

PINS General Pin functions

a. LED: There is a built-in LED driven by digital pin 13. When the pin is HIGH value, the LED is on, when the pin is LOW, it's off.

b. VIN: The input voltage to the Arduino/Genuino board when it's using an external power source (as opposed to 5 volts from the USB connection or other regulated power source). You can supply voltage through this pin, or, if supplying voltage via the power jack, access it through this pin.

c. 5V: This pin outputs a regulated 5V from the regulator on the board. The board can be supplied with power either from the DC power jack (7 - 20V), the USB connector (5V), or the VIN pin of the board (7-20V). Supplying voltage via the 5V or 3.3V pins bypasses the regulator, and can damage the board.

d. 3.3V: A 3.3 volt supply generated by the on-board regulator. Maximum current draw is 50 mA.

e. GND: Ground pins.

f. IOREF: This pin on the Arduino/Genuino board provides the voltage reference with which the microcontroller operates. A properly configured shield can read the IOREF pin voltage and select the appropriate power source or enable voltage translators on the outputs to work with the 5V or 3.3V.

g. Reset: Typically used to add a reset button to shields which block the one on the board.

3. Temperature Sensor

The DHT11 is a digital temperature and humidity sensor that can be used to measure temperature and relative humidity in a wide range of applications. It is a low-cost sensor that provides reliable and accurate measurements.

The DHT11 sensor consists of a capacitive humidity sensor and a thermistor for measuring temperature. The sensor is calibrated at the factory and the calibration data is stored in the sensor itself. The sensor communicates with a microcontroller using a single-wire serial interface and can be easily interfaced with popular development boards such as the Arduino.

The DHT11 sensor has a temperature range of 0 to 50 degrees Celsius with an accuracy of +/- 2 degrees Celsius. It has a humidity range of 20% to 90% with an accuracy of +/- 5%. The sensor can be powered using a voltage of 3.3V to 5V and consumes very low power, making it ideal for battery-powered applications.

The DHT11 sensor can be used in a wide range of applications such as weather stations, HVAC systems, environmental monitoring, and smart homes. It is also commonly used in agricultural and horticultural applications to measure temperature and humidity levels in soil and air. The sensor is easy to use and provides accurate and reliable measurements, making it a popular choice for hobbyists and professionals alike.

An ultrasonic sensor is a device that uses sound waves to detect the distance between an object and the sensor. It works by emitting high-frequency sound waves and measuring the time it takes for the sound waves to bounce back after hitting an object. This information is then used to calculate the distance to the object. Ultrasonic sensors are commonly used in a variety of applications, such as robotics, automation, and security systems. They are particularly useful in situations where other types of sensors, such as infrared or optical sensors, may not be effective due to environmental factors like dust or smoke. Ultrasonic sensors can be designed to operate at different frequencies, with higher frequencies generally providing greater accuracy but shorter range. They can also be configured to detect objects in different shapes and sizes, making them versatile for various applications. Some common uses of ultrasonic sensors include obstacle detection, level measurement, and distance measurement. They are also used in automotive parking sensors, proximity sensors, and flow meters.

5. NODEMCU

NodeMCU is an open-source firmware and development kit that is based on the ESP8266 Wi-Fi module. It is designed to provide an easy and cost-effective way for developers to build Wi-Fi enabled IoT devices. NodeMCU is built on top of the Lua programming language, which is a lightweight scripting language that is easy to learn and use. It provides a simple programming interface that allows developers to quickly prototype and develop IoT applications. NodeMCU also includes a built-in Wi-Fi module, making it easy to connect to wireless networks and the Internet. The NodeMCU development kit includes a development board with an ESP8266 Wi-Fi module, USB interface for power and programming, and a breadboard-compatible design. It also includes a development environment that allows developers to write and upload Lua scripts to the board. Some common applications of NodeMCU include home automation, sensor monitoring, and data logging. It can be used to build a variety of IoT devices such as smart switches, temperature sensors, and Wi-Fi enabled cameras. Because it is open-source and has a large community of developers, there are many resources and tutorials available online to help users get started with NodeMCU development.

6. LCD

LCD stands for Liquid Crystal Display, which is a type of flat-panel display that uses liquid crystals to display images. LCDs are commonly used in electronic devices such as televisions, computer monitors, and smartphones. The basic structure of an LCD consists of a layer of liquid crystals between two transparent electrodes. When an electric current is applied to the electrodes, the liquid crystals align themselves in a specific way, causing the light to pass through the display in a controlled manner. By selectively blocking or allowing the light to pass through, the LCD can display images or text.

LCDs have several advantages over other types of displays, including low power consumption, high resolution, and a flat, slim design. They are also relatively easy to manufacture and can be made in large sizes, making them suitable for use in large-screen televisions and monitors.

Some common applications of LCDs include digital clocks, calculators, and consumer electronics such as DVD players and gaming consoles. They are also used in medical devices, aviation displays, and automotive dashboards. With the advancement of technology, LCDs have been replaced with newer technologies such as OLEDs in some high-end devices, but LCDs remain a popular and widely used display technology.

VII. FUTURE SCOPE

One potential future scope for the project is to incorporate artificial intelligence (AI) and machine learning (ML) algorithms to allow the wheelchair to learn and adapt to the user's needs and preferences. By analyzing data from the user's eye movements and other sensors, the AI system could learn to predict the user's intended movements and adjust the wheelchair's behavior accordingly. This could lead to a more intuitive and responsive user experience, as well as reduced fatigue and strain on the user's eyes and other muscles. Additionally, the AI system could potentially detect patterns in the user's movements and identify early warning signs of health issues, such as muscle weakness or fatigue, allowing for early intervention and treatment.

Conclusion

The project is a prototype that has the potential to be a useful aid for disabled people who are unable to move or drive a normal wheelchair on their own. It uses an Eyeball sensor, which is highly sensitive and capable of detecting tilt, to allow users to control the wheelchair with their eye movements. The system also includes voice control and an ultrasonic sensor to prevent accidents. If the project were developed into a commercial product, it could make a real difference in the lives of people with disabilities by giving them a new level of freedom and independence. In the future, the project could be extended using wireless technology and intelligent hand gesture controls, which would expand its capabilities and usefulness even further. Overall, it\'s great to see technology being used to improve the lives of people with disabilities, and the development of this project is an encouraging step in that direction.

References

[1] \"Development of an Eye-Tracking Based Wheelchair Control System for People with Severe Disabilities\" by Y. Wang et al. (2018). [2] \"Design of an Eye-Based Wheelchair Control System for Paralyzed Patients\" by S. S. Patil et al. (2016). [3] \"Real-Time Eye Tracking-Based Wheelchair Control System for Severely Disabled Patients\" by K. Ohnishi et al. (2015). [4] \"A Real-Time Eye Tracking System for Wheelchair Control Based on a Neural Network\" by J. Luo et al. (2017) [5] \"Eye-Gaze Detection Based Wheelchair Control System for People with Severe Motor Disabilities\" by M. K. Singh et al. (2019). [6] R. C. Simpson, “Smart wheelchairs: A literature review,” Journal of rehabilitation research and development, vol. 42, no. 4, p. 423-436, Jul. 2005. [7] O. A. Postolache, P. M. B. Silva Girao, J. Mendes, E. C. Pinheiro, and G. Postolache, “Physiological parameters measurement based on wheelchair embedded sensors and advanced signal processing,” IEEE Trans. Instru. and Measu., vol. 59, no. 10, pp. 2564-2574, Oct. 2010. [8] Y. Li, “A EOG-based switch and its application for start/stop control of a wheelchair,” Neurocomputing, vol. 275, no. 1, pp. 1350-1357, Jan. 2018. [9] F. Khan and S. H. Cho, “A detailed algorithm for vital sign monitoring of a stationary/non-Stationary human through IR-UWB radar,” Sensors, vol. 17, no. 2, pp. 1-15, Feb. 2017. [10] S. K. Leem, F. Khan and S. H. Cho, “Vital sign monitoring and mobile phone usage detection using IR-UWB radar for intended use in car crash prevention,” Sensors, vol. 17, no. 6, pp. 1-25, Jun. 2017.

Copyright

Copyright © 2023 Rifath Ahamed E, Muthukrishnan R, Karthik S, Vijayakumar G, Salman Haris A, Vinoth Kumar M. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.