A Portable Wireless Head Movement Controlled Human-Computer Interface for People with Disabilities

Abstract

This paper describes about an economical head operated computer mouse for people with disabilities. It focuses on the invention of a head operated computer mouse that employs one tilt sensor placed in the helmet to determine head position and to function as simple head-operated computer mouse. The system uses accelerometer based tilt sensor to detect the user\'s head tilt in order to direct the mouse movement on the computer screen. Clicking of mouse is activated by the user\'s eye brow movement through a sensor. The keyboard function is designed to allow the user to scroll letters with head tilt and with eye brow movement as the selection mechanism. Voice recognition section is also present in the head section to identify the small letters which are pronounced by the paralyzed user. This system was invented to assist people with disabilities to live an independent professional life.

Introduction

I. PROBLEM STATEMENT

Restricted operation of computers and laptops for disabled person due to not being able to carry out easy inputs which are required for the use of those gadgets Although, for regular operations the gadgets multiple input devices are being used which can be minimised by using a single input device with the motor ability of the disabled person.

II. INTRODUCTION

Objective: OWING to the lack of appropriate input devices, people with disabilities often encounter several obstacles when using computers. Currently, keyboard and mouse are the most common input devices. Due to the increasing popularity of the Microsoft Windows interface, i.e., Windows 98 and NT, computer mouse has become even added important. Therefore, it is necessary to invent a simple mouse system for people with disabilities to operate their computers.

People with spinal cord injuries (SCIs) and who are paralyzed have increasingly applied electronic assistive devices to improve their ability to perform certain essential functions. Electronic equipment, which has been modified to benefit people with disabilities include communication and daily activity devices, and powered wheelchairs. From our literature analysis there are many computer input devices are available. Finger mounted device using pressure sensors, but no hardware has been realized so far and it needs physical kind of interaction with computer system. A wide range of interfaces are available between the user and device and the interfaces can be enlarged keyboards or a complex system that allows the user to operate or control a movement with the aid of a mouth stick, However, for many people the mouth stick method is not accurate and comfortable to use. An eye imaged input system, electrooculograpy (EOG) signals , electromyogram (EMG) signals Electroencephalogram (EEG) signals are capable of providing only a few controlled movements have slow response time for signal processing and require substantial motor coordination. In infrared or ultrasound-controlled mouse system (origin instruments’ head mouse and prentke romish’s head master) , etc. There are two primary determinants that are of concern to the user. The first one being whether the transmitter is designed to aim at an effective range or not with respect to receiver, the other one being whether the cursor of computer mouse can move with his head or not. These considerations increase the load for people with disabilities. Thus, alternative systems that utilize commercially available electronics to perform tasks with easy operation and easy interface control are sorely required.

The ability to operate a computer mouse has become increasingly important to people with disabilities especially as the advancement of technology allows more and more functions to be controlled by computer. There are many reasons for people with disabilities to operate a computer.

For instance, they need to acquire new knowledge and communicate with the outside world through the Internet. In addition, they need to work at home, enjoy leisure activities, and manage many other things, such as home shopping and internet banking. This research focuses on a tilt sensor controlled computer mouse. The tilt sensors or inclinometers detect the angle between a sensing axis and a reference vector such as gravity or the earth’s magnetic field. In the area of medicine science, tilt sensors have been used mainly in occupational medicine research. For example, application of tilt sensors in gait analysis is currently being investigated. Andrews et al. used tilt sensors attached to a floor reaction type ankle foot orthosis as a biofeedback source via an electrocutaneous display to improve postural control during functional electrical stimulation (FES) standing. Bowker and Heath recommended using a tilt sensor to synchronize peroneal nerve stimulation to the gait cycle of hemiplegics by monitoring angular velocity. Basically, tilt sensors have potential applications of improving the abilities for persons with other disabilities . The system uses MEMS accelerometers to detect the user’s head tilt in order to direct mouse movement on the computer screen. Clicking of the mouse is activated by the user’s eye-brow movement through a sensor. The keyboard function is designed to allow the user to scroll letters with head tilt and with eye brow movement as the selection mechanism. Voice recognition section is also present in the head section to identify the small letters which are pronounced by the paralyzed user. The tilt sensors can sense the operator’s head motion up, down, left, and right, etc. Accordingly, the cursor direction can be determined

III. LITERATURE SURVEY

A. The Direction of Mouse

Gesture controlled mouse is a device in which we can move the mouse by giving the direction from our wrist and turn the direction of mouse through it. After using a accelerometer we can able to move and change the direction of our mouse and by the help of accelerometer we can also change thespeed of the mouse. This is a new technology for controlling any device through our wrist movement. After upgrading it we can make many type of other devices which can be very useful for the human being can create a turn in our life. It can be used at many places and also be the part of the future. The gesture will be main part as the movements of our body will play the important role for operations. A stationary point will be taken as the directions will be in x-axis, y-axis and as well as the z-axis. An accelerometer will be used as the direction indicators.

B. Mouse Control Using Image Processing

The technology is growing up the scope of Human Computer Interaction is increasing. We used mouse, keyboard, trackpad, joysticks to communicate with the computer. Nowadays, they have used a wireless devices example wireless mouse or wireless keyboard. One disadvantage of these devices is required Bluetooth hardware attached with computer and respective software device also. The new technology touch screen comes in the market which is used in HCI (Human Computer interaction) but a cost of a touch screen is not cheap. That’s why we cannot use this technology everywhere. The proposed system has no such disadvantage. In this System, we go through three phases. Object Detection, Image Processing and Call Event. Our main aim is to control the mouse event like mouse pointer move, left click and right click without any hardware use such as a mouse or wireless mouse. No extra hardware is required other than webcam which is connected with laptop or computer.

C. Gesture Controlled Mouse

Recently, due to digitization in every field, data processing speeds have increased dramatically, with computers being ad vanced to the levels where they can assist hu-mans in complex tasks. Yet, input technologies seem to cause a major bottleneck in performing some of the tasks by under-utilizing the available resources and usage restriction as a result of cost constraint. Compared to the optical mouse which offers a limited range of length of connecting cable and also they require a surface to work on, the wireless in not of much use other than allowing for a desktop with fewer wires attached. HUMAN COMPUTING INTERACTION (HCI) is one of the important area of research were people try to improve computer technology. Gesture is a very natural human communication capability.

IV. GENERAL PROJECT DESCRIPTION

The design of an interface between humans and computers has a direct impact on the efficiency of the interaction between the two parties. HCI emerged in the 1980s with the popularization of personal computing.

Computers were no longer being built just for experts, and the goal of HCI was to make all interaction with computers easy and efficient for broad groups of users at different skill levels.

HCI practitioners observe the ways in which people interact with computers and then design technologies to help them use computers more efficiently. The goal is to minimize interaction cost–the amount of physical and mental effort a user must exert when using the technology–and make interactions more human. As a discipline, HCI is a forerunner of user experience design. The HCI field seeks to improve human-computer interaction by improving the functionality, reliability, usability, and comfort of computer interfaces.

Iterative design is one of the foundational principles of HCI. Once a designer has gained an understanding of his or her target audience, their tasks, and the empirical measurements surrounding an interaction, designers follow several iterative design steps: design the user interface; conduct user testing; analyze the results of testing; repeat. The iterative design process is repeated until a user-friendly interface is created.

The interaction between a machine and a human can be facilitated in multiple ways. Generally, it’s possible to utilize one or more human senses to form the basis of a UI, such as tactile UI (touch), visual UI (sight), and auditory UI (sound). HCI practitioners find the optimal combination that fits the purpose of the product. For example, for a mobile app, this might be a combination of visual UI and auditory UI. The rise of mobile devices has prompted UX practitioners to focus on making HCI accessible to large groups of users including cognitively or physically impaired users.

 Another challenge of HCI design is the need to create omnichannel experiences while considering emerging technologies like VR devices. A typical user might have a range of devices that they interact with, and they should be able to switch from device to device and continue the user journey in a comfortable and effective way. This means HCI practitioners now need to think about every digital product in a holistic way and place even more focus on the needs and wants of end-users.

An accelerometer is an electromechanical device that will measure acceleration force. It shows acceleration, only due to cause of gravity i.e. g force. It measures acceleration in g unit.

On the earth, 1g means acceleration of 9.8 m/s2 is present. On moon, it is 1/6th of earth and on mars it is 1/3rd of earth.

Accelerometer can be used for tilt-sensing applications as well as dynamic acceleration resulting from motion, shock, or vibration.

ADXL335 module

1. The ADXL335 gives complete 3-axis acceleration measurement.
2. This module measures acceleration within range ±3 g in the x, y and z axis.
3. The output signals of this module are analog voltages that are proportional to the acceleration.
4. It contains a polysilicon surface-micro machined sensor and signal conditioning circuitry.

C. Arduino Nano

The Arduino Nano is a small Arduino board based on  ATmega328P or ATmega628 Microcontroller. The connectivity is the same as the Arduino UNO board.

The Nano board is defined as a sustainable, small, consistent, and flexible microcontroller board. It is small in size compared to the UNO board. The Arduino Nano is organized using the Arduino (IDE), which can run on various platforms. Here, IDE stands for Integrated Development Environment.

The devices required to start our projects using the Arduino Nano board are Arduino IDE and mini USB. The Arduino IDE software must be installed on our respected laptop or desktop. The mini USB transfers the code from the computer to the Arduino Nano board.

D. HC-05 Bluetooth Module

Bluetooth is a technology for wireless communication. It is designed to replace cable connections.It uses serial communication to communicate with devices. It communicates with microcontroller using serial port (USART). Usually, it connects small devices like mobile phones, PDAs and TVs using a short-range wireless connection to exchange documents. It uses the 2.45GHz frequency band. The connection can be point-to-point or multi-point where the maximum range is 10 meters. The transfer rate of the data is 1Mbps.

HC-05 Bluetooth module provides switching mode between master and slave mode which means it able to use neither receiving nor transmitting data.

Comparing it to the HC-06 module, which can only be set as a Slave, the HC-05 can be set as Master as well which enables making a communication between two separate Arduino Boards.

You can use Bluetooth module simply for a serial port replacement to establish connection between MCU, PC to your embedded project and etc.

E. IR Sensor

The IR sensor or infrared sensor is one kind of electronic component, used to detect specific characteristics in its surroundings through emitting or detecting IR radiation. These sensors can also be used to detect or measure the heat of a target and its motion. In many electronic devices, the IR sensor circuit is a very essential module. This kind of sensor is similar to human’s visionary senses to detect obstacles.

The sensor which simply measures IR radiation instead of emitting is called PIR or passive infrared. Generally in the IR spectrum, the radiation of all the targets radiation and some kind of thermal radiation are not visible to the eyes but can be sensed through IR sensors.

In this sensor, an IR LED is used as an emitter whereas the photodiode is used as a detector. Once an infrared light drops on the photodiode, the output voltage & resistance will be changed in proportion to the received IR light magnitude.

F. CP 2102 USB To TTL Module

CP2102 USB To TTL Converter Module is a small and easy USB to TTL converter for STC microcontroller download DVD/ hard disk/router/GPS upgrade, which is based on CP2102 USB-to-UART bridge controller chip. This compact MLP-28 package chip consists of a USB controller, USB transceiver (no external resistors required), oscillator (no external crystal required), 1024-Byte EEPROM (to keep data such as vendor ID, product ID, serial number, power descriptor, release number, and product description strings), and asynchronous serial data bus (UART) with full modem control signals. This module has a 500mA auto recovery fuse path, preventing to burn computer USB due to power failures. It can directly plug into the PC USB port and be supplied by this port and no need external power supply. It has an LED indicator to show power and transceiver stats. Also, this LED indicator shows the transceiver speed by speed flashing, so the higher the intensity means the lower the baud rate. This converter supports many Operating Systems such as Windows XP, Windows Vista, Windows server 2003 & 2000, Mac OS-X/OS-9, and Linux.

 G. 9v Battery and ON- OFF Switch

The 9V battery is an extremely common battery that was first used in transistor radios. It features a rectangular prism shape that utilizes a pair of snap connectors which are located at the top of the battery. A wide array of both large and small battery manufacturers produce versions of the 9V battery.

H. Switch

Among these different types of commands, the switch allows the operator to intervene directly on the application by modifying its status, opening or closing a circuit.

The switches we are dealing with can be unipolar or bipolar. The unipolar switch opens and closes the phase contact, interrupting only one pole of the electrical connection. Unlike the unipolar switch, the bipolar switch is an electrical component that interrupts the current on both poles of the circuit, i.e. both the phase and the neutral. The on-off switch is most often marked with an O/I binary code symbol on the on/off switch of the device on which it is placed

I. Advantages And Applications

1. Advantages

a. Helping people who primarily work with computer mouse & Keyboard.

b. Ease of use and flexibility.

c. Multiple click options.

d. Provides a sense of independence.

e. Portable and Adaptable.

f. Headmouse for People with Disability.

2. Applications

a. Helping People who primarily work computer mouse and keyboard.

b. headmouse For People With Disability.

V. FUTURE SCOPE

Human Computer Interaction is gaining mass popularity in the present days. This project provides a greater scope for improvement in the near future. Effective control in increasing of writing speed is still one of the sectors to be improved in future. Better methods of transmission and reception channel can also be developed on further experiment. In the future this interface can be introduced into many control systems at home such as powered wheelchairs, telephones, and appliances with great potential demanded by the market.

Conclusion

The main advantage of this project is to eliminate the disability for the handicapped people so that they can enjoy this world as a normal human being are enjoying. Those people can control or operate all the computer application by the gesture of their eye movement and the interactive application are done by their tooth click and also gaming, swapping, page scrolling, etc. are also done using their head movement by placing a MEMS (Micro-Electro Mechanical System). The complete replacement of wired communication It finds the solution to the disabled person to operate the computer fully with the enabled mode. The HCI (Human Computer Interface) is an evolving area of research interest nowadays. This project aims to be a convenient process for helping out the disabled to operate computers. These systems can also be used in other application like robotics efforts, in process to make the device cost effective and more complex thereby reducing the size. Thus we have developed a real hand free mouse. This project will be very effective and accurate using of both MEMS and eye blink sensors as a wireless mouse for future.

References

[1] http://www.tutorialspoint.com/human_computer_interface/interactive_devices.html [2] Gray b. and Adrian k. “Learning OpenCV computer vision with library” published by O’Reilly [3] Kaehler a. and Bradski g. “ Learning Open CV 3 Computer vision in C++ with the open cv library ” Second Edition published by O’Reilly [4] http://opencvsrf.blogspot.in/2011/09/capturing -images-videos.html [5] https://www.kirupa.com/design/little_about_c olor_hsv_rgb.html [6] http://docs.opencv.org/trunk/d9/d61/tutorial_p y_morphological_ops.html [7] Debasis Samanta, “Object-Oriendted Programming with c++ and java ” page no150 Phi Learning Pvt. Ltd 8. [8] https://msdn.microsoft.com/enus/library/windows/desktop/ms646260(v=vs. 85).aspx [9] http://www.cplusplus.com/forum/beginner/13 0487

Copyright

Copyright © 2023 Saurabh Barve, Prof. Megha Beedkar, Poonam Gawale, Rajan Jadhav. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.