Gesture Controlled Magical Keyboard using Arduino

Abstract

Every component of a computer has evolved since its inception. Our daily lives are made very convenient and simple by computers. These days, a controller with several keys is used to operate these machines. A keyboard has several restrictions and resource requirements, but it also has a wide variety of physical features for the keys. However, the in-touch keyboard, which developed from a physical keyboard, has a lot of disadvantages. The Internet of Things (IoT) architecture is used by the remote-based gesture keyboard, which also includes an accelerometer, Arduino, and machine learning technology to force the system to act in a particular way. In current work, the detailed review and experimentation of Gesture Controlled Magical Keyboard is presented. The idea behind the gesture-controlled keyboard is that it allows users to enter text and numbers into text editors by moving their hands in a particular way to represent a character in the air using an Arduino board. The Arduino will function as a remote control, transforming airborne movements into text that is shown on a computer screen.

Introduction

I. INTRODUCTION

The essential field of gesture recognition uses computer vision techniques and algorithms to describe human gesture. There are many physiological movements that can result in gesture, but the face and hands are typically where they are produced. Gesture recognition refers to the entire process of following a gesture to its representation and then translating it into a usable command [1]. Clearly, one of the most important tools that people have for enhancing their non-verbal communication and enhancing their capacity for learning is the movement of their upper limbs, particularly their hands. Since the 1980s, tracking human hand movements has been a focus of research [2]. Hand gestures are a further means of communication with machines like computers or robots. We can use hand gestures to operate some computer features, such as playing or pausing a video, navigating left or right through a photo slide show, scrolling up or down a web page, and more, rather of using a keyboard, mouse, or joystick [3]. Gesture-based interaction has entered our daily lives as a result of the growing use of smart phones, but we have not yet fully utilized its potential. The user is given a fresh kind of engagement that mimics their real-world experience through gestures. They don't feel forced and don't call for any extra equipment [4]. The keyboard is an integral part of the computer system because it allows users to enter data by pressing a variety of keys. Today's generation prefers screen touch keyboards over gesture keyboards, which are reserved for usage by those with physical disabilities. The physical keyboard comes in several different layouts, including AZERTY, QWERTY, Dvorak Colemak, Maltron, and JCUKEN [5].

Numerous methodologies have been developed in the past to address the hand gesture recognition issue. The methods utilised in the earlier research have been identified based on the literature review that was done. Electronic gadget size is rapidly shrinking as a result of new innovations and improvements in computer technology. As a result, these gadgets require a new input interface. We can decrease the amount of workspace needed by utilising vision technology and allowing users to operate the gadgets with simple hand motions [6]. Abhijit M. et. al has developed a gadget [7]. For a gesture-controlled system, GESTO, a hand glove containing an accelerometer and gyroscope sensor, is used. There are transmitting and receiving sections in this system. Rashmi et. al [8] showed the gesture-controlled device enables the user to manipulate the mouse pointer with just his fingertip and carry out all mouse pointer-related tasks. A Beagle board-XM, a webcam, some colour bands, and a link to the PC make up the system. The thumb and index finger of the user of the proposed gadget must be covered by two distinct coloured bands. The user must wear a Magic Ring on his index finger in order to use the human-computer interface system as designed in Nidhibahen Patel and Dr. Selena (Jing) He’s research [9], an acceleration detecting processor, combined with wireless communication sensors, and Magic Ring (MR), were previously created. The acceleration sensing processor is essential to the functioning of the system as a whole. The PC stores the corresponding acceleration value whenever the user makes a certain gesture while holding the gadget in his hand. Later, the function that was assigned to that specific value is carried out. These hand movements include swinging in all directions, pushing or pulling, sliding, shaking, rotating the fingers, and other hand motions. Cheng et. al [10] suggested employing three-dimensional depth sensors to recognize hand gestures. Within this application, there are many other types of fields, such as static hand motion, 3D hand modelling, and hand route gesture. This study concentrated on gesture identification approaches, and it also discussed the fields in which those methods are used.

II. TECHNICAL REQUIREMENTS:

A. Hardware Used

1. Atmega Microcontroller

It is 8-bit microcontroller with RISC (Reduced Instruction set). It has standard features like on-chip ROM (Read Only Memory), Data RAM (Random Access Memory), data EEPROM (Electrical Erasable Programmable Read Only Memory), Timers and Input / Output Ports, along with extra peripherals like Analog to Digital Converters (ADC), Serial Interface Ports etc. They have 120 and more instruction set and program memory ranges from 4K to 256K Bytes.

2. ADXL Sensor

ADXL345 is a small 3-axis accelerometer.  This module measures Acceleration with range +3, -3 in the x,y& z axis. The output signals of this module are proportional to acceleration and are analog voltages. An accelerometer is an electromechanical device that measures acceleration force.

3. Bluetooth HC 05

HC-05 is a Bluetooth module which is designed for wireless communication. It is used for many applications like wireless headset, game controllers, wireless mouse, wireless keyboard and many more consumer applications. It uses serial communication to communicate with devices. It communicates with microcontroller using serial port (USART). It has range up to <100m which depends upon transmitter and receiver, atmosphere, geographic & urban conditions.

4. LCD Display

A liquid-crystal display (LCD) is a flat-panel display that uses the light-modulating properties of liquid crystals combined with polarizers.  LCDs are available to display arbitrary images or fixed images with low information content, which can be displayed or hidden. LCDs can either be normally on positive or off negative, depending on the polarizer arrangement.

???????5.  Switches

A switch is an electrical component that can connect or disconnect the conducting path in an electrical circuit. Switches are made in many different configurations they may have multiple sets of contacts controlled by the same knob or actuator, and the contacts may operate simultaneously, sequentially, or alternately.

???????6. Resistors

A resistor is a passive two-terminal electrical component that implements electrical resistance as a circuit element. High-power resistors that can dissipate many watts of electrical power as heat may be used as part of motor controls, in power distribution systems. Fixed resistors have resistances that only change slightly with temperature, time or operating voltage.  Variable resistors can be used to adjust circuit elements or as sensing devices for heat, light, humidity, force, or chemical activity.

???????7. Capacitors

A capacitor is a two-terminal, electrical component along with resistors and inductors, they are one of the most fundamental passive components we use. A capacitor is a device that stores electrical energy in an electric field. It is a passive electronic component with two terminals. The effect of a capacitor is known as capacitance.  

???????8.  Transistors

A transistor is a mini semiconductor that regulates or controls current or voltage flow. Transistors consist of three layers, or terminals, of a semiconductor material, each of which can carry a current. A transistor has only one circuit element on its own. In small quantities, transistors are used to create simple electronic switches. They are the basic elements in integrated circuits (ICs), which consist of a large number of transistors interconnected with circuit. In large numbers, transistors are used to create microprocessors where millions of transistors are embedded into a single IC.

???????9. PCB

A printed circuit board (PCB) or printed wiring board (PWB) is a laminated structure of conductive and insulating layers. PCBs have two complementary functions. The first is to affix electronic components in designated locations on the outer layers by soldering. The second is to provide reliable electrical connections between the component's terminals in a controlled manner referred to as PCB design. PCBs can be single-sided i.e., one copper layer, double-sided i.e., two copper layers on both sides of one substrate layer, or multi-layer i.e., outer and inner layers of copper, alternating with layers of substrate.

???????10. LED’s:

Light-emitting diode (LED) is a semiconductor device that emits light when an electric current flows through it. When current passes through an LED, the electrons recombine with holes emitting light in the process. LEDs allow the current to flow in the forward direction and block the current in the reverse direction. They are light sources with smaller size, longer lifetime, lower energy, consumption.

III. ARCHITECTURE

We need to give about 12V power supplies to the hardware through AC supply. The LCD is used to display the transmission of gestures. We have used Bluetooth to maintain the connection between our device and cell phone. The buzzer will buzz after a successful connection between our device and cell phone. We have used the ADXL335 sensor through which gestures will pass. LEDs are used to turn on and off the light of a particular room. All these are connected to Arduino.

IV. METHODOLOGY

When we start ,If x sensor is greater than 340 direction then serially data puts on application,if no then it checks whether if x+  is greater than direction then it serially put the dada on Application if no then it checks whether y- greater than direction then serially data puts on application if no then it checks whether y + greater than direction then it serially data puts on application or if no then it's stop the functioning and after the serially data is put on application then it directly stops the functioning after getting result.

V. FUTURE SCOPE

Gesture can be given by face, hand so by using facial gesture we can develop a vehicle driver alert, to make driver alert who are falling asleep. The gesture control device acts as the bridge between computer and human so it can play important role in interaction between computer-human in future. The device can be controlled just by hand so it can be effectively work in the fields of industry to operate machines and also in medical .The gesture device can future be converted into gesture hand and can be used in robotics as a gesture robot hand. By testing gesture along with Augmented Reality, Virtual Reality and coding in VC/VC.Net the speed could be improved.

VI. ACKNOWLEDGEMENT

Authors want to acknowledge Inquest: Mechanical Engineering Student Research Forum of D.Y. Patil College of Engineering and Technology for guidance to frame this research article.

Conclusion

The project is based on IOT where objects are embedded with objects, sensors and other technologies for exchanging of data. In this project we have focused on creating of keyboard by having some reference of Microsoft Kinect which have many functions like voice, body and face recognition. From this project it can be concluded that: Gesture Control technology is based on recognition of gestures it is a way for computer to understand human body language; it also builds a bridge between human and computer. Having reference of Microsoft’s, we have designed a Gesture Control Keyboard this keyboard is found to be convenient device. This is developed using a band and sensors where all the alphabet, numbers and special characters can be controlled. This project can operate the keyboard, light of room, hall etc. without actual or physical interaction with them but just by the band key. For making the project more compatible and user friendly a voice module reads / speaks all the things that are displaced and selected on the screen. The best use of this project is for society i.e. for the physically challenged people. It can also be used in gaming, while giving presentation. As the world is on the way of automation the IOT based image processing and many more projects are existing but the gesture control keyboard will give brief idea of human interaction with computer. e.g.: Just a band key will switch on and off the light.

References

[1] Pradnya Kedari, Shubhangi Kadam, Rutika Chaudhary, Rajesh Prasad, “A Survey Paper on Controlling Computer using Hand Gestures”, International Research Journal of Engineering and Technology (IRJET). [2] Thomas G. Zimmerman, Jaron Lanier, Chuck Blanchard, Steve Bryson and Young Harvill, “A hand gesture interface device”, ACM-0-89791-213-6/87/0004/0189, 1987. [3] Sawardekar, Gaurav and Thaker, Parthil and Singh, Rishiraj and Gaikwad (Mohite), Vaishali, “Arduino Based Hand Gesture Control of Computer Application”, (October 29,2018), SSRN: https://ssrn.com/abstract=3274460. [4] Controlling Computer Application Using Hand Gestures Using Arduino, Andrea Attwenger published in IEEE. 2012. [5] Shakunthaladevi, M., R. B. Revathi, “Real time hand gesture Recognition using AVR microcontroller”, International Conference on Information Communication and Embedded Systems (ICICES2014). [6] Recognizing hand gestures for human computer interaction, Dushyant Kumar Singh published in IEEE. 2010. [7] Abhijit M., Anjana Nair, Jikhil John, Shabasbasheer, Munna Basil Mathai, “Hand Gesture Based Home Automation.,International Journal of Advance Research in Electrical”, Electronics and instrumentation Engineering(IJAREEIE) 2017 Vol. 6, Issue 3. [8] Rashmi Vashisth, Akshit Sharma, Shantanu Malhotra, Saurabh Deswal, Aman Budhraja, “Gesture Control Robot Using Accelerometer”, 4th IEEE International Conference on Signal Processing, Computing and Control (ISPCC 2k17), Solan, India. [9] Nidhibahen Patel, Dr. Selena (Jing) He, “Survey on Hand Gesture Recognition Techniques, Methods and Tools”, International Journal of Research in Advent Technology, 2018 Vol.6, No.6. [10] Cheng, Hong, Lu Yang, and Zicheng Liu, “ Survey on 3D hand gesture recognition”, IEEE Transactions on Circuits and Systems for Video Technology 26, no. 9 1659-1673.

Copyright

Copyright © 2022 Dhanashree R. Patil, Saloni M. Padwal, Suchanta B. Ravan, Rajavi S. Sankpal, Tejaswini M. Gavali, Pratiksha P. Desai. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.