Gesture Controlled Robot with Robotic Arm

Abstract

In today\'s world, industry and daily routine tasks are increasingly attracted to and implemented through robotic automation. Well, this study proposes a method for replacing buttons and joysticks with a more novel technology, namely, commanding the entire Robotic Arm with the user\'s hand movement, motion, or gesture. The goal is to alter how people think about remote controllers for manually driven robotic arms. Pick and place robots are a type of technology used in the industrial industry to execute pick and place tasks. The system is constructed in such a way that it reduces human error and intervention, resulting in more exact work. The system is designed with a simple, flexible, and minimum control technique in mind. There are numerous domains in which human intervention is problematic, yet the process in question must be operated and regulated, which leads to the application of robots. According to the literature, pick and place robots are conceived and deployed in a variety of industries, including the bottle filling industry, surveillance to identify and eliminate bombs, and so on. The goal of the project is to create a gesture-controlled robot with a robotic arm that can perform any pick-and-place tasks. A radio frequency transmission is used to control the pick and place robot. For the gripper open and close, up and down, forward and backward, base clockwise and counterclockwise, the Robotic Hand has certain independent commands. Four Omni wheels support the robotic arm\'s displacement on the chassis. The implemented robotic arm has four degrees of freedom. Line follower, wall hugger, obstacle avoider, metal detector, and other capabilities can be added to this robot to increase its versatility.

Introduction

I. INTRODUCTION

The goal of this work is to design and implement a "Gesture Controlled Robot with Robotic Arm." The system is made up of three parts: an accelerometer, a robotic hand, and a platform. It's basically an Accelerometer-based Robotic Hand system that uses RF signals to operate a Robotic Hand utilizing a compact and low-cost 3-axis (DOF's) accelerometer. The Robotic Hand is attached to a mobile platform that is controlled wirelessly by an accelerometer.

An accelerometer is installed or connected to a human hand to capture its gesture. When the switch is in the on position, the robotic arm moves appropriately, and when the switch is in the off position, the platform moves accordingly. The robotic arm and platform are synced with the user's and operator's hand motions and postures. PICK and PLACE / DROP, RAISING and LOWERING, ROTATING CLOCKWISE/ COUNTERCLOCKWISE the items are some of the actions done by Robotic Hand.

Also, the motions performed by the platform are: FORWARD, BACKWARD, RIGHT and LEFT.

The primary goal of this work is to develop and build a volatile robotic arm with four degrees of freedom (DOF) capable of performing a basic reflexive grab on a wide range of items. The system is based on a servo, point-to-point, and cylindrical robot construction with four degrees of freedom (DOF). This method focuses mostly on the problem of gripping things of various forms rather than manipulating or building objects. This sort of gripping device has a wide range of uses, including impaired object retrieving systems, planetary and undersea exploration, and robotic surgery. This study is primarily concerned with item selecting and dropping. It is based on hand gestures and uses glove-based technologies.

II. RELATED WORK

Today, various robotic hands are employed in robotics research, each with its own set of capabilities and design requirements. This section provides a summary of several recent extensively utilized and/or influential robotic arms. Several research initiatives in the field of robotics have been aimed toward recognizing human gestures. Among the most common systems are:

A. Gesture Recognition Based on Vision

It basically worked in the field of Service Robotics, and the researchers eventually created a cleaning robot. They created a hand gesture-based interface for controlling a mobile robot with a manipulator. A camera will be used to follow a person and identify arm motion motions. A quick, adaptive tracking algorithm allows the robot to easily track and follow a human through changing illumination conditions in office surroundings. It will clean and utilize a camera to track its progress.

B. Sensor Recognition for Motion Capture

This identification approach enabled the development of an accelerometer-based system for wireless communication with an industrial robotic arm. The robotic arm in this project is powered by an ARM7-based LPC1768 core. MEMS is a three-dimensional accelerometer sensor that captures human-arm motions and generates three separate analogue output voltages in three spatial axes. Two flex sensors are employed to detect gripper movement.

C. Active Tracking Mechanisms-Based Finger Gesture Recognition System

The primary goal of the system presented by the author (based on the recognition approach) is to make it possible for it to communicate with a portable device or a computer through the recognition of finger motions. Apart from movements, voice may also be used as a way of interaction, hence this system can be used as part of a Perceptual User Interface (PUI). The technology has the potential to be utilized for Virtual Reality or Augmented Reality systems.

D. Gesture Recognition Using Accelerometers

In a relatively short period of time, this Gesture Recognition technology has grown in popularity. The low moderate cost and relatively compact size of the accelerometer make it a useful instrument for detecting and recognizing human body motions. Several investigations on the identification of gestures using acceleration data using Artificial Neural Networks have been undertaken (ANNs).

III. LITERATURE SURVEY

The book by John Iovine describes several elements of constructing a robot. It is concerned with various forms of arm design, controlling techniques, vehicle design, and so on. The operation and control of robots is described in this book by ER. Rajput. The Arduino cookbook discusses the intricacies and techniques of connecting hardware components such as DC motors, servo motors, and RF transmitters and receivers.

Other sources in the references section explore similar concepts in their own domains, such as color identification and segregation robots, surveillance robots, pick and place robots controlled by android, and so on.

IV. PROBLEM STATEMENT

The pick and place robot is being used to help with sorting, carrying heavy goods, and other tasks. Heavy goods are often transferred using labour, and if the activity is performed for an extended length of time, the operator may get injuries. The operator will no longer have to bend and lift large weights by employing the robot, reducing injuries and enhancing job productivity. Every operator, whether little or large, will make mistakes. In today's industrial environment, no mistakes can be tolerated. Because every mistake costs interns time, money, and materials. 

V. OBJECTIVE

The main objectives of this project are 

1. To control the displacement of the robotic arm so that the arm can be used to pick and place the elements from any source to destination. 
2. To control the displacement and movement of robotic arm using RF Transmitter and Receiver.
3. To implement a robotic arm with two degrees of freedom.

VI. OVERALL DESIGN OF THE SYSTEM

A. Proposed Block Diagram

The total system design is depicted in fig. Flex Sensors, Servo Motors, Arduino UNO, Arduino Nano, RF transmitter, RF receiver, Motor Driver L293D, accelerometer, and a power supply are all included.

The pick and place robotic arm is made up of a robotic arm that is mounted to a moving vehicle (chassis). The vehicle can go on any sort of surface, whether smooth or bumpy. The pick and place robot includes four motors for the chassis and four servo motors for the pick and place function. The pick and place arm is made consisting of an arm assembly with a jaw that can move up and down, forward and backward, and spin clockwise and counterclockwise.

There are four motors for the arm assembly:

1. For clockwise and anticlockwise rotation
2. For upward and downward motion
3. For forward and backward motion
4. For opening and closing the gripper

A motor driver IC and an Atmega328 microcontroller are utilized to control the motor. A wireless play station provides the input or controlling signal, which is interfaced with the microcontroller through an RF receiver module. When a signal is delivered from the play station, it is decoded in the controller and the necessary controlling signal is given to the system's actuators (dc motors or servo motors).

B. Major Parts of Robotic Hand

In this paper, we design a robotic arm with four degrees of freedom, which is able to pick the different object and place them at the different location. Based on functionality, the system has been divided into the following parts: -

1. Robotic arm 
2. Platform  
3. Communication system 

All these parts are discussed below:

a. Robotic Arm: This is the most important aspect of the system since it is responsible for picking up and dropping project tasks. The robotic arm is outfitted with a Gripper (for selecting and putting things) and an Arm (for raising and lowering objects and rotating). Both the Arm and the Gripper are outfitted with Servo Motors to control movement. These motions are coordinated with the user's hand gestures when controlling the Robotic Arm. The lowest point servo is mounted in such a way that it moves the top base horizontally from 0 to 180 degrees based on the RF module values.

b. Platform: The Platform is nothing more than the component of the project on which the Robotic Arm is installed. The platform is powered by DC motors and moves in synchronization with the user's other hand movements for controlling the Robotic Arm. The accelerometer is installed on the user's one hand, capturing hand motions. Also, the other hand movements that cause the platform to move. This section of the project is responsible for moving the complete project from one location to another.

• Gesture 1: To make the platform move in Forward direction
• Gesture 2: To make the platform move in Backward direction 
• Gesture 3: To make the platform take a turn towards Right
• Gesture 4: To make the platform take a turn towards Left

c. Communication System: That communication is critical to the success of the article. Without a communication system, no system or project can function. This is also true for this project. In this work, the only communication equipment required is the RF Module. This Module is used to wirelessly convey the user's various hand movements (encoded as 4-bit digital data) to the receiver, which decodes the received 4-bit digital data and moves the arm, gripper, and platform accordingly. The block diagrams in Figures exhibit the whole project's communication infrastructure. The Linker (Circle labelled "A") in Figures 2 and 3 is used to depict the connection (signal flow) between the Transmitter End and the Receiver End.

VII. FLOWCHART OF THE IMPLEMENTED CODE

The methods used to programme the system is depicted in the two figures above. The signal received by the play station may be read on the Arduino platform using the built-in function Psx.read() by adding the Psx.h header file when included the supporting files for execution. The Arduino controller's receiver flag is watched to detect any command applied to the controller.

When the receiver flag becomes high, it means a control signal has been transmitted to the robot. The next step is to decipher the instruction that has been given to you. The applied signal is read and saved in the variable "DATA," and the algorithm then searches for the opcode that executes when it matches the code stored in "DATA."

 Example: As we know that when UP button of the play station is pressed the code been read on Arduino by function is DATA = Psx.read(); , due to which the value or data stored in variable DATA is Psx.Up now the program is constructed such that what the controller should perform when data is read as Psx.Up. 

 In this case all the four motors are instructed to move in the forward direction by the controller. Hence chassis moves in forward direction. 

 Similarly, if data been read and stored in the variable is Psx.Down then all the dc motors re instructed to move in the reverse direction. Hence the chassis moves in reverse direction.

VIII. HARDWARE IMPLEMENTATION

Figure 4 represents the pick and place robotic arm mounted on chassis. The pick and place robot so implemented is controlled using RF signal. The chassis is supported for the displacement of robotic arm by four Omni wheels. The robotic arm implemented has four degrees of freedom. The circuit powered up by 12V 1A battery fitted within the body of the chassis.

Figure 5 represents the transmitter that transmit the signal for the movement of robot ant the arm based on the hand gesture using accelerometer. When the toggle button is on the hand gesture controls the movement of arm and when the toggle button is off then hand gesture controls the movement bot.

IX. FUTURE SCOPE

The robot configured for pick and place operations may be made more adaptable and efficient by giving feedback and allowing it to perform without human involvement. It is made feasible by an image processing tool that is linked to this Arduino. Line follower, wall hugger, obstacle avoider, metal detector, bomb diffuser, and more characteristics can be added to increase its efficiency and allow it to run on its own without human interaction.

Conclusion

This study accomplished its goal of constructing the hardware and software for a gesture-based robot with a robotic arm. Based on the observations gathered, it is obvious that its movement is precise, accurate, easy to handle, and user pleasant. The robotic hand has been created effectively because the robot\'s movement can be precisely controlled. This robotic hand control approach is anticipated to solve problems such as putting or picking things that are far away from the user, picking and placing dangerous objects quickly and easily, and boosting our ability to accomplish such activities.

References

[1] Mohd Ashiq Kamaril Yusoffa, Reza Ezuan Saminb, Babul Salam Kader Ibrahimc, Wireless Mobile Robotic Arm, International Symposium on Robotics and Intelligent Sensors 2012 (IRIS 2012), 1072 – 1078. [2] Nikesh. R. Vange, Atul.V. Nar, Dhananjay .B. Surve, Anita. P. Trimukhe, Manisha .M. Patil, Rajesh .A. Patil, Object Sorting Robotic Arm Based on Color Sensing Mechanism , International Journal of Emerging Technology and Advanced Engineering , (ISSN 2250-2459, ISO 9001:2008 Certified Journal, Volume 5, Issue 5, May 2015) [3] Er. R.K. Rajput, „Robotics and Industrial Automation?, S. Chand Publications, 2014. [4] http://www.robotics.org [5] Accelerometer Availableat: http://en.wikipedia.org/wiki/Accelerometer> [6] ADXL3xx accelerometer available at:http://arduino.cc/en/tutorial/ ADXL3xx [7] K. Murakami and H. Taguchi, 1991, “Gesture Recognition using Recurrent Neural Networks”, In Proceedings of ACM CHI’91 Conference on Human Factors in Computing Systems, New Orleans, USA, pp. 237-242. Available at:http://openexhibits.org/wpcontent/ uploads/papers/Murakami_NNgesturerecognition_ 1993.pdf [8] K. Brahmani, K. S. Roy, Mahaboob Ali, April 2013, “Arm 7 Based Robotic Arm Control by Electronic Gesture Recognition Unit Using Mems”, International Journal of Engineering Trends and Technology, Vol. 4 Issue 4 Available at: http://www.ijettjournal.org/volume-4/issue4/IJETT-V4I4P347.pdf

Copyright

Copyright © 2022 Priyank Garg, Mansi Patel, Harshit Verma. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.