Gesture Replication Robo-Arm

Abstract

Abstract: In these times robotic arm is used in several areas analogous as military, defense, medical surgeries pick and place function in artificial automation operations. The Gesture Replicating Robotic arm is a servo controlled robotic arm which replicates the mortal arm gestures in a three- dimensional terrain. The main ideal of this design is to control the robotic arm using mortal gestures. This system replicates the conduct of mortal hands, and the robotic arm moves and performs the task. It\'s coded in such a way that the demanded conduct for the mortal gesture is done. thus, this model we proposed will be truly helpful.

Introduction

I. INTRODUCTION

Robots are decreasingly being integrated into diligence to replace humans specially to perform the dangerous tasks. A robot is an electro-mechanical machine able of carrying out a complex series of conduct automatically or under mortal supervision. These are used in colorful fields similar as diligence, service, healthcare, and exploration. It might be dangerous for humans to perform some specific tasks like working with explosive chemicals, defusing losers, and other dangerous workshop. thus, humans can be replaced by robotic arm to perform the operations. Robots can be used in places where mortal life is placed at threat similar operations are lemon diffusing, fireworks assiduity. Gesture recognition is a content in computer wisdom and language technology with the thing of interpreting mortal gestures via fine algorithms. Gestures can appear from any fleshly stir or state but generally appear from the face or hand.

Currently the focus is on the field include emotion identification from the face and hand gesture recognition. A robotic arm is a robot manipulator, generally programmable, with similar functions to a mortal arm. The robotic arms can be free and controlled manually and can be used to do different type of tasks with great elegance.

Gesture identification technology is used to command and control the robotic arm. This technology enables humans to be in touch with the machine and interact naturally without any mechanical bias. Gesture recognition is content in computer wisdom and language technology with the thing of interpreting mortal gestures via fine algorithms. Gestures can appear from any physical stir or state but generally appear from the face or hand. Gesture recognition empowers the mortal to communicate with the machine and interact without any mechanical bias. Hand gestures are vastly used for robotic control operations and robotic systems can be controlled naturally and privately with analogous robotic communication

II. LITERATURE REVIEW

There are numerous ways in which the robotic arm can be controlled. But in the past few years there have been many working to control the robotic arm through computer terminals, joysticks, even interfacing it with the internet.

So that the arm can be controlled from any part of the world. There are several other methods that have been made by using cameras and computer vision algorithms to illuminate sign language.

Although, the identification and recognition of posture and human actions is also the subject of gesture recognition techniques. Gesture reputation can be seen as a way for computers to start to apprehend human body language, this constructing a richer bridge among machines and human beings than primitive textual content user interfaces or maybe GUIs (graphical user interfaces), which nonetheless restriction most of the enter to keyboard and mouse. The literature consists of ongoing paintings in the computer imaginative and prescient subject on taking pictures gestures or greater fashionable human pose and actions by using cameras connected to a computer.

Robots are labeled into two kinds: independent Robots like line sensing or edge sensing robots and far off-controlled Robots like Gesture controlled Robots. There are several ways to capture a human gesture that a system could be able to recognize. For giving input to the hand we can also give it through camera for capturing gestures and also by glove and various other techniques.

Nowadays on this world in which the era and technological know-how is growing every day the sphere of Robotics additionally had to be upgraded. As a result, it become found that there are some troubles within the existing device as the present system may be controlled simplest with the Controller so the consumer cannot without delay manipulate the robotic.

Secondly the physical work is more to govern the robotic. Thirdly the strength intake turned into more as the controller calls for electricity deliver to a higher quantity than the Accelerometer. Finally, the rate become greater as greater hardware became required. These were a few Drawbacks in the present device.

III. METHODOLOGY/EXPERIMENTAL

A. Robotic Arm

It consists of total 5 motors, Arduino uno which inputs the values from the python code and sends the data accordingly to the servo and dc motors attached to center most finger. It consists of aggregate of Servos, connected with unique mechanism that A micro regulator Arduino UNO which inputs the values from the module and sends the data consequently to the servos. Both the circuitry and base are cropped upon the common base, made of tempera to ameliorate the stability. The whole structure is made of polylactic acid (PLA). As mentioned over, the robotic arm mimics the movement of glove worn by the stoner, when the glove is listed in the forward direction or any similar direction, the arm spontaneously follows suit. The Grabber functions with the use of flex detectors attached to Centre most cutlet, the degree of movement of the cutlet determines the angle of the mini servo which is attached to the gears controlling the movement of the theft.

B. Hardware Part

Factors demanded for robot to made are Arduino mega, HC-05 Bluetooth module, servo motors, battery. The explanation of these above parts are mentioned below.

C. Arduino Uno

The Arduino Uno is a microcontroller chip which is mainly based on the ATmega328. This board consists of 14 digital input and output pins. It includes the entirety needed to help the microcontroller; connect it to a laptop with a USB cable or energy it with an AC-to-DC adapter or battery to get started out. "Uno" approach one in Italian and is named to mark the approaching launch of Arduino 1.0.

2. Servo Motor

The MG996R is largely an upgraded version of the famous MG995 servo, and capabilities upgraded shock-proofing and a redesigned PCB and IC control device that make it plenty more correct than its predecessor. This excessive torque well known servo can rotate about a hundred and twenty degrees (60 in each direction). A servomotor is a rotary actuator that lets in for control of angular or linear position, speed, and acceleration. It includes an appropriate motor coupled to a sensor for position remarks. These are used in industrial programs, robotics, in-line manufacturing, pharmaceutics, and meals offerings.

3. 3D Printed Robo Arm

This robotic arm is a system that replicates the action of a human arm. A pc-controlled mechanical arm is called a robot arm. However, a robot arm is simply certainly one of many styles of distinctive mechanical fingers. Robotic arms can be as simple, but it can get complex as prosthetic hands. A human arm, by using evaluation, has seven ranges of freedom. Your arm's job is to transport your hand from place to region. In addition, the robotic arm's job is to transport a cease effector from region to place. You can outfit robot palms with all varieties of give up effectors, which might be suitable to a particular utility.

4. Fishing Wires

The robotic arms used in industries are mainly of four to six joints. This provides them with equal number of axis rotation for movement. Here it is just a prototype, so we have used fishing wire whereas in the industries for the robots they use thick cables.

5. Jumper Wires

Here these wires are used to connect servo motors to the Arduino board. There are various types of jumper wires, we are mainly using Male-Male jumper wires. These wires come in rainbow colors. Prime part is that this wire comes in in a 40-pin ribbon cable. Generally, the users always pull the ribbon wires off to make separate jumper wire.

C. Functioning of Robotic Arm

The operator stands at a certain distance from the camera and moves the right/left hand, then the pixels catch the human gestures and send the data to the Arduino and Arduino command the motors to follow the direction The arm is completely opened to pick up the object. The robotic arm also is moved over and rotated to another asked position, also lowered. This way the robotic arm can be operated and controlled in any manner as supposed necessary by the driver from a distance 0.5-1 m.

IV. DESIGN

The construction of the mechanical assembly consisted of design of the arm and the design of the fingers. The arm was constructed by having a metal plate as its base. Two clamps on two sides which supports the arm with the base were fixed with the help of screws. The upper end of the clamps was given with the servo motors and these motors are responsible for the up and down movement of the arm. The entire body of the arm were constructed by thick aluminum plates which is light weight in construction, and it supports the weight of the palm and the fingers. The fingers move in a way such that the entire hand can pick an object. So, the weight of the finger should be as light as possible, and the structure should be stiff and not fragile. The springs were used for the structure of the fingers. At the top of every finger the fish wire is attached so that the fingers can operate. Then the strings are then attached the servo motors which is placed in the body of the arm. To cover the entire hand and to have aesthetic look, the entire fingers and palm is covered with gloves. The prime action of the arm is up and down movement and the finger movement of the hand.

V. PROGRAMMING PART

B. Code Explanation

1. Creating a function which will receive values in the form of string. This will have 5 inputs representing 5 fingers.
2. Create a function in which concatenate the string which is coming as an output from python to the empty string.
3. Create substring and add it convert it to int values.
4. Reset variables to initial values so that next time we use it we will get the accurate results.
5. Send values to Servo.
6. Create individual objects of Servo class for each finger. Example servo Thumb, servo Ring, servo Middle etc.
7. Connect Arduino to hand.
8. Define pin numbers in setup for each finger.
9. Send command to motors. So, if value received is 1 the rotate finger by 180 degrees upwards and vice versa.

C. Python Programming

1. Install OpenCV, cvZone, and serial Device packages in IDE.
2. Set camera type (device cam or external).
3. Read the frame using OpenCV functions.
4. Now add hand tracking part using cvZone.HandDetector.

D. Flowchart

rt

VI. APPLICATIONS

These types of robotic hands can be used in different domains and various types of applications as follows:

1. Control of colorful functions of robots in laborious and fine atmospheres artificial jobs as in oil shops, shot blasting chambers etc. The driver controls the robotic functions from outside the dangerous chambers looking through a glass door.
2. Automatic selecting of small objects (bottles, bags, trapezii.) moving on a conveyor and placing at other asked position in diligence manufacturing colorful types of cosmetics, food products, and dragster.
3. Automatic essence cutting machines in particular desire biographies, which are in high temperature zones.
4. Robots controlled cranes, lifting soonest. Operated from a distance with fritters/ hands controlled remote system. Characterization/Pseudo Code/ Testing
5. In medical science: "Neuro-arm" uses miniaturized tools such as laser scalpels with pinpoint accuracy

Type of Testing Applied

The testing is being done whether there is any signal sensed by the servo motor.

The objective has been achieved without any hurdles i.e., the control of robotic hand through human hand gestures. The robot is showing proper responses whenever the hands are tilted.

Thirdly, an on-board camera can be installed for monitoring the robot from faraway places. All what is needed is a wireless camera which will broadcast and a receiver module which will provide live streaming.

VII. RESULTS AND DISCUSSIONS

 The complete design of a hand was developed by additive manufacturing technology which we have studied in last semester. This project is a combination of two different domains that is medical science, computer aided designing with manufacturing. More than other prosthetic hand in the market this is cheaper and more affordable. Our project is unique in every aspect which makes it more favorable and appropriate.

VIII. FUTURE SCOPE

1. The future scope of this project can be controlling the arm from a distant location with the help of camera and high range wireless transmission modules. So that the arm perfectly performs the piece of work, and the arm can be rooted to a rover so that it can move wirelessly and can do the assigned task. This will be very useful in bomb diffusion applications
2. Sign language recognition using images processing sign language detection is possible where we can convert a symbol to text information.
3. Remote control from the use of hand gestures detection, the user can be able to control RC cars, drones etc.
4. From facial features Controlling a computer through facial gesture is possible via image processing where eye tracking is used to control the cursor.
5. In gaming technology Through the use of gesture-based control a player will be able to play games without using any controllers.
6. Alternative computer interfaces Instead of using mouse and keyboard using hand gestures a user can interfaces with the computer by using only gesture-based control.
7. Full robot gestures using full body gesture-based control the user will be able to control a humanoid robot.

IX. ACKNOWLEDGMENT

I would like to express my gratitude towards our college (Vishwakarma institute of technology, Pune) for their kind co-operation and encouragement which help me in completion of this project and to our guardian & members of the group.

Conclusion

The ideal of this paper has been achieved which was developing the tackle and software for a gesture grounded robotic hand. From observation that has been made, it easily shows that the movement is exact, accurate, and is easy to control and stoner friendly to use. This robotic hand control system is anticipated to overcome the problem similar as placing or picking objects that are down from the stoner, pick and place dangerous objects in a veritably fast and easy manner or accelerating our capacities to perform similar tasks.

References

[1] https://ieeexplore.ieee.org/abstract/document/6724315 [2] https://ieeexplore.ieee.org/document/6724315 [3] https://www.researchgate.net/publication/221923729_Real-Time_Robotic_Hand_Control_Using_Hand_Gestures [4] Chris Carter,Brian Dwyer, Rachel Lutsic,“Wireless User Controlled Robotic Arm”,Central Michigan University, Mt. Pleasant, MI 48859 [5] Amithash E. Prasad, Murtuza Tambawala,” Human Controlled Remote Robotic Arm” (HCRRA) [6] N. Martel-Brisson & A. Zaccarin\'s , \"Moving Cast Gesture Replicating Robotic Detection from a Gaussian Mixture Gesture Replicating Robotic Model\".Graphcut Background Subtraction. [7] T. Bouwmans, “Background Subtraction for Visual Surveillance: A Fuzzy Approach”, Handbook on Soft Computing for Video Surveillance, Taylor and Francis Group, Chapter 5, March 2012. [8] https://www.ijraset.com/research-paper/gesture-controlled-robot-with-robotic-arm [9] https://www.frontiersin.org/articles/10.3389/fnbot.2018.00027/full [10] https://steinmetz.union.edu/index.php/abstract/motion-controlled-robotic-hand-mimic-human-gestures

Copyright

Copyright © 2022 Dr. (Prof.) Laxmikant Mangate, Sahil Kadu, Dhruva Kale, Karan Agrawal, Pavan Kandurke, Tanmay Kasliwal. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.