Hand Gesture-Controlled Robotic Arm with All-Terrain Surveillance Car

Abstract

: In this proposed work, a gesture-based all-terrain car and a gesture-based robotic arm are designed to perform various real-life tasks. The motion of the robotic arm is based on the gesture input provided by the user. The use of a gesture controller provides ease for the user to control it efficiently. To have a clear and proper vision around the car, First-person View (FPV) cameras have been brought into the picture with a featured mechanism of pan and tilt placed in a particular position per the user\'s requirement and operation. A setup of two FPV cameras placed side by side can be made so as to have a panoramic view of the front of the car. Basic hand gestures are required for the motion of the surveillance car. A conveyor belt and off-road sturdy tires can be used for the purpose of all-terrain. The user can also get a virtual view of the robot by using a Virtual Reality (VR) set.

Introduction

I. INTRODUCTION

As the generation is evolving the need for surveillance has increased immensely. Almost every sector of the industry requires surveillance facilities for their workstation. There are various types of robots available in the market for surveillance purposes. There are many operation-specific robots used for specified tasks. The idea of the project is to implement various operations in a single robot so that the user will get multiple operations in a single robot. The robot uses gesture-controlled locomotion and First Person View [FPV] cameras for surveillance and a gesture-controlled robotic arm to help the user carry out specified tasks. The FPV camera is a VR set or any radio frequency receiving monitor so that the user can get a panoramic view around the car for smooth surveillance and locomotion. This project's main motive is surveillance in remote areas where human intervention is not feasible and in various workstations. This robot can be helpful in various research activities to collect and store data.

II. LITERATURE SURVEY

Dhanashree Wadaye, Yash Nayak, Vishal Yadav, Dr. Vaqar Ansari - Gesture-controlled surveillance Car with robotic arm: In this project, the proposed work focuses on designing a surveillance robot that can be controlled by different human gestures. [1] The proposed work also mentions the all-terrain feature of the robot along with the robotic arm which would be helpful for the robot to carry out various tasks. The user can operate the robot using different motions such as wrist movement of hand for the motion of the robot and for other actions of the robotic arm there are different flex sensors attached to control the different movements of the robotic arm. The proposed work also mentions the head gesture movements in order to control the VR set which would help the user to get a virtual view of the robot. The proposed work also mentions the use of the First-person view [FPV] camera to get different views of the surroundings.

T. Kiriki, Y. Kimuro, and T. Hasegawa- A 4-legged mobile robot control to observe human behavior: In this proposed work the project is demonstrating a 4- legged walking robot that is navigating through human gestures. [2] The robot comprises a navigation system that consists of two phases. The initial phase of the robot is a human following the robot or phase in this phase the robot follows the human by recognizing the basic gesture of the human. The second phase is the recognition of human gestures. The robot uses a visual tracking system for tracking of the human head so that the robot can perform locomotion.

Roy Chaoming Hsu, Po-Cheng Su, Jia-Le Hsu, Chi-Yong Wang - Real-Time Interaction System of Human-Robot with Hand Gestures: The proposed work focuses on developing a robot that would act as a substitute for human labor in different industries. [3] The project uses Kinect sensors for real-time gesture-based human-robot interaction. The robot can also recognize various kinds of elderly conditions such as falling and abnormality. The iRobot-created mobile robot is a robot that can recognize gestures like waving of a hand, and raising of hands in order to control the robot’s locomotion

Nakshtra Popli, Sarthak Batra, Kailash Masiwal, Chaitanya Mamgain - Surveillance Car Bot Future of Surveillance Car Bot: In this proposed work the authors have demonstrated how to control a car using a Wi-Fi-based module and a mobile phone. [4] This kind of wirelessly operated car can be used for various purposes such as spying, military, etc. This kind of robot can be very much portable with a feature of wireless communication over a Wi-Fi connection which can improve the range of the robot.  The basic component of the robot is a camera, motor driver, motor, batteries, and wheels.  A wide variety of cameras can be used but, in this project, an AI-Thinker ESP32-CAM module with ESP32-S processor, and an OV2640 camera. The ESP32-cam module has a microSD card slot so that the image or video captured can be stored Images captured by the camera can be stored on a MicroSD card slot. The video transmitted from the ESP32 can be streamed using HTTP protocol via web browsing so as to have a clear view and proper vision around the bot. The web page on which the live video around the bot will be streamed will also have some virtual buttons so as to control the locomotion of the bot. Wi-fi wireless communication comes into the picture for receiving the live footage around the bot onto the user’s smartphone.

Rutwik Shah, Vinay Deshmukh, Viraj Kulkarni, Shatakshi Mulay, Madhuri Pote - Hand Gesture Control Car: In this project, the authors have conveyed their idea of moving a physical object at a distance by just the movement of hands. [5] Using this concept one can achieve convenience, and comfort in handling the car, and the most important thing is physical cost reduction in day-to-day life and in industries as well. The authors have come up with an innovative idea of clubbing the field of IoT and embedded systems for creating this project. An accelerometer provides the input to the gesture-controlled car which is placed on the user’s hand. A particular hand movement indicates the particular direction of the car. Arduino UNO receives the value generated by the accelerometer, then an HT12E encoder will encode the values which are connected to Arduino and send them to a receiver. The receiver receives the encoded values through the RF433 transmitter. The HT12D decoder that is placed on the car will decode the values. These decoded values will then be transmitted to the L298 motor driver. As the L298 receives the command it will instruct the motors accordingly.

III. METHODOLOGY

A. Robotic Arm.

The project focuses to design a six-axis robotic arm that can be controlled by simple hand gestures. For the gesture controller, the designed idea is to use flex sensors to sense a particular human gesture. The receiver of the gesture controller of the robotic arm will receive the sensed information by the flex sensor and perform a particular task. The basic hand gestures of the robot are the pinch and wrist movements. In order to control the robotic arm with ease and accurately the gesture controller would prove to be feasible and efficient. The robot consists of six joints for the smooth motion of the robotic arm. There are two main parts of the robotic arm. Firstly, it is the robotic arm itself and secondly, it is the hand gloves transmitter that will be controlling the robotic arm. There are various components used in the project to design the robotic arm some of the components used can be listed as servo motors, stepper motors, Arduino Uno, Bluetooth modules, motor drivers, and flex sensors. The project also uses an accelerometer (MPU6050). Change in the readings of the accelerometer signifies the particular task being performed by the robotic arm.  The stepper motor used in this project used at the base of the robotic arm basically focuses on the 360 degrees of motion of the robotic arm. The reference image of the robotic arm to be designed is given in Fig. 1.

B. Locomotion.

The project focuses on gesture-based locomotion. Basic hand gestures are used to represent the motion of the car. The wrist movement plays an important role in controlling the robot’s directions. The gesture or tilt provided by the user gives the direction to the car. Arduino Nano, NRF24L01, and MPU6050 are being used for building the transmitter and receiver. The NRF24L01 models used in the transmitter and receiver play an important role to connect the robot through the gesture controller. The transmitter and receiver are provided a particular address so that they can be connected efficiently. The receiver is then connected to the motor drive to give commands and perform the specific task of locomotion. The image of the gesture controller mounted on a car is given in Fig. 2. And Fig. 3.

C. Surveillance.

In the revolutionizing world, the need for surveillance has increased immensely so the project also gives importance to surveillance activities. The project consists of FPV cameras which will be placed at different destined positions for appropriate surveillance. One camera will be placed in the front end of the robot to get the front view. The second camera is placed at the top of the robotic arm so that the user can get the proper view of the objects surrounding it and can perform a particular task accordingly. The other motions of the robot such as pan tilt can be controlled by using a VR set which would make it easy to control the camera through head gestures. The image of the FPV camera to be used is given in Fig. 4.

In Fig. 5. Block Diagram, the block diagram portrays the entire flow of how the commands are transmitted to all the components of the bot for its smooth and proper functioning. The two main mechanisms for controlling the locomotion and the movement of the robotic arm are the gesture-controlled transmitter and receiver. As the user moves his hands in a specified direction the transmitter transmits this information to the receiver and the receiver forwards this information to the specified component for which that information is meant. For generalized explanation, if the user tilts his hands in the forward direction, then the receiver receives this data and gives a command to the motor driver, and the motor driver, in general, gives the command to the motors so that the locomotion is performed in the forward direction. A hand glove has been designed on which the gesture controller will be placed to control the robotic arm. As displayed in Fig. 1. Robotic Arm, each joint on the robotic arm depicts the joint on a human hand. Due to this, the users may not need to remember some particular motion to control the robotic arm, and controlling the robotic arms becomes very much easy. The user needs to move his fingers which will perform the motion of the claw in the robotic arm for picking up the objects. The wrist motion to move the wrist of the robotic arm. For surveillance, part FPV camera comes into the picture as it is very much compact and lightweight. The most important feature of FPV cameras is instead of being so much compact the video resolution is very much high. FPV camera with a built-in transmitter is used for ease of connection. A pan-tilt mechanism is designed and synced with a servo motor so that if the users move the head in a particular direction, then the camera will also face in that direction. For this purpose, the camera needs to be synced with a VR set.

Conclusion

The motive of the proposed work is to develop a well efficient robot that can perform different tasks in different fields. As the proposed work gets completed as per the calculations the accuracy of the robot can be increased by 5 to 6 percent which would in turn enhance its efficiency too. The speed of the robot can also be increased by 10 percent depending on the weight of the robot. There are slight delays in the locomotion of the robot which can be overcome with appropriate calculations and slight changes in the values in the program. There is also some delay in the robotic arm that can be overcome with the proper calculations. This will result in accurate results to perform various tasks of the robot. The appropriate positioning of the robot after changes in the directions is also important. The implementation of gesture controller is successfully implemented and tested. The transmitter-receiver is efficiently programmed and synced to perform gesture-based actions. The field surveillance industry is in immense demand as there is an increasing need for surveillance in various fields there is a lot of research being done in this field and there are many emerging ideas upcoming for efficient and easy surveillance. The application of robots can differ according to their use in different fields. Some might as for easy and efficient surveillance some might as for security. Many more things can be added or improved in the projects instead of a conveyor belt one can add a robotic leg that can be switched in different terrains. One could use a Lipo battery of 11.1V, 2200mAh instead of using a Lithium battery setup. Different types of different cameras can be used for surveillance which could be more efficient instead of using FPV cameras. Also, for controlling the robot the use of a transmitter-receiver is also efficient.

References

[1] Dhanashree Wadaye, Yash Nayak, Vishal Yadav, and Dr. Vaqar Ansari, “Gesture-controlled surveillance Car with robotic arm”, Journal for Basic Sciences, ISSN NO. 1006-8341 vol. 22, Issue 10, 2022. [2] T. Kiriki, Y. Kimuro, and T. Hasegawa “A 4-legged mobile robot control to observe human behavior,” Proceedings of the 2IEEE International Workshop on Robot and Human Interaction, 1999. [3] Roy Chaoming Hsu, Po-Cheng Su, Jia-Le Hsu, and Chi-Yong Wang, “Real-Time Interaction System of Human-Robot with Hand Gestures”, 2nd IEEE Eurasia Conference on IoT, Communication, and Engineering, 2020. [4] Nakshtra Popli, Sarthak Batra, Kailash Masiwal, and Chaitanya Mamgain, “Surveillance Car Bot Future of Surveillance Car Bot”, International Journal of Engineering Research & Technology, ISSN: 2278-0181, Vol. 10 Issue 10, October-2021. [5] Rutwik Shah, Vinay Deshmukh, Viraj Kulkarni, Shatakshi Mulay, and Madhuri Pote, “Hand Gesture Control Car”, In Proceedings to ICSITS - 2020 Conference, International Journal of Engineering Research & Technology, ISSN: 2278-0181. [6] “DIY Arduino Robot Arm - Controlled by Hand Gestures”, YouTube, Jan. 11, 2021 [Video file]. Available: https://youtu.be/F0ZvF-FbCr0. [Accessed: Oct. 15, 2022].

Copyright

Copyright © 2023 Dhanashree Wadaye, Yash Nayak, Vishal Yadav, Dr. Vaqar Ansari. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.