Hand Gesture Recognition and Cleaning Robot

Abstract

We know that technology changes and develops every day, and humans need change as well. Human-Robot Interaction (HRI) has become a popular and in-demand technology in the recent decade. One-way humans can interact with robots is through hand gestures. The use of hand gestures as a method of communication with machines, robots, and gadgets is becoming more and more common. This paper will focus on how we can use machines in our day-to-day lifestyle and how to communicate with them for helping purposes. The robot will be equipped with a variety of sensors, a motor driver, and the NodeMCU.

Introduction

I. INTRODUCTION

As our daily lives have become increasingly automated, Human-Robot Interaction machines have seen intense growth in the past few decades. It's made a world of difference whether you're in a mall, hospital, school, workplace, or military because simply using hand gestures to communicate with machines makes things a lot easier.

Through this approach, we are building a robot car that can be used by us for cleaning. Today, most of us eat a lot of junk food, so we are prone to suffering from back pain, joint pain, and other ailments as a result. Due to the fact that bending and walking are becoming more difficult and as humans are becoming lazier, we are developing a robot that will be operated by hand gestures to remove any dirt and debris from any surface.

II. INTERNET OF THINGS (IOT)

In an internet of things, computers, electronics, machines, objects, and animals are interconnected through unique identification identifiers (UIDs), allowing them to communicate over a network without involving humans. A thing within the net of things is often a private with monitoring of home appliances remotely, a farm controlling bot with a biochip transponder, a wearable that contains built-in sensors to inform women in extremely risky situations, or the opposite natural or man-made object that will be assigned an Internet protocol (IP) address and is capable of Companies across a wide range of industries use IoT to improve their operational efficiency, better understand their customers with the intention of improving customer service, improving decision-making, and boosting corporate value.

What is IoT and how does it work? An Internet of Things ecosystem is made up of web-enabled smart devices that acquire, send, and act on data from their surroundings using embedded systems such as CPUs, sensors, and communication gear. By connecting to an IoT gateway or another edge device, IoT devices can share the sensor data they acquire, which can then be forwarded to the cloud or evaluated locally. These devices communicate with one another on occasion and perform actions based on the information they get. Individuals can interact with the devices to line them up, give them guidance, or obtain data, but the machines can do the majority of the job without human involvement.

A. IoT Device Example

Basically, everything that can gather data from the actual environment and send it back home is a member of the IoT ecosystem. Automation of home appliances, RFID tags, and industrial automation sensors are just a few examples. Temperature and pressure in industrial systems, the status of crucial elements in equipment, patient vital signs, and the consumption of water and energy are just a few examples of what these sensors can monitor IoT devices include entire factory robots, as well as autonomous trucks that transfer merchandise around industrial settings and warehouses.

Fitness wearables and residential security systems are two more examples. More general devices, such as the Raspberry Pi or Arduino, let you design your own IoT endpoints, whether or not you imagine your smartphone as a pocket computer. A computer may be sending data about your position and behavior to back-end services in a very way that's extremely IoT like sending data about your position and behavior to back-end services in a very way that's extremely IoT-like.

III. SYSTEM BLOCK DIAGRAM

One of the conceptual models that define the structure, behavior, and view of a system. It is one of the formal descriptions and representations of a system.

IV. COMPONENT DESCRIPTION

A. Hardware Description

1. NodeMCU: NodeMCU is nothing but an open-source platform based on ESP8266. This will connect things and using Wi-Fi protocol data will be transferred. It is easy to use. Editing can be done in Arduino IDE or IUA languages. Furthermore, the board has 13 GPIO pins, 10 PWM channels, SPI, ADC, UART, and one call that Occurs in API-driven API programs. It features a Development Board that runs on Wi-Fi, analog pins, digital pins, and serial protocols.  Its Development Board is powered by Wi-Fi, analog PINs, digital pins, and serial communication protocols. We can connect to the Internet via Wi-Fi using NodeMCU, we can apply our desired instructions by creating an HTTP page. As a chip, ESP8266 is hard to reach and use. You have to sell the cables, with the analog voltage appropriate to its pins to perform simple tasks like sending a computer click on a computer to a chip. You should also set it up, with low-level machine guides so that it can translate with chip hardware. ESP8266 is useful as a control chip embedded in mass-produced electronics. However, this level of integration can create a considerable burden for students who want to test it in their IoT projects.

The open-source firmware ESP8266 is built on the SDK of chip manufacturers. It provides a simple editing platform based on eLua. It is a very simple and fast writing language for the established engineering community. The DEVKIT board incorporates the ESP8266 chip into a standard circuit board. With a built-in USB port prewired with a chip, LED lights, GPIO pins that fit properly on a breadboard, and a hardware reset button, this board can handle any project you can dream up.

2. L298N Motor Driver: Using the L298 Dual H-Bridge Motor Driver Integrated Circuit, this dual bidirectional motor driver functions both in the forward and reverse direction. Using this circuit, you will be able to easily and independently control two motors of up to 2A each in both directions. With just a few control lines per motor, it is ideal for robotic applications and well suited for integration into a microcontroller. Additionally, it can be interfaced with simple manual switches, TTL logic gates, relays, etc. The board is equipped with a power LED indicator, an onboard +5V regulator, and protection diodes

3. Relay: This is a simple electromechanical relay that can work at low voltages like 3.3V like the ESP32, ESP8266, or the 5V like your Arduino with a max switching voltage 250VAC/30VDC and current of 5A. There is an indicator LED on the board, which will light up when the controlled terminals get closed. This is an SPST relay which is great for applications that need only an on or off state.

B. Software Description

1. Operating System: No Operating System is required for NodeMCU.
2. Programming Language: C/C++, is one of the powerful and general-purpose programming languages which is used to develop the operating system, browsers, and games.

V. PIN DESCRIPTION

1. NodeMCU: NodeMCU has a total of 30 pins on it.  15 pins on each left & right side. Esp. 8266 NodeMCU has 16 GPIO & 1 analog pin. However, out of 16 GPIO pins, Only 10 pins are for general-purpose input-output operation. So NodeMCU ESP8266 access these GPIO pins. GPIO means general-purpose input-output pins. APC analog to digital Converter channel Ao. SPI means serial peripheral interface pins. I2C means inter-integrated circuit pins, and UART means Universal asynchronous receiver-transmitter pins. There are a total of 4 grounds and 4 VCC available on NodeMCU. It has a reset and flash button built-in. RX TX, SD2, and SD3 pins are used for another internal process. The EN & RST. is the pin & button that rests the microcontroller. Analog pin(Ao) is used to measure voltage in the range of 0 to 3.3 V . SD1, CMD, SDO, and CLK are the pins available for SPI Communications.

2. L298N Motor Driver: L298 N motor driver operates at very low voltage (5v), It’s like a little current amplifier, it takes a low current signal & gives out a high current signal. This motor driver has four motor terminals separately to attach motors. It takes 12 v input voltage. The input pins are Drive voltage input, and ground & Logic voltage input to drive the motors. It also has four PWM pins for the motor and & each motor is provided with 2 pins. It has 4 control pins, each motor with two pins. It uses H-bridges to drive the motor.

1. Acquire the Mobile application data
2. Read the data from the mobile application
3. If the signal is received start data processing
4. Send received data to NodeMCU
5. Process the data and send it to the Motor driver 
6. If dirt or debris recognize start the mop motor
7. If the hand gesture signal recognizes start the wheel motors
8. If the signal is not received switch off the robot

Conclusion

In an age when machines and robots make life easier, our robot will soon be one of the potential machines for humans to clean their floors. Arduino, motor drivers, and other important components will provide the robot with power so that it can run. As soon as it receives the signal, it will get to know the direction where it must go, and once it recognizes the dirt or debris, it will start its motor and begin cleaning until humans fail to provide any further signals or information.

References

[1] Anuradha Patel1, Ms. P. Devaki, Chethan Raj “Body Sensor Network using raspberry pi3: IoT,” [2] Surya Narayan Sharma, Dr. A Rengarajan “Hand Gesture Recognition using OpenCV and Python” [3] Afreen Bano, Mitun Kundu, Ayan Jana, Suman Kumar Pal “Gesture Control Interface Of A Robot-Car Using Raspberry Pi” [4] https://www.instructables.com/Using-Motors-With-L293D-IC/ [5] https://www.engineersgarage.com/ht12e-encoder-ic/ [6] https://www.researchgate.net/publication/322263316_Design_and_Implementation_of_Gesture_Controlled_Robotic_Arm_for_Industrial_Applications [7] Designing a receptionist robot: Effect of voice and appearance on anthropomorphism. G. Trovato, J. Ramos, H. Azevedo, A. Moroni, S. Magossi. IEEE, 2015 [8] Mobile robot system with remote monitoring for the home environment. Ahn, H.S.; Sa, I.K.; Choi, J.Y. IEEE, 2009 [9] Spontaneous, short-term interaction with mobile robots. Schulte, J., Rosenberg, C., Thrun. IEEE, 1999 [10] Designing robots for long-term social interaction. Gockley, R., Bruce, A., Forlizzi, J., Michalowski, M., Mundell. IEEE, 2005 [11] Cory D. Kidd and Cynthia Breazeal MIT Media Lab Personal Robots Group “Robots at Home: Understanding Long-Term Human-Robot Interaction”

Copyright

Copyright © 2022 Abhinav Mandal, Utkarsh Bhujade, Ronit Bagde, Abhishek Manmode. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.