Computer Vision Based Virtual Sketch Using Detection

Abstract

In today\'s environment, and in the midst of a pandemic, online learning is the only way to learn. Students become more curious about knowledge as a result of online learning, and they choose their own learning route. Academics, on the other hand, must devote time to studying and be disciplined in their dedication in order to pass the course or exam. There are also other hurdles to Online learning. Students are losing their comprehending power since they have been accustomed to relying on their teachers and offline classes. In recent years, virtual writing and controlling systems have become demanding study areas in the fields of image processing and pattern recognition. It makes a significant contribution to progress

Introduction

I. PROBLEM STATEMENT

Developing an interface between human palm and the system using open cv techniques and python language to pick the  tool and draw using hand on the developed     drawing area. For making teaching videos more explanatory.

II. LIBRARY

CV2, Mediapipe, Numpy, Time, pyttsx3

III. GOALS/ OBJECTIVE

• To create a virtual sketch.
• To detect the human palm and finger  as a color marker.
• To create an interface between user            and the system.

IV. INTRODUCTION

Traditional forms of art and their text have been supplanted by digital art in the digital age. By employing technology for generating, storing, and exhibiting digital art has evolved from traditional art, that is, physical art. Traditional art refers to art created before the advent of digital art.

In the current situation, digital art and traditional art are both part of a symbiotic state, therefore we must thoroughly comprehend the fundamental understanding of the form between the two.

Pen and paper, as well as chalk and board, are examples of traditional writing methods. Building a hand gesture recognition system to write digitally is the major goal of performing digital art.

Many different types of writing are included in digital art, such as employing.

V. BACKGROUND/ APPROACH

Many online learning platforms, such as BYJUS and Cognitive class, use tutors and technology such as graphical animation to make their videos more informative.

In our approach, we attempted to create a prototype tool that may be used as an alternative to such software. Our tool would be highly useful and would improve online learning.

And, because it is cost-effective, it might be used by any teacher to make their teaching films more informative.

We also sought to make it as simple and user-friendly as possible, with the very minimum of hardware requirements, so that even someone with no prior computer experience could use it.

VI. LITERATURE REVIEW

1. Author proposed object colour recognition and masking with morphological processes, as well as a task bar for different colours. It's still difficult since it only detects one colour as a marker; if another single spot of colour is discovered elsewhere, the marker moves and captures a random route.
2. In Author is employing various fingertip recognition algorithms, as well as manually labelling the dataset and constructing a module by training the dataset with a pre-trained model. However, it isn't entirely correct.
3. The technique proposed in uses the Kinect sensor's depth and colour information to detect the hand shape. When using the Kinect sensor to recognise gestures. It is an extremely difficult procedure. Because the kinetic sensor's resolution is so low. It is appropriate for large objects, such as the human body. However, it is not appropriate for an object/thing like a finger.

VII. OVERVIEW OF MODULE

Below mentioned modules implemented in our project

1. Hand Land Mark Detection: It is a process of finding joints  on the finger as well as the finger tips on a image. Basically this module produce 22 keypoints of which 21 keypoints are on the hand/palm, and the 22nd point is considered as the background.
2. Drawing Tools Module: In this we created function for selecting tools such as curve, straight line, circle, rectangle and eraser. In addition to above module we have also added audio module for which we can get audio of our selected tool but it has adverse effect on system with lower configuration but we very much sure that it would fine with the higher configuration system

A. Algorithm Diagram

1. Advantages: It is an alternative for advanced teaching software, it could be used by any one    having no prior knowledge about computer, being cost effective it could be used by teachers in remote area as well for making their videos more explanatory.
2. Disadvantages: The project works in ample of light because palm of our hand should be clearly visible.And considering the audio module it has adverse effect such screen get freeze but it could be overcome by obtaining higher configuration system and also by parallel processing/programming

VIII. FUTURE SCOPE/WORK

This system could be used as an alternative     for teaching software used by teachers. If further interpreted various virtual based physical games could be made. Controlling the robot using gestures considered as one of the interesting applications in this field proposed a system controlling a robot using hand pose signs. The orders could be given to robot to execute some task, where each sign has a specific meaning and represents different function

Conclusion

The system has the potential to challenge traditional writing/teaching methods. The ultimate goal is to create a computer vision machine learning application that promotes Human computer interaction also named Man-Machine Interaction refers to the relation between the human and the computer or more precisely the machine. System functionality referred to the set of functions or services that the system equip is to the users while system can operate and perform specific user purposes activity efficiently such as virtual drawing.

References

[1] Pranavi Srungavarapu, Eswar Pavan Maganti, Srilekkha Sakhamuri, Sai Pavan Kalyan Veerada, Anuradha Chinta, Virtual Sketch using Open CV,ISSN: 2278-3075 (Online), Volume-10 Issue-8, June 2021 [2] Prof. S.U. Saoj, Nishtha Dua, Akash Kumar Choudhary ,Bharat Phogat AIR CANVAS APPLICATION USING OPENCV AND NUMPY IN PYTHON,e-ISSN: 2395-005 [3] Saira Beg, M. Fahad Khan and Faisal B?ig, \"Text Writing in Air,\" Journal of Information Display Volume 14, [4] sidra Mehtab , Jaydip Sen , Object Detection and Tracking Using Opencv in Python. [5] https://towardsdatascience.com/Painting- with-opencv [6] https://www.geeksforgeeks.org/live- webcam-drawing-using-opencv [7] Numpy.org, Available: http://www.numpy.org [8] https://pypi.org [9] https://stackoverflow.com/ [10] First International Conference on Computer Engineering International Journal of Scientific Research in Science and Technology Virtual Painting with Opencv Using Python Yash Patil1, Mihir Paun1, Deep Paun1, Karunesh Singh1, Vishal Kisan Borate Volume 5 Issue 8, November-December-2020. [11] Suryansh Pratap Singh a, Akshat Mittal b, Manas Gupta c, Soumalya Ghosh d, Anupam Lakhanpale Comparing Various Tracking Algorithms In OpenCV Vol.12 No.6 (2021), 5193-5198 [12] Bhumika Nandwana , Satyanarayan Tazi , Sheifalee Trivedi Dinesh kumar khunteta , Santosh kumar vipparthi ,A Survey Paper on Hand Gesture Recognition [13] Hemalatha Vadlamudi ,Evaluation of Object Tracking System using Open-CV In Python International Journal of Engineering Rese?rch & Technology (IJERT) Vol. 9 Issue 09, September-2020 [14] Siddharth Mandgi1, Shubham Ghatge1, Mangesh Khairnar1, Kun?l Gurnani1, Prof. Amit H?tekar2 Object Detection And Tracking Using Image Processing Vol. 8, Issue 2, ( Part -1) February 2018, pp.39-41 [15] https://www.geeksforgeeks.org/python-text-to-speech-by-using-pyttsx3/

Copyright

Copyright © 2022 Santosh Dhaigude, Sumedh Bansode, Shivam Waghmare, Saurabh Varkhad, Prof. Shubhangi Suryawanshi. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.