Gesture-Controlled User Interfaces, What Have We Done and Whats Next?

Abstract

To meet the challenges of present computing, researchers working towards break away from traditional types of interaction. This paper identifies new trends in technology, it’s uses and advantage. We concluded that Gesture controlled user interface gives realistic opportunities especially for those users who are uncomfortable with commonly used devices. It’s appropriate for older and disabled people. Gesture controlled user interface has some potential to improve quality of life especially for older and disabled people along with general users.

Introduction

I. INTRODUCTION

The fact that use of body gesture to communicate with computer system is usable in real world, intensified by evolution in gesture based commercial solutions. This paper described background of development of Gesture based technology. This paper also describes next options we have.

Virtually all inputs now provide by keyboard and mouse. Use of another means like virtual objects, body gestures, hand gestures are becoming popular with devices like digital television. Gesture controlled user interface is also commonplace in gaming automotive, medical industries.

II. GESTURE CONTROLLED SYSTEM

As humans, we generally use gestures to communicate with anyone. Even a child can read gesture to learn to communicate before he learns to talk. We can use gesture without or with verbal communication. Gesture can be used as command to control different type of devices. With gesture, we can control smart home appliances, tele-care system etc.

III. FINDINGS

Findings shows interest of people towards Gesture controlled technology. There is different aspect to present like gesture types, user choices, use, technology and commercial product. These are important area to mention.

IV. TYPES OF GESTURE

In gesture control system, hand gesture was the most popular. Most of the researches which are based on hand gestures shows that hand gestures provide direct control which is instant, but has limitation in number of choices. There are also researches about body gesture, finger movement, voice gesture etc.

V. USERS

Initially gesture controlled system used for users to work with objects and presentation. Current findings are focused on older and disabled users. Wheelchair users are highly observed for Accelerometer based Gesture controlled system. Researchers begin to see the value which can be provide by Gesture controlled user interface. It’s time to focus on such user group to see potential of GCUI in every aspect.

VI. APPLICATION

Gesture based applications can be utilized for a wide range of things like entertainment, controlling home appliances, tele-care, tele-health, older care. The scope of applications shows importance of Gesture controlled system. Earlier technologies were developed to replace traditional devices like keyboard and mouse. People can easily interact with any media now using gesture to control wide range of applications. They also can get Gesture controlled hardware and software with their choice.

VII. TECHNOLOGY

The significant process of technology can be understood by the way of recognize the gestures. The process of technology of image processing played main role. Gesture can be captured in past by many interconnected technologies like glove, infrared signal etc. Now it is possible to capture any intuitive gesture with vision technique, video and web cam-based gesture recognition for any present devices. Gesture based research are now moving towards everyday uses. The game industry is the main target of these product. Healthcare, training, hand held applications and industrial 3d simulations are becoming workable too.

VIII. CURRENT OPPORTUNITIES FOR GESTURE TECHNOLOGY

There is interest for interface of gesture commands as we HCI is progressing in recent years. Gestural interfaces are electronic analogues to paper and pencil. These interfaces have many potential advantages and disadvantages too. From study we can see the possibility of implementing the tech in different zones.

IX. ARTIFICIAL INTELLIGENCE

Use of gesture-based technology will play key role part in this intelligent lifestyle. People, devices and computation will become more integrated with each other and will soon become part of our daily lives.  Body gestures can provide the commands to communicate or even to control things like the curtain of windows. Robotic industry is also working with gesture technology. Industry is using this technology to manage and control the actions of the robot as section of the Human Robot interaction. Users can connect various devices using networking technologies and hand gestures. Many researches, like Select-and-Point, are well received by consumers and can greatly improve users' interaction with numerous gadgets in a ubiquitous computing environment.

X. SIMULATION

The simulation of human body activity in the screen is created by body gestures. In numerous ways, physical simulation can improve the realism of the final gestural animation. Gesture Tek creates an engaging computer-generated virtual reality therapeutic world that guides patients through interactive rehabilitation exercises, games, and activities that can target specific body regions as prescribed by clinicians. The performance of the patient is assessed and documented.

XI. TRAINING & EDUCATION

The technology solution can be created for the aim of training and teaching. It can train people automatically in rehabilitation or fitness centres based on the user's profile and body structure. The ability to use natural input through body movements rather than a mouse or keyboard is the most significant advantage here. WII showed a number of clips of people being trained at their own pace, based on gestures, with rapid results.

XII. ASSISTIVE LIVING

Multi-agent systems, secure communications, hypermedia interfaces, rich environments, enhanced intelligence of home appliances, and collaborative virtual environments are all converging, and this is a key enabler for the design and development of virtual senior support community environments. The goal of Telecare is to build and develop a flexible framework for virtual communities that support older persons. We still need to do more for them, as their numbers are increasing. For the independent and assistive living of old and disabled people, modern technologies can play an essential role. They can achieve this by reclaiming some psychologically valuable control and independence. This tendency can be seen in telehealth, telecare, telemedicine, and personal safety technologies. Due to external circumstances or personal preferences, they choose a multimodal approach. When designing for elderly users, a variety of factors influence interface design decisions. Removing the isolation of age-restricted users has little impact on interface design.

XIII. ELDERLY USERS AND WII

Some research provides us with a new interface and intuitive interaction style in our daily computer use. Users may avoid time-consuming processes in managing connections and controls between various devices, as well as transferring information/data, by using simple selecting and pointing hand movements. Using gestures, the elderly and disabled can effortlessly maintain communication aids with many devices or appliances. Elderly individuals, who are more mature than most gamers and have physical restrictions, are actively using Wii for entertainment and rehabilitation in front of a television or computer screen. As a personalised gesture-based interface, it gets people moving and keeps their minds busy without the necessity of a keyboard or mouse.

XIV. FUTURE WORK

Gesture-based technologies are now quite affordable and have fused with familiar and popular technology such as television and huge screens. We can install a camera or a remote with the TV, making it ubiquitous and non-intrusive. We can see the trends in gesture-controlled communication systems in this research. Because of the ease of use, cost, and familiarity, gesture-based user interfaces can provide new opportunities for the elderly and disabled. There will be a greater number of elderly individuals and fewer younger people to look after them. As a result, we must invest much more in Assistive Living solutions. The research project 'A gesture-controlled communication assistance for elderly and disabled individuals' has the potential to be a major undertaking in the future. The research's two main goals are to identify the many communication gestures used by elderly and disabled individuals and to build a rich augmented-reality interface for communication via a ubiquitous device such as a television set.

Conclusion

From studies conducted in the last 2-3 years, we can see how the gesture-based system has evolved. Initially, it required difficult equipment such as a sensor or a glove; now, webcams, image processing software, and gaming tools have made it easier. In the beginning, poor usability was a problem, but now it\'s intuitive and natural. Gesture control or recognition was once a complicated procedure, but now it\'s a simple vision technique that uses hand, head, or even entire body gestures. In the beginning, the major goal was to get a computer application to work. However, it is now widely accepted in the context of ambient computing and ubiquitous computing. In recent studies, more emphasis has been placed on controlling home appliances, using mobile devices, huge screens, tabletop screens, and managing group work, as well as the activities of home inhabitants. Another key feature is that it is now quite affordable, when it was formerly quite costly. In our final project, \'A gesture-controlled communication aid for the elderly and disabled,\' we are developing a rich augmented interface in common appliances such as televisions to control ordinary communication through gesture.

References

[1] https://www.scirp.org/journal/paperinformation.aspx?paperid=7503 [2] https://www.computer.org/csdl/magazine/co/2013/10/mco2013100022/13rRUxAASNX [3] https://blog.cranksoftware.com/what-is-gesture-based-ui [4] https://link.springer.com/chapter/10.1007/978-3-319-20886-2_7 [5] https://www.scirp.org/html/7503.html [6] https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.562.6140&rep=rep1&type=pdf [7] https://link.springer.com/chapter/10.1007/978-3-642-39330-3_20 [8] https://en.wikipedia.org/wiki/Gesture_recognition#Gesture_types [9] https://ieeexplore.ieee.org/abstract/document/4383638 [10] https://www.sciencedirect.com/science/article/abs/pii/S0360131514001419 [11] https://ieeexplore.ieee.org/abstract/document/6835606 [12] https://www.researchgate.net/profile/Maria-Karam-2/publication/39994510_PhD_Thesis_A_framework_for_research_and_design_of_gesture-based_human-computer_interactions/links/00b7d5239b3f298a88000000/PhD-Thesis-A-framework-for-research-and-design-of-gesture-based-human-computer-interactions.pdf

Copyright

Copyright © 2022 Vishakha Chandra, Dr. Devesh Katiyar, Gaurav Goel. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.