Gaze Assisted Transaction

Abstract

Automated Teller Machine (ATM) was introduced in the year 1959 for encouraging self service in retail banking. The single smart card ATM for multiple bank accounts is used for security applications for withdrawing cash from ATM machines. Eye tracking has been introduced as a way of direct interaction which could pave the way for new technologies and devices to be introduced for end-users. It is becoming a popular way of interaction. Gaze coordinates could be used to pinpoint the target selected by the user on the screen, then proceed to do a command more efficiently. We hypothesize that the limitations and disadvantages mentioned for touch screen interfaces can be improved by adding gaze capabilities. It is worth mentioning that gaze along with the touch method. Therefore, the targeting can be done via gaze and the selection by touch, which will improve the efficiency and also the screen is not obscured by hand during targeting and even increases the security. The proposed project is based on Event based architecture. CNN algorithm has been chosen to improve the efficiency of the project. The model is easily adaptable to complex, often chaotic environments and the system is easily extendable when new event types appear. The User interface will be designed to be user friendly and comfortable. The design uses a toolkit DLIB for making real world machine learning and data analysis applications. Dlib is majorly used for face detection and facial landmark detection. The system can be divided into two parts. The first part consists of eye typing implementation and the second part is text entry into the system.

Introduction

I. INTRODUCTION

Eye, being the organ of sight, is one among the sense organs .Eyes collect a lot of information about our daily activities and they reveal the information about an individual’s purpose and awareness. People are able to read, write and execute all their work using the sense organs. In contemporary society, being healthy means more than just not being sick. It is essential to not only take care of the bodily aspect but also the psychological one to maintain good wellness, with agreement with the faction. Communication is an important tool to staying engaged. However, physically challenged people with motor disabilities have issues in conveying their words. To empower handicapped people in typing, eye gaze can be used as an input for the keyboard. The users can command the system without a mouse by using eye movement as a mouse cursor and selecting a method for event click. The system will use a camera to capture the eye movements of the user. The user has to select the letter to form a word on the screen and the word gets displayed on screen. The system can be divided into two parts. The first part consists of eye typing implementation and the second part is text entry into the system. Users can express their thoughts by writing using eye gaze technology and CNN algorithms. The eye gaze technology is used for letters recognized. This project is also capable of object selection and word prediction aspects as well. Accurate eye tracking needs high-resolution human eye images, which require a good camera and optimum lighting condition.

II. IDENTIFY,RESEARCH AND COLLECTION OF IDEA

There have been related works done in the past on gaze assisted systems using machine learning and deep learning. Eye Gaze or Eye Tracking is a method of accessing a computer or communication aid Mouse operated by eyes. The system follows your eyes to see where you are looking on the screen. You can then select the item you are looking at by blinking. But eye-tracking methods can be used in all ways we use our eyes. Eye typing, visual search, and human-computer interaction (HCI) for the physically challenged, to name a few.  Gaze is important for security applications to analyze suspicious gaze behaviour. A use case in educational institutes is the automated analysis of the student’s eye gazes during an examination to help minimize malpractices. Real-time eye tracking and eye gazing estimation using deep neural networks. Personal identification numbers are widely used for user authentication and security. Password authentication using PINs requires users to physically input the PIN, which could be vulnerable to password cracking via shoulder surfing or thermal tracking. PIN authentication with hands-off gaze-based PIN entry techniques, on the other hand, leaves no physical footprints behind and therefore offer a more secure password entry option. Gaze based authentication refers to finding the eye location across sequential image frames, and tracking eye centre over time.

This paper presents a real-time application for gaze-based PIN entry, and eye detection and tracking for PIN identification using a smart camera. Financial transactions without human interactions are on exponential rise. While many of the transactions are carried out by the individuals using their hand-held device or their personal computers, still we need to depend on public devices such as bank ATMs and card readers used by the merchant establishments. These transactions in the public domain are normally authenticated by the use of 4-digit personal identification number (PIN). Physical PIN entry using the keyboard / handheld card readers is vulnerable to attacks and leakage. Fundamental requirements for terminal authentication systems are ease of operation, speed of operation and security. Password or PIN entry systems do qualify this fundamental requirement, however these techniques are not safe because they are viewed by malicious observers who use surveillance techniques such as shoulder-surfing or thermal tracking. Intruders try to acquire personal valuable information such as passwords or PIN by peeping over the user's shoulder or by using binocular/ vision magnifying devices. This method is known as shoulder surfing. Another method employed by the intruders is Thermal tracking. The heat transmitted to the keypad is traced by the intruder by the use of false keypads embedded on the original key pad or through thermal tracking cameras. Security problems are compounded due to poor interactions between systems and users.

III. FINDINGS AND STUDIES

A. Bits and Pieces Together

1. Digital financial transactions are becoming more and more common and will soon rule the day. The input of the Personal Identification is necessary for these transactions.(PIN) entered by the user. The PIN is used frequently a means of authentication for numerous applications, including ATMs, unlocking individual gadgets Cybercriminals are those who thermal tracking or shoulder surfing. Finding PIN entry being susceptible to password assaults like shoulder surfing or thermal monitoring. Hackers aim to access passwords or personal information the user's personal identification numbers (PIN) by scanning watching the PIN entering pattern while looking over your shoulder surfing.
2. For user authentication and security, personal identification numbers are frequently employed. PIN-based password authentication requires users to physically enter the PIN, which makes it possible for a password to be cracked using shoulder surfing or infrared imaging. On the other side, PIN authentication using hands-free gaze-based PIN entry techniques leaves no physical traces and thus provides a more secure password entry option. The term "gaze-based authentication" refers to locating the eyes across successive frames of an image and tracking their centres over time. This paper describes a real-time application for eye recognition and tracking for PIN identification utilising a smart camera, as well as gaze-based PIN entering.
3. Another contemporary input method that has the potential to become popular in the future is eye gazing. Because it is quicker and creates the impression that the computer is responding to the user's idea, using eye movement to control the computer enhances the user experience. It can be used either independently or in conjunction with other input technologies, such as eye movement, to confirm the users' intentions for carrying out important tasks and lower the likelihood of error. It may be used by a layperson because it doesn't require any special training.
4. The majority of them employ intrusive approaches to gauge a person's gaze. In this paper, a non-intrusive method for real-time eye gaze tracking using a basic camera is presented. Detecting the eye, monitoring the eye, and detecting the eye's gaze on the screen that a user is looking at are the three main issues we must address in order to track the eye gaze. In this article, we discuss many approaches that can be used to fix these issues quickly and with a high rate of detection. We must determine the functional connection between the user's eye and the location on the screen that they are focusing on. We will estimate the distribution of this function rather than using a neural network to train it. We compute the mean and covariance of this function using the Gaussian process, and we utilise this function to forecast fresh inputs that weren't present in the training set.
5. Users run the danger of having their passwords stolen if they enter them in a public space. In a busy area, the PIN entering might be seen more effectively by nearby enemies. To address this issue, a new method known as cryptography prevention approaches has been developed. Alternative methods have been used instead, and the PIN entry was elegant because to its ease of use and accessibility. The fundamental BW technique is designed to fend off an assault from a human shoulder surfer. A neatly organised numeric keypad is coloured odd in each round. By pressing the unique colour key, a user who is aware of the correct PIN digit can log in. Because humans have a limited capacity for cognitive thought, the IBW approach is considered to be confidential when applied to a human adversary. The IBW approach has additionally shown to be resistant to all hacking attempts.

B. Proposed Approach

Authentication refers to verification of the identity of an user. There exist various types of authentication techniques, starting from simple password based authentication up to behaviour biometric based authentication.

In this paper, a new way of authentication is proposed where the user provides her password through eye gaze. It is based on a graphical password scheme where she can choose her password from a large image data set. At the time of authentication, she needs to recall it and look at the chosen passcode appearing in a display in a correct sequence. The method uses a machine learning technique where a convolutional neural network is used to determine gaze locations using inputs from a simple web camera. It takes her cropped eye as input and provides gaze location as output. Proposed method is a cost effective solution as gaze tracking is done through a simple web camera. The proposed method is also free from attacks such as shoulder surfing, smudge, brute-force attacks. Experiments have been carried out to validate the system. It has been observed to perform accurately for all the volunteers.

First we will gaze the eye inputs by calibrating it with facial mapping. Facial mapping process is scaled down referring to x, y coordinates. The next step is to set the gaze points. Once the points are being set by referring to the model then it sends the points to the eye gaze model so that it should recognize the eye coordinates then calibrate these coordinates to key mapping and then validate the password for ATM transaction with the eye gaze points. Gaze-based PIN entry involves the user entering the PIN code by looking at the PIN pad (Fig. 1). The user stares at each digit of the PIN for a few seconds, and sequentially moves to the next digit with his/her eyes. While the user is viewing the PIN digits on the keypad, the smart camera captures the image of the partial face in sequential frames, computes the eye centre location using implemented image processing algorithms, and records the Cartesian coordinates representing the eye centre in an on-board spreadsheet file, together with the associated image frame number. The eye tracking application is stopped when PIN entry is completed.

A single technology can have a wide scope of applications. Now days the laptops, tablets, smart phones with 3G, 4G technology are very commonly used. We have gone through the popularity and use of touch control in various devices. Similarly, eye tracking technology is also going to take off a big way. The reason behind this is all smart phones have high resolution cameras. This can be combined with eye tracking technology to get data about the user’s vision when the user is performing his routine task. Here is a wide scope for analyzing huge data generated from an eye tracker and applied for solving a specific problem.

C. Use Of Simulation Software

Matplotlib - Matplotlib is a Python library used for plotting the beautiful and attractive Graphs. This is one of the python libraries which provides functions to plot various data sets.

Pandas - Pandas is a software library written for the Python programming language for data manipulation and analysis. In particular, it offers data structures and operations for manipulating numerical tables and time series. It is free software released under the three-clause BSD license. 

Scikit-learn  - Simple and efficient tools for data mining and data analysis. Accessible to everybody, and reusable in various contexts. Built on NumPy, SciPy, and matplotlib. Open source, commercially usable - BSD license.  

NumPy - NumPy stands for Numerical Python. NumPy is a library for the Python programming language, adding support for large, multi-dimensional arrays and matrices, along with a large collection of high-level mathematical functions to operate on these arrays. Scikit-image - Scikit-image is an open-source image processing library for the Python programming language. It includes algorithms for segmentation, geometric transformations, color space manipulation, analysis, filtering, morphology, feature detection, and more.

IV. ACKNOWLEDGMENT

The project team would like to express their gratitude on to their mentor Gourav.J.Pujari and guide Dr.J.D.Pujari and Prof.Varsha.Jadhav for helping and guiding them in completing the project successfully.

Conclusion

Even though some of the methods have been implemented they suffer from their own limitations. High-level design provides an overview of a system, product or process. This overview helps supporting components be compatible with others. A high-level design document or HLD adds the necessary details to the current project description to represent a suitable model for coding. This document includes a high-level architecture diagram depicting the structure of the system, such as the hardware, database architecture, (layers), application architecture flow (navigation), security and technology architecture. The project aims to provide a means of accessing for normal people, handicapped people and for people suffering from MND (Motor Neuron Disease). HLD provides a technical foundation for planning the program. Therefore the model could be comfortably used by normal people as well as disabled people. This project can further be developed or reframed for gaming purposes or more importantly for enhancing the security, voluntary eye movements can be used as security keys for a security system.

References

[1] M. Ms.R Revathy, \"Advanced Safe PIN-Entry Against Human Shoulder-Surfing,\" 2019IOSR Journal of Computer Engineering, Chennai,India, 2019. [2] G. P. B. G. Hari Krishna S M, \"Development of Personal Identification Number Authorization Algorithm Using Real-Time Eye Tracking & Dynamic Keypad Generation,\" Hari Krishna S M,Gautam Pradyumna,Aishwarya B,Chinmaya Gayathri, Pune,India, 2021. [3] N. Mehrube MEHRUBEOGL, \"Real-time Eye Tracking for Password Authentication,\" IEEE, Dr.J.D.PujariTexas,USA, 2018. [4] U. Z. Z. Ramsha Fatima, \"Eye Movement Based Human Computer Interaction,\" Int’l Conf. on Recent Advances in Information Technology, Aligarh,India, 2016. [5] L. NGUYEN, \"Eye Gaze Tracking,\" IEEE, Paris,France, 2020.

Copyright

Copyright © 2023 Ektha Bijapur, Pooja Joshi, Shrinivas Vaidya, Sneha Sarawad, Vaishnavi . This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.