Eye-Tracker to Control Pointer Using Machine Learning

Abstract

For preprocessing and analyzing eye-tracking data In an increasingly digital world, human-computer interaction has become a vital aspect of daily life. Traditional input devices like mice and touchpads may not be accessible to all individuals, especially those with physical disabilities. This project aims to develop an innovative solution to address this issue by creating an eye-tracking- based pointer control system using machine learning techniques. The primary goal of this project is to design and imp element a system that allows users to control a computer\'s cursor or pointer solely through the movement of their eyes. This will be achieved by leveraging state-of-the-art machine learning algorithms to accurately track and interpret the user\'s eye movements. we envision a novel eye-tracking-based pointer control system that can enhance the accessibility and usability of computers for diverse range of users. This technology has the potential to empower individuals with disabilities and make computing more intuitive and inclusive for everyone. The front-end involves Html, CSS, and JavaScript which are the fundamental web technologies for creating the userinterface and displaying the eye-tracking results. and the back-end involves Python. The framework used is Tkinter and OpenCV is used which is a computer vision library that can be used

Introduction

I. INTRODUCTION

In an era dominated by digital interactions, human-computer engagement holds pivotal significance in daily life. This project addresses the accessibility challenges faced by individuals, particularly those with physical disabilities, by introducing an innovative eye-tracking-based pointer control system. In today’s world computers are widely used, but still there is an hindrance for the physically disabled people.They cannot use the computer independently, they need help from someone .To ensure that even people with physical disability can use the computers , and that too independently , the proposed project will ensure that they can operate the computer with muchease. Also this project is useful for all types of people who would use their systems efficiently and with greater speed.This project is not only useful for the physically disabled people, but anyone can use it, because it has been observedthat people after using computer for long hours are facing discomfort in their wrists . The system performs by usinga webcam that will capture live images of the user. A particular area of interest, the eyes in this situation, are considered and some image processing techniques are used to work better in eye tracking. The cursor can be controlled by certain functions in the Python library. The users face and eyes are captured on webcam and the eye movements are used to control the cursor.

II. LITERATURE REVIEW

Machine Learning (ML) plays a pivotal role in the development of eye-tracking-based pointer control systems, aiming to enhance human-computer interaction, particularly for individuals with physical disabilities. This literature review delves into existing research, emphasizing strengths, limitations, and identified gaps in the application of ML in this context.

A. Strengths of Existing Approach

1. Classification Algorithms for Gaze Prediction: Several studies have              successfully implemented classification algorithms like Support Vector Machines (SVM) and Neural Networks to predict gaze points accurately. These algorithms demonstrate high accuracy in interpreting eye movements for precise cursor control
2. Real-Time Adaptation: Existing literature explores real-time ML models that dynamically adapt to users changing gaze patterns during interaction. Real-time capabilities are recognized for their potential to improve user experience in dynamic computing environments.
3. Assistive Technologies for Accessibility: ML-driven eye-tracking systems have been integrated into assistive technologies, contributing to making computers more accessible for users with physical impairments. These applications showcase the positive impact of ML on inclusivity in computing.

B. Limitations in Current Approaches

1. Data Variability in Real-World Environments: Many studies face challenges when transitioning from controlled environments to real-world scenarios, where factors like diverse lighting conditions and individual user characteristics introduce data variability. Addressing real- world variability is a critical challenge for widespread implementation.
2. User-Specific Calibration Processes: Existing systems often rely on user-specific calibration, necessitating individual setup and calibration processes. Simplifying and streamlining calibration for diverse users remains an area of improvement.
3. Privacy Concerns and Ethical Considerations: Privacy concerns associated with the collection and use of eye-tracking data are recognized in the literature. Robust security measures and ethical considerations are imperative for user trust and system deployment.

III. PROBLEM STATEMENT

The Developing an eye-tracking system to control a pointer through machine learning, aiming to enhance user interface interaction by accurately predicting and responding to eye movements.

The goal is to create an eye-tracking-based pointer control system using machine learning, aiming to provide a means for users to control a computer's cursor solely through their eye movements. The project seeks to overcome the challenges posed by conventional input methods, making computing more inclusive and intuitive for a diverse range of users.

IV. METHODOLOGY

A. Model and Algorithm Used

Support vector machines (SVMs): SVMs are a powerful machine learning algorithm that can be used for classification and regression tasks. They have been shown to be effective for eye tracking applications, such as gaze estimation and pointing prediction.

SVMs: SVMs is used for gaze estimation and pointing prediction tasks. SVMs are a powerful machine learning algorithm that can be used for classification and regression tasks. They have been shown to be effective for eye tracking applications, such as gaze estimation and pointing prediction.

Support Vector Machines (SVMs) are applied in eye-tracking ML projects by training on eye movement data. Extracting features like gaze coordinates, SVMs learn to map these to pointer movements. Collected data is labeled for training, and the SVM's accuracy is validated. Once integrated, the SVM controls the pointer in real-time based on eye gaze, offering a precise human-computer interaction. Tuning parameters enhances performance, making SVMs a robust choice for eye- tracking systems controlling pointers.

B. Data Preprocessing Techniques

1. Eye blink removal: Eye blinks can interfere with the accuracy of eye tracking data, so they need to be removed before the data can be used for training machine learning models. This can be done by using a threshold to identify and remove frames where the user's eyes are completely closed.
2. Noise reduction: Eye tracking data can be noisy due to factors such as camera noise, eye movements outside of the field of view, and reflections. Noise reduction techniques can be used to smooth out the data and improve its accuracy. This can be done using filters such as median filters and Gaussian blurs.
3. Feature extraction: Once the noise has been reduced, machine learning models need to be able to extract features from the eye tracking data that are relevant to the task at hand. For example, features such as the user's gaze position, gaze duration, and saccade amplitude can be used to train a model to predict where the user wants to move the cursor.
4. Normalization: It is important to normalize the eye tracking data before training machine learning models. This means converting the data to a common scale so that the model can learn more effectively. This can be done by dividing  the data by the maximum value in each dimension.

V. EXPERIMENTAL RESULTS

1) Accuracy: The percentage of time that the model predicts the user's     intended     cursor      movement      correctly.

2) Precision: The percentage of predicted cursor movements that are within a certain distance of the user's intended cursor movement.

3) Recall: The percentage of user's intended cursor movements that are predicted within a certain distance of the predicted cursor movement.

4) F1 score : A weighted harmonic mean of precision and recall.

These metrics can be used to assess the overall performance of the model in predicting cursor movements. For example, if the model has an accuracy of 90%, precision of 80%, recall of 70%, and F1 score of 75%, then it is predicting the user's intended cursor movements correctly 90% of the time, and 80% of the predicted cursor movements are within a certain distance of the user's intended cursor movement.

The specific model evaluation metrics that are most important for this project will depend on the specific goals of the system. For example, if the goal is to develop a system that is highly accurate, then the accuracy metric would be most important. If the goal is to develop a system that is fast and responsive, then the precision and recall      metrics      would      be      more      important. The developers of the eye tracking pointer control system should choose the model evaluation metrics that are most relevant to their specific goals and use them to track the performance of the model over time.

VI. FUTURE WORK

Improve accuracy and reliability of eye tracking and Develop more intuitive interactions and make easy eye- tracking systems affordable and easy to use.

Conclusion

Traditional input devices like mice and touchpads may not be accessible to individuals with physical disabilities, making it difficult to interact with computers. This project aims to address this issue by creating an eye-tracking-based pointer control system using machine learning techniques. This system would allow users to control a computer\'s cursor or pointer solely through the movement of their eyes. The project leverages state-of-the-art machine learning algorithms to accurately track and interpret the user\'s eye movements, enabling them to interact with computers more effectively. The primary objective of this project is to design, develop, and implement an eye-tracking- based pointer control system that enables users, especially those with physical disabilities, to manipulate a computer\'s cursor solely through the movement of their eyes. The specific goals include leveraging state-of- the-art machine learning algorithms to accurately track and interpret eye movements.

References

[1] B. Rebsamen, C. L. Teo, Q. Zeng, M. Ang Jr. \"Controlling a wheel chair indoors using thought\" IEEE Intelligent Systems, 2007, pp. 18-24 [2] C. A. Chin \"Enhanced Hybrid Electromyogram / Eye gaze tracking cursor control system for hands-free computer interaction\". Proceedings of the 28th IEEE EMBS Annual International Conference, New York City, USA, Aug 30-Sept 3, 2006, pp 2296-2299. [3] J. Kierkels, J. Riani, J. Bergmans, \"Using an Eye tracker for Accurate Eye Movement Artifact Correction\", IEEE Transactions on Biomedical Engineering, vol. 54, no. 7, July 007, pp. 1257-1267 [4] A. E. Kaufman, A. Bandyopadhyay. B. D. Shaviv, \"An Eye Tracking Computer User Interface\", Research Frontier in Virtual Reality Workshop Proceedings, IEEE Computer Society Press, October 1993, pp. 78-84. [ [5] T. Kocejko, \"Device which will allow people suffered from Lateral Amyotrophic Sclerosis to communicate with the environment\", MSc thesis, January 2008. G. A. Myers, K. R. Sherman, L. Stark, \"Eye Monitor\", IEEE Computer Magazine, Vol. March 1991, pp. 14-21. [6] C. Collet, A. Finkel, R. Gherbi, \"A Gaze Tracking System in Man-Machine Interaction\", Proceedings of IEEE International Conference on Systems, September 1997. Intelligent Engineering [7] B. Hu, M. Qiu, \"A New Method for Human- Computer Interaction by using Eye-Gaze\", Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, October 1994. [8] P. Ballard, G. C. Stockman, \"Computer operation via Face Orientation\", Pattern Recognition vol 1. Conference A: Computer Vision and Applications. [9] Proceedings., 11th IAPR International Conference, 1992. https://www.mathworks.com/matlabcentral/fileexchange/ 247-vfm [10] Christian Szegedy et al. “Going Deeper with Convolutions”. In: CoRR abs/1409.4842 (2014). arXiv: 1409.4842. URL: http://arxiv.org/abs/1409.4842.

Copyright

Copyright © 2023 Mahender K, Mahesh Chandra A, Malika. A, Mallesh E, Malleswari Neelam, Mallikarjun O, Ragipati Karthik. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.