Morse code Translator Using Eye Blinks

Abstract

A Morse code Translator which translates Morse code into speech and text of any language chosen by the user, Morse code is given as input ,it can be either a recorded video or live feed of a person who is blinking eyes in a sequence .OpenCV is used to take Morse code input ,Mediapipe a google api detects face and maps facial landmarks, These landmarks are used to map various eye co-ordinates and then considered to formulate Eye Aspect ratios ,which in turn determines eye blink .Morse code which has been already loaded in the form of dictionary is going to map with the morse code given through video or cam ,and find the Alphabet assigned to it.

Introduction

I. INTRODUCTION

Communication is a process which is essential in day-to-day life for a person, every person has needs to communicate with people, the mode of communication can be any, it can be either speech, text, signs Etcetera. People communicate using any of the modes mentioned, but some persons are disable, disability can be of many forms like deaf, dumb, blind Etcetera. Disability can be caused due to many reasons, from birth or other medical conditions so every communication mode is not Feasible for every person or the person with whom he/she is trying to communicate.

Also, some people want to communicate in secret so that they can pass messages to specified person in presence of third person who will have no knowledge about that mode, this can be done using Morse code. Morse code is a communication process that uses two distinct signal durations to encode text characters.

In the Morse code there are 26 letters from a to z, one non-Latin letter, numbers, and a small set of punctuations and procedures (signs). Morse code is not case sensitive.

Morse code is accompanied with sequence of dits and dahs. Morse code transmission is measured by dit durations. dah is generally three times longer than a dit. Normally, after each dit or dah in an encoded character, there is a signal absence, called a space, which is equal to the dit duration.

There is a space of duration equal to three dots between each letter of a word, and seven dots separate words Morse code can be memorised and sent in a form that is apparent to human senses, such as sound waves or visible light, eye blinks, so that it can be directly deciphered by people who have learned the technique.  Morse code is frequently communicated through an information-carrying medium such as visible light, radio waves, or sound waves, which is keyed on and off in sequence.  The current or wave is present during the dit or dah's duration and missing between dits and dah's duration. As mentioned, we are using Eye blinks to communicate through Morse code, A short blink is a dit and a long blink is a dah and sequence of the is an alphabet these alphabets together form a word. Jeremiah Andrew Denton Jr. (July 15, 1924 – March 28, 2014) was an American politician and military officer, while answering questions Denton blinked his eyes in Morse code, spelling the word "T O R T U R E" and sent a secret message through his eyes to his US NAVAL peers. This example shows how one can use Morse code in many situations. but one to receive it and understand message given in morse code must know morse code, but now one can simply use this software upload the video and convert the morse code to text, speech and also into any other language they know this software allows users to translate morse code in an efficient, user friendly and cost-effective manner.

II. LITERATURE REVIEW

Technology has a wide area which updates day by day, there are various technologies used to convert various forms of morse codes, sign languages Etcetera. there are a lot of projects which detect faces and eye blinks. Morse code translation can be done can be done using may technologies some of the projects are

1. Using CNN (Convolutional Neural Network) or MTCNN (Multi-Task Cascaded Convolutional Neural Network) which in turn uses three different layers.
2. Using openCV and tree-based algorithms.

III. PROPOSED MODEL

A. Approach

Our approach is by using mediapipe and openCV. Mediapipe is a google api offers cross-platform, customizable ML solutions for live and streaming media. There are various ML solutions available such as Face Mesh model, Object-detection, pose, hands, holistic, Box tracking and many more. Face Mesh Model is a model which is used frequently by developers. Face Mesh model allows users to map 468 3D landmarks upon a face. Bringing real-time augmented reality (AR) applications, a transform module bridges the gap between face landmark estimation and useful AR applications that are commonly used in everyday life.

Steps

1. Connect your cam to OpenCV and load the live input or give already recorded video
2. Apply mediapipe’s face mesh model on input and initialize Morse code
3. Calculate initial EAR’s (Eye Aspect ratios)
4. Check if the EAR’s values are less than the threshold value
5. If yes that’s a blink, then find out whether it’s a long or short blink based on duration
6. Append dits and dahs to create a sequence
7. Map matching character for the sequence
8. Append the character to a string to form a word
9. Translate word string and convert it into speech

B. Modules

1. EAR Formulation: EAR (Eye Aspect Ratio) can be calculated by below formula where each point represents landmark of eye’s X and Y co-ordinate. The total points for each eye is six since we are considering two vertical lines and two horizontal lines.

Where , (p1,p2),(p3,p4) are 2 vertical lines anf (p5,p6) is horzontal line.

1. Determining Blinks and No Blinks: A comparison is made between the ear values of two eyes. the threshold value is calculated Using a formula which contains previous and current time values. if the EAR is less than Threshold then is a blink, else it isn’t a blink. It is determined by the duration of the blink whether it is a long blink or a short blink.
2. Mapping Morse Code: The dits and dahs are then attached once the blinks have been found, this sequence maps with the Morse code in the software if it’s a match then mapped character is append to word string and when user stops the program the word string is converted to speech and text as selected by the user.

IV. RESULT AND ANALYSIS

This program gives speech and text which is understandable by a normal human language, as chosen by user. Assuming the user selects Hindi as the language, the program will translate the word into Hindi text and speech and play the sound using the play sound library. If the word is not a valid one then it takes nearest possible word.

Conclusion

It has been shown that the Eye Aspect ratio is effective at detecting blinks, compared to the traditional method of detecting blinks. For people who are paralyzed or disabled but still have some eye movement, Morse code blink detection can be very useful. Blink detection can be improved with the algorithms and techniques discussed. For these blinks, Morse code conversion is also in progress. Those who are unable to normally communicate may be able to use this method as a means of communicating with those who doesn’t have knowledge in Morse code.

References

[1] Alternative Voice Communication Device using Eye Blink Detection for people with Speech Disorders Srividhya G, Murali S, A. Keerthana, Jaya Rubi [2] BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs Valentin Bazarevsky, Yury Kartynnik, Andrey Vakunov, Karthik Raveendran, Matthias Grundmann [3] Supervised Transformer Network for Efficient Face Detection Dong Chen, Gang Hua, Fang Wen, Jian Sun [4] Attention Mesh: High-fidelity Face Mesh Prediction in Real-time I. Grishchenko, Artsiom Ablavatski, Y. Kartynnik, Karthik Raveendran, Matthias Grundmann [5] Morse Codes Enter Using Finger Gesture Recognition Ricky Li, Minh Nguyen, Wei Qi Yan Department of Computer Science Auckland University of Technology, Auckland, 1010 New Zealand. [6] Real-Time Eye Blink Detection using Facial Landmarks Tereza Soukupova and Jan ´ Cech • Center for Machine Perception, Department of Cybernetics Faculty of Electrical Engineering, Czech Technical University in Prague [7] A Novel Method for Eye Tracking and Blink Detection in video frames Leo Pauly, Deepa Sankar Division of Electronics and Communication Engineering School of Engineering Cochin University of Science and Technology

Copyright

Copyright © 2022 Kavitha Reddy Guda, Sainath Cheparthy, Srikar Gangipally, Pranay Goud Iruvuri. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.