Design of Smart Cane with Integrated Camera Module for Visually Impaired People

Abstract

In the world, millions of visually impaired or blind people are living. They are always dependent on other people or trained pets to move through an unknown environment. A typical walking stick cannot meet the safety monitoring and guidance requirement of blind people. This paper focuses on designing and developing a smart cane capable of sensing and signaling the environmental features around visually impaired people. The proposed smart cane has two ultrasonic sensors for measuring the distance between the smart cane to the stationary and hanging objects found to be an obstacle. Alert signal is provided by the actuator when the obstacle is falling inside a threshold level of 2 meters. The design integrates a camera module ESP-32 to capture the moving object/people\'s image in addition to the obstacle detection. The camera module gets enable signal from the PIR sensor to capture the image of the moving object/person and save processor time. It also provides information about the situation around the blind person and their relatives in case of an emergency. It provides the implementation cost and power requirement of the smart cane to develop a camera-enabled stick .

Introduction

I. INTRODUCTION

World Health Organization (WHO) report of blindness and visual impairment 2020 states that one billion people are visually impaired, totally blind, and vision loss for people over 50 years old . The increase in the rate of visually impaired problems keeps increasing day by day due to various reasons like a reflective problem, cataract, etc. Those people highly depend on other people or their trained pet animals to ensure their safety  while moving through an unknown environment. A visually impaired person commonly uses a walking stick when they do not have any people to help them. The basic walking stick is a mechanical design that can be foldable and ensure safety but restricted to their strength to handle it. A smart cane is an electronically equipped stick powered by a battery and indicates the challenges before the victim using microcontroller-based sensing and control. It focuses on detecting the obstacles ahead, spotting the person stored in the database, and connecting the victim to their close relations using GSM tracking. It helps the visually impaired person formoving inside the house or through unknown terrain. It leads to a smart cane design capable of assisting the visually impaired with a cost-effective design that makes their life easy and comfortable. The paper proposes an intelligent solution to blind people for making their daily lives convenient and connected to their close relations to take care of them from remote locations

Many researchers worked on the design and development of smart cane with various facilities and advancements. Smart Stick for the Blind is a favorable solution to travel in an unknown location towards a destination. This system uses an IR sensor, an ultrasonic sensor, and a water sensor to detect various obstacles. However, these systems notify the external environment through an actuation signal if any one of the sensors gets triggered. Recorded audio messages are used with Bluetooth-connected speakers to indicate the distance of the person's obstacles . Ultrasonic sensors are used in many designs to calculate the barrier's location and indicate the static blocks around the person using a group of sensors like ultrasonic sensors and infrared sensors . The IR sensor detects the staircases, and the ultrasonic sensors measure the distance of the puddles and indicate using speech warning messages and vibration motor. 18F46K80 microcontroller, vibration motor, and ISD1932 flash memory detect obstacles in the range of 4 meters within 39 ms as processing time. It gives a suitable message that helps the user move twice the normal speed, ensuring his/her safety. The smart s tick design gives the fast response, low cost, low power consumption, lightweight, and folding ability In recent years, many pieces of research focus on image data for various safety applications. It is employed in that proposes biometric instruments of blind peoples to sense and detect obstacles. They are provided with spectacles to wear on, embedded with ultrasonic distance measurement scale equipment and a camera with a headphone. The device uses the target finding using ultrasonic sound. The camera in the device stores the face image and helps identify the person and recall from the memory whenever the person reappears before him. Pothole detection  for the visually impaired uses a camera that captures the image at 15 frames per second, and based on the concept of image processing, the pothole is detected. The problem with this system is the use of a camera makes it expensive, and also, a lot of images captured per second increase overhead and storage requirement.

The warning sound signal has a frequency correlated with the pixel orientation . Global Positioning System (GPS) integration is also getting focus on smart stick design for few applications. It's also noteworthy that a GPS receiver can provide the subject's current location and nearby landmarks. The feedback information indicates the user with a vibration signal or as a sound signal . Consequently, training is necessary to help the user understand the signals and react to them in real-time, having its own time and cost . There is also a proposed solution that helps blind to avoid obstacles around them without holding any sticks or other heavy things. There is a system that is a spectacle module associated with a sonar sensor, the vibrator, which detects the barrier and alerts. The camera calculates the surface's smoothness during daylight and dark using RGB data and sends images to the smart phone via Bluetooth controlled through a microcontroller . The design is a very small, wearable, lightweight, low-cost spectacle for the blind and feasible for their assistance. There is a system designed as a wireless system using Wi-fi communication to the ultrasonic sensor to be comfortable to be carried by the user. The system helps visually impaired people to move around in indoor or outdoor scenarios  Electronic Travel Aids (ETAs) helps to navigate a person by collecting information about the environment and exhibits in a form that allows a blind to understand the environment. It proposes a computer vision-based pothole detection and uneven surface detection approach that helps blind people meet their needs. The system uses laser projected patterns and records the patterns as a monocular video. Upon analyzing the patterns, the features are extracted and predict the path. Over 90% accuracy in detecting potholes is possible with the proposed system that assists blind people in real-time navigation . A sonification process builds the map of the surrounding by the disparity image and converts it to a stereo musical sound. It has information about the features of the environment in front of the user. The sound indication through stereo headphones conveys the blind people about person or object around . There is a sensor fusion-based obstacle avoiding system that utilizes a camera for acquiring the depth information of the surroundings. An ultrasonic rangefinder measure the distance of the obstacle from the stick . Akshi devices employed with Radio frequency identification detection (RFID) reader can detect pedestrian crossing. Obstacles are detected using RFID, ultrasonic sensors, and raspberry-pi fast processing board and available through the AKSHI tracker app. The app allows the guardian to log in and. access a blind person's location through GSM and GPS module.[10] If the device meets an emergency, it communicates to the guardian. It avoids the wrong trigger using a press button interfaced with the mobile app . The Guide Cane uses the same mobile robotics technology as the Nav-Belt. It is a wheeled device pushed ahead of the user via an attached cane. When the Guide Cane detects an obstacle, it steers around it. The user immediately feels this steering action and can easily follow the Guide Cane's new path without any conscious effort . A proposed system for converting the images into sound patterns and provides auditory image representations within human hearing systems . The proposed research work considers the advantages of previous developments with sensors and actuators to develop a smart stick capable of delivering alerts and providing surrounding information through vibrators and messagesignals. It also concerned the processing complexity and affordable cost. It is a simple design with an integrated camera covering the entire foreground information and identify the obstacle's existence and distance from the cane. Two ultrasonic sensors are in place for ensuring the presence of an obstacle and calculate the distance. Ultrasonic sensor one is placed at the bottom to sense the steps or stone in the pathway. Ultrasonic sensor two detects the hanging obstacles like wires or tree branches and indicates it to the user. An integrated camera module gets the image of the person standing in front and compares the face with the database to indicate the user.[3]

II.  EXISTING SYSTEM

Blindness is the condition of lacking discernment because of physiological or neurological components. Different techniques have been established to define the degree of vision misfortune and visual deficiency. The problems that visually impaired people are facing are that they cannot avoid the objects around them while moving, so accident may occur. Various researches have been conducted to minimize the sufferings of the visually impaired people. But it does not reach to the optimized level. The contribution of this paper is to investigate a solution for the visually impaired people. This paper targets to provide a solution that helps blind to avoid obstacles around them without holding any sticks or other heavy things. The system calculated the smoothness of the surface both for the daylight and dark using RGB data through microcontroller and smart phone. The highest smoothness of the surface is achieved by the system is 96.341% and 98.683% for the day-night and dark respectively. It designs and develops a system which is consisting of small, wearable, lightweight, low-cost spectacle for the blind by which they can get assistance to walk easily and detect obstacles around them.

An ultrasonic sensor module,HC-SR04 is used for obstacle detection in the path of the blind person and a buzzer is used to make the person alert. The proposed system is implemented using PIC microcontroller 16F877A. Blind persons can use this walking stick for safe navigation. It can detect obstacle within 5 to 35 cm range of distance

III. PROPOSED SYSTEM

When captured button is clicked,this system will capture the documenting image placed infront of the camera which is connected to raspberry pi through USB.After selecting the process button the captured document image undergoes Optical Character Recognition(OCR) Technology. OCR technology allows the conversion of scanned images of printed text or symbols into text or information that can be understood or edited using a computer program In our system for OCR technology. we are using TESSERACT library Using Text-to speech library the data will be converted to audio. Camera acts as main vision in detecting the image of the placed.

document, then image is processed internally and separates label from image by using open CV library and finally identifies the text which is pronounced through voice. Now the converted text into audio output is listened either by connecting headsets via 3.5mm audio jack or by connecting speakers via Bluetooth.The Raspberry Pi is a computer, and from a connections point of view it doesn’t look much different to a normal desktop computer. It has USB ports for connecting a keyboard and mouse, and a video output port for connecting up a display.Because it is more compact and lower cost than a large desktop PC, it becomes possible to use the Raspberry Pi or other small single board computers (SBCs) as they are

In this project I used Raspberry Pi because of its following features:Raspberry Pi is cheap.Many connected devices.Raspberry Pi is small.Raspberry Pi is powerful.

The main components include the following:Broadcom BCM2835 ARM11700MHz.HDMI out for plugging into a TV or monitor.The audio output also uses HDMI.CSI (Camera Serial Interface) is a serial connector for a camera interface.Ethernet Out for plugging into a hardwired network. USB 2.0 for mouse, keyboard, Wi-Fi, and Bluetooth adapters.Status LEDs.Audio out to an external amplifier.JTAG headers are used for debugging during the development of embedded software and hardware. JTAG stands for Joint Test Action Group.RCA video out for older TVs.consumption of sensor nodes has to be balanced to prolong the WSN lifetime [2]. SD card slot which will contains the operating system and additional data.Micro USB power input for powering the Raspberry Pi.The Raspberry Pi has a Broadcom BCM2835 system on a chip (SoC), which includes and ARM1176JZF-S 700 MHz processor, Video Core IV GPU, and was originally shipped with 256 megabytes of RAM, later upgraded to 512 MB. It does not include a built-in hard disk or solid-state drive, but uses an SD card for booting and long-term storage.

IV. IMPLEMENTATION AND RESULT

A webcamor USB camera is a video camera that feed sits image in real time to a computer or computer network. A USB camera is generally connected by a USB cable, FireWire cable, or similar cable. The common use as a video camera for the World Wide Web gave the webcam its name. Other popular uses include security surveillance, computer vision, video broadcasting, and for recording social videos. Webcams are known for their low manufacturing cost and flexibility ,making them the lowest cost form of video telephony. They have also become a source of security and privacy issues, as some built-in webcams can be remotely activated via spyware.

Speakers.Regardless of their design ,the purpose of speakers is to produce audio output that can be heard by the listener. Speakers are transducers that convert electromagnetic waves into sound waves. Speakers are transducers that convert electromagnetic waves into sound waves.

The power supply section contains a step-down transformer for stepping down the AC 230V into 9V and is rectified using bridge rectifier and filtered using capacitor filtering method and given to PIC as its source voltage, 5V DC. Power supply is a device or system that supplies electrical or other types of energy to an output load or group of loads. A simple AC powered linear power supply usually uses a transformer to convert the voltage from the wall outlet (mains) to a different, usually a lower voltage. If it is used to produce DC a rectifier circuit is employed either as a singlechip, an array of diodes sometimes called a diode bridge or Bridge Rectifier, both for full wave rectification or a single diode yielding a half wave (pulsating) output. More elaborate configurations rectify the AC voltage at first to pulsating DC. Then a capacitor smooth out  part of the pulses giving a type of DCvoltage .The smaller pulses remaining  are known as ripple .Because of a full wave rectification they occur at twice the mains frequency. Finally depending on the requirements of the load, a linear regulator may be used to reduce the ripple sometimes also allowing for adjustment of the output to the desired but lower voltage

References

REFERENCES [1] \"Blindness and vision impairment\" Newsroom of World Health Organization (WHO), 8 Oct 2020. [2] Jismi J, Nikhil Rajan P, Nivya M T, Rakendh C.S, Sijo TcVarghese “Smart Stick for Blind,” IJESIRD, Vol. 3, Issue 9, March 2017. [3] Amy Nordrum, \" Pothole detection for blind,\" IEEE Spectrum, 30 May 2016. [4] Ayat N, Samia M, Mahmoud A. Fakhr, and Ahmed F. Seddik, \"Effective Fast Response Smart Stick for Blind People,\" ICABEE, April 2015. [5] Ayush Wattal, Ashuthosh Ojha, Manoj Kumar “Obstacle Detection Belt for Visually Impaired Using Raspberry Pi and Ultrasonic Sensors,\" National Conference on Product Design, July 2016. [6] V. Diana Earshia, S.M Kalaivanan, K.Bala Subramanian “A Wearable Ultrasonic Obstacle Sensor` for Aiding Visually Impaired and Blind Individuals, \"International Journal of Computer Applications, January 2016. [7] G. P. Fajarnes, L. Dunai, V. S. Praderas, I. Dunai \"CASBLiP - a new cognitive object detection and orientation system for impaired people,\" International Conference on Cognitive Systems, Switzerland, 2010. [8] R. Madli, S. Hebbar, P. Pattar and V. Golla, \"Automatic Detection and Notification of Potholes and Humps on Roads to Aid Drivers,\" in IEEE Sensors Journal, vol. 15, no. 8, pp. 4313-4318, Aug. 2015. [9] M. M. Kamal, A. I. Bayazid, M. S. Sadi, M. M. Islam, and N. Hasan “Towards developing walking assistants for the visually impaired people” Proc. IEEE Region [10] Humanitarian Technology Conference (R10-HTC), Dhaka, pp. 238-241, 2017. [10] Vera, D., Marcillo, D., Pereira, A. \"Blind Guide: anytime, anywhere solution for guiding blind p`eople,\" in Proc. World Conference on Information Systems & Technologies, pp. 353–363. Springer, Cham, Apr.2017. [11] A. S. Rao, J. Gubbi, M. Palaniswami, and E. Wong, \"A vision-based system to detect potholes and uneven surfaces for assisting blind people,\" in Proc. IEEE (ICC), Kuala Lumpur, pp. 1-6, 2016. [12] Balakrishnan G, Sainarayanan G , Nagarajan Rand Yaacob S, \"Wearable Real-Time Stereo Vision for the Visually Impaired,\" Engineering Letters, vol. 14, no. 2, 2007. [ [13] J. Bai, S. Lian, Z. Liu, K. Wang, and D. Liu, \" Smart guiding glasses for visually impaired people in indoor environment,\" in IEEE Trans. on Consumer Electronics, vol. 63, no. 3, pp. 258-266, Aug. 2017. [14] V. S. S. Kaushalya, K. D. D. P. Premarathne, H. M. Shadir, P. Krithika, S. G. S. Fernando, \"AKSHI\": Automated Help aid for Visually Impaired People using Obstacle Detection and GPS Technology,\" International Journal of Scientific & Research Publications, vol. 6, no. 11, 2016. [15] Shoval, Ulrich I, and Borenstein J, \"NavBelt and the guide-cane obstacle-avoidance systems for the blind and visually impaired,” IEEE Robotics and Automation Magazine, vol. 10, no. 1, pp. 9–20, 2003. [15] Meijer. P, \"An Experimental System for Auditory Image Representations,\" IEEE Trans. on Biomedical Engineering, vol.39, no 2, pp. 112-121, Feb 1992.

Copyright

Copyright © 2022 Namrutha U. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.