Smart Shopping Facilitator for Visually Impaired using Artificial Intelligence

Abstract

It is the state of a person in which one has to depend on others for their own needs. Visual impairment is one of the disabilities of a human being. To date numerous methods had been proposed to enhance the life style of visually impaired and blind people. Still purchasing products in the e-shopping application without others support is tricky one for them. The project describes a system that provides the guidance for them to identify and purchase their products in the e-commerce application. The audio instructions will assist them inside the e-commerce application based on the real time situations. To make the e-commerce in a smarter way the billing system is automated. Hence it eliminates the existing queuing system in the e-commerce. The ultimate aim of this system is to eliminate others support for visually impaired people in shopping and provide them a convenient and sophisticated environment. On implementing this system, it facilitates the blind people shopping, save the customer’s time and promotes business sales.

Introduction

I. INTRODUCTION

This project is developed to make the life of blind people easy. This is very beneficial in case of finding out the description of packaged goods to the blind people and thus helping them in deciding to purchase a product or not especially which are packaged. This is because it becomes very difficult for the blind people to distinguish between the packaged goods. Thus this application really benefits blind and visually impaired people and thus making their work of identifying products easy. In this smart world, no one can end up the day without using any kind of embedded system products. It makes our hu- man life very smarter and to feel comfortable. In worldwide, the great regret factor is visual impairment. Based on the statistics of World Health Organization (WHO) in 2012, 285 million people are visually challenged in the world. Among them 39 million people are blind and 246 million having low power vision. About 90% of them are living in developing countries.

Shopping is a place where people get their daily necessities ranging from food products, clothing, electrical appliances etc. Sometimes customers have problems regarding the incomplete information about the product on sale and waste of unnecessary time at the billing counters. Continuous improvement is re- quired in the traditional billing system to improve the quality of shopping experience to the customers. Now days numbers of large as well as small shopping applications has increased throughout the global due to increasing public demand spending. At the time of festivals, special discounts, holidays, etc. there is a huge rush in shopping mall. At present, many supermarkets still adopt traditional shopping mode and bar code scanning, which is a waste of manpower and material resources. Also, long time waiting to pay and the tire of pushing shopping cart all make clients suffer a lot and may cause passenger volume go down. Consequently, the needs to help reduce queuing time for customers to check-out and to free people from pushing the shopping carts hard have been an urgent issue to tackle. So here the paper introduces a system to reduce and possibly eliminate the total waiting time of customers and can avoid manpower. Here the system helps to find the required products from the shopping cart. Here it uses feature extraction and stop-word filtering technology to identify the product as per your wants and to determine the exact product lists from available items as well as the person. The rate and name of each product taken by the person is added to the bill in addition it is displayed on the screen and is announced using speaker. The billing can be done using announcement techniques.

II. LITERARURE SURVEY

Paper1 : Design Proposal for a Virtual Shopping Assistant for People with Vision Problems Applying Artificial Intelligence Techniques. Big Data Cogn. Computation. 2023, 7, 96. https://doi.org/10.3390/bdcc7020096

Authors: Sheng Li, Yi Zhao and Cunhang Fan

Year : Received: 31 March 2023 Revised: 24 April 2023 Accepted: 28 April 2023 Published: 12 May 2023

Abstract: Abstract: Accessibility is an increasingly important topic for Ecommerce, especially for individuals with vision problems. To improve their online experience, the design of a voice assistant has been proposed to allow these individuals to browse and shop online more quickly and efficiently. This voice assistant forms an intelligent system that can understand and respond to users’ voice commands. The design considers the visual limitations of the users, such as difficulty reading information on the screen or identifying images. The voice assistant provides detailed product descriptions and ideas in a clear, easy-to-understand voice. In addition, the voice assistant has a series of additional features to improve the shopping experience. For example, the assistant can provide product recommendations based on the user’s previous purchases and information about special  promotions and discounts. The main goal of this design is to create an accessible and inclusive online shopping experience for the visually impaired. The voice assistant is based on a conversational user interface, allowing users to easily navigate an eCommerce website, search for

 products, and make purchases. Keywords: artificial intelligence; accessibility; eCommerce.

Paper 2 : An insight into smartphone-based assistive solutions for visually impaired and blind people: issues, challenges and opportunities Authors: Akif Khan Shah Khusro Year: 2021

Abstract: Blind people are confronting a number of challenges in performing activities of daily life such as reading labels on a product, identification of currency notes, exploring unknown spaces, identifying the appearance of an object of interest, interacting with digital artifacts, operating a smartphone’s user interface and selecting non-visual items on a screen. The emergence of smartphone-based assistive technologies promotes independence, ease of use and usability resulting in improved quality of life yet poses several challenging opportunities. We have reviewed research avenues in smartphone-based assistive technologies for blind people, highlighted the need for techno logical advancements, accessibility-inclusive interface paradigm, and collaboration between medical specialists, computer professionals, usability experts and domain users to realize the potential of ICT-based interventions for blind people. This paper analyzes a comprehensive review of the issues and challenges for visually impaired and blind people with the aim to highlight the benefits and limitations of the existing techniques and technologies. Future research ventures are also highlighted as a contribution to the field.

Paper 3 : Tools and Technologies for Blind and Visually Impaired Navigation Support Authors: Bineeth Kuriakose, Raju Shresthaand and Frode Eika Sandnes Year: 2020

Abstract: The development of navigation tools for people who are visually impaired had become an important concern in the research area of assistive technologies. This paper gives a comprehensive review of different articles published in the area of navigation solutions for people who are visually impaired. Unlike other review papers, this review considers major solutions that work in both the indoor or/and outdoor environments which are based on different technology. 10 From the review, it became clear that the navigation systems proposed for the target users lack some core features that are quite important for independent navigation. Also, there can be instances in which humanitarian conditions also have to be considered in the navigation system design. Based on these findings, a set of recommendations are also given which can be considered in the future design of navigation systems for blind and visually impaired people.

III. PROBLEM DEFINITION AND SCOPE

A. Problem Statement

People who are sighted have the luxury of looking at items and reading the labels on clothing to make decisions about what to purchase. People who are visually impaired can feel the items, but they cannot easily tell the color of the garment, the type of clothing, or the care of the garment. In essence, the fashion industry is criminates against visually impaired people by not making the standard information available to them in a form that they can read. The fashion industry should identify an acceptable way for visually impaired individuals to identify specific characteristics of interest about the clothing they purchase.

B. Software Contexts

We are using Operating system like Windows XP Professional, Java Version JDK 1.7 & above, Coding Language JAVA/J2EE Back-End(DB) MYSQL 5.5, Web Server Apache Tomcat/XAMPP.

C. Goals and Objective

This app useful for blind person shopping.

The main objective of Ambient Assisted Living is to provide blind people the facility to shop themselves without the help of anyone or detect household stuffs by themselves.

• The scope is to provide the facility of text to speech conversion for impaired person to shop independently by solving the navigation problem for blind person with less number of hardware requirements.
• To design this app useful for blind person shopping.

???????D. Statement Of Scope

It provides the facility of text to speech conversion for independent shopping of person. The areas of improvement are: Smart Phone can be replaced by any other device if available and the information of items above and below the currently detected item will be played in the users smart phone.

???????E. Applications

• Blind People.
• Normal People.
• Online Shopping center.

IV. PROJECT PLAN

A. Project Estimates

We are using waterfall model for our project estimation.

1. Requirement Gathering and Analysis: In this step of waterfall we identify what are various requirements are need for our project such are software and hardware required, database, and interfaces.
2. System Design: In this system design phase we design the system which is easily understood for end user i.e. user friendly. We design some UML diagrams and data flow diagram to understand the system flow and system module and sequence of execution.
3. Implementation: In implementation phase of our project we have implemented various module required of successfully getting expected outcome at the different module levels. With inputs from system design, the system is first developed in small programs called units, which are integrated in the next phase. Each unit is developed and tested for its functionality which is referred to as Unit Testing.
4. Testing: The different test cases are performed to test whether the project module are giving expected outcome in assumed time. All the units developed in the implementation phase are integrated into a system after testing of each unit. Post integration the entire system is tested for any faults and failures.
5. Deployment of System: Once the functional and nonfunctional testing is done, the product is deployed in the customer environment or released into the market.
6. Maintenance: There are some issues which come up in the client environment. To fix those issues patches are released. Also to enhance the product some better versions are released. Maintenance is done to deliver these changes in the customer environment.

All these phases are cascaded to each other in which progress is seen as flowing steadily downwards like a waterfall through the phases. The next phase is started only after the defined set of goals are achieved for previous phase and it is signed off, so the name “Waterfall Model”. In this model phases do not overlap.

???????B. Risk Management wrt NP Complete Analysis:

1. NP Complete: Using PTTP algorithm it is possible to find out the optimized solution in polynomial time by reducing the actual problem P.
2. Risk Analysis: The risks for the Project can be analysed within the constraints of time and quality.

???????C. Project Schedule

The project schedule starts from Aug 2023 and ends in March 2024. The different phases identified are:

1. Requirement Analysis.
2. Requirement Specification.
3. System Design.
4. Detailed Design.
5. Coding.
6. Testing.

The Gantt chart as shown below represents the approximate schedule followed for the completion of each phase:

VII.  DETAILED DESIGN DOCUMENT

A. Architectural Design

According to WHO, there are around 161 million people who are visually impaired people and around 36 million are blind. These people are taking help from family member or friends to each and every for their needs. It is a difficult for blind people to choose clothes with different colors or they find difficult to online shopping. A system is developed to assist blind people that automatically recognizing clothing patterns and colors. Camera-based clothing pattern recognition is a problematic due to many clothing pattern and corresponding large intra class variations. Existing texture analysis methods focus on textures with variations in viewpoint and scaling. The level of accuracy in clothing pattern recognition cannot achieve by texture analysis methods. Here, we introduce a system to assist color blind people to select different clothing colors along with different categories. The system contains three major components:

Database which contain clothing images, a microphone for speech command in- put.

Speech converter that convert the input speech into text format.

Display of resulting items on the screen of desktop or wearable computer along with speech.

The system process the voice input, captured voice input is acoustic sound waves that are to be converted into the character string format for semantic interpretation. The result of character string format obtained is then proceed for the global and local feature identification, global feature such as the gender and category, category such as pant, sari, dresses, shirt and so on while the local feature such as red pant then the red is the local feature of the global feature. Conventional speech recognition systems use Gaussian mixture model (GMM) based hidden Markov models to represent speech signals. HMMs based speech recognition systems can be trained automatically and are simple and computationally feasible to use. However, one of the main drawbacks of Gaussian mixture models is that they are statistically inefficient for modeling data that lie on or near a non-linear manifold in the data space. We are using Deep learning, sometimes referred as representation learning or unsupervised feature learning is a new area of machine learning.

IX. ACKNOWLEDGMENT

It gives us great pleasure in presenting the preliminary project report on "Smart Shopping Facilitator for Visually Impaired using Artificial Intelligence web application".

We would like to take this opportunity to thank our internal guide Prof. Amrin Sheikh and Prof. Radha Tripathi for giving us all the help and guidance we needed. We really are grateful to them for their kind support.. Their valuable insights were very helpful.

We are also grateful to Prof. Sagar Rajebhosale, Head of Computer Engineering Department and Keystone School of Engineering, for his indispensable support and suggestions.

Conclusion

The Assistant close Living system permits the visually impaired folks to buy problem free. It eliminates the dependence on others and therefore takes care of persons comfort. It conjointly helps them to navigate within the looking complicated and find things with none issue. It provides the power of text to speech conversion for freelance looking of person. The areas of improvement are: sensible Phone may be replaced by the other device if obtainable and therefore the data of things on top of and below the presently detected item are contend within the users sensible phone .

References

[1] Design Proposal for a Virtual Shopping Assistant for People with Vision Problems Applying Artificial Intelligence Techniques. Big Data Cogn. Comput. 2023, 7, 96. https://doi.org/10.3390/bdcc7020096 Academic Editors: Sheng Li, Yi Zhao and Cunhang Fan Received: 31 March 2023 Revised: 24 April 2023 Accepted: 28 April 2023 Published: 12 May 2023. [2] Xiaodong Yang, Shuai Yuan, andYingLiTian, Assistive Clothing Pattern Recognition for Visually Impaired People, IEEE transactions on human machine systems, vol. 44, NO. 2, APRIL 2019. [3] L. Atzori, A. Iera, and G. Morabito, From Smart Objects to Social Ob- jects: The Next Evolutionary Step of the Internet of Things, IEEE Comm., vol. 52, no. 1, 2020, pp. 97105. [4] Rubi, Chhavi Rana, A Review: Speech Recognition with Deep Learning Methods International Journal of Computer Science and Mobile Com- puting. Vol. 4, Issue. 5, May. [5] Xiaodong Yang, Shuai Yuan, andYingLiTian, Assistive Clothing Pattern Recognition for Visually Impaired People, IEEE transactions on human machine systems, vol. 44, NO. 2, APRIL 2019. Kulyukin, V., Kutiyanawala, A. Accessible Shopping Systems for Blind and Visually Im- paired Individuals Design Requirements and the State of the Art The Open Rehabilitation Journal. [6] Nicholson, J., Kulyukin, ShopTalk: Independent blind shopping verbal route directions barcode scans In: Proceedings of the 30th Rehabilita- tion Engineering and Assistive Technology Society of North America (RESNA), Phoenix, Arizona (2022). [7] Portable Camera-Based Assistive Text and ProductLabel Reading from Hand-Held Objects for Blind Persons Chucai Yi, Student Member, IEEE, Yingli Tian, Senior Member, IEEE and Aries Arditi. [8] Mobile Camera Based Text Detection and Translation Derek Ma,Department of Electrical Engineering Stanford University Qiuhau Lin Department of Electrical Engineering Stanford University, Tong Zhang Department of Mechanical Engineering Stanford University. [9] C. Magnusson, K. Rassmus-Grohn, C. Sjostrom, H Danielsson, \"Navigation and recognition in complex haptic virtual environmentsreports from an extensive study with blind users\", Proc. Eurohaptics 2019. [10] C. Sjostrom, \"Touch Access for People with Disabilities\", CERTEC Lund University Sweden, 2020. [11] J Suryaprasad, B O Praveen Kumar, D Roopa, A K Arjun, \"A Novel Low-Cost Intelligent Shopping Cart\", Proceedings of the 2nd IEEE International Conference on Networked Embedded Systems for Enterprise Applications NESEA 2011, December 8–9, 2021. [12] Kulyukin, V., Kutiyanawala, A. Accessible Shopping Systems for Blind and Visually Im- paired Individuals Design Requirements and the State of the Art The Open Rehabilitation Journal. [13] Xiaodong Yang, Shuai Yuan, andYingLiTian, Assistive Clothing Pattern Recognition for Visually Impaired People, IEEE transactions on human machine systems, vol. 44, NO. 2, APRIL 2014. [14] Rubi, Chhavi Rana, A Review: Speech Recognition with Deep Learning Methods International Journal of Computer Science and Mobile Com- puting. Vol. 4, Issue. 5, May. [15] L. Atzori, A. Iera, and G. Morabito, From Smart Objects to Social Ob- jects: The Next Evolutionary Step of the Internet of Things, IEEE Comm., vol. 52, no. 1, 2014, pp. 97105.

Copyright

Copyright © 2024 Amrin Sheikh , Radha Tripathi, Aditya Dayanad More, Yash Shivshankar Sankh, Atharva Bhausaheb Sarode, Avinash Ankush Kakade. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.