Using Mixed Reality Application for Super Market: A Case Study

Abstract

Augmented Reality and Virtual reality applications are not a new thing in today’s era and are getting popular day by day. Mixed reality is where physical and virtual environments are combined to give life like experience to the customer. Due to user friendly nature of the mixed reality applications, they are mainly developed for end user’s. Supermarkets are frequently visited and crowded place many due to market competition store try to churn in customers with exciting offers and discounts but due to large variety and large quantity of the products combined with different brands there is so much information which gets lost as intended users don’t get to know products which they require due to shelf arrangement and not knowing location of the products to the user leads to loss of the super market. This paper focuses on how mixed reality can be applied in super market environment to help customers find required products and provide inherent user experience to the end user.

Introduction

I. INTRODUCTION

Supermarkets are very large industry and there is huge opportunity according to recent survey Retail markets generate 27.34 trillion USD each year (source: Statista) [1]. Although online markets are getting revolutionized there is very little innovation happening in offline stores i.e. retail super markets. Customers today want easy, intuitive and customized experience while shopping. Due to advancements in technology and data collection it has become easy to create user profiles and recommending items using users buying patterns and shopping history but as these things are not tracked in offline supermarket.

There are multiple factors and metrics that need to be considered while implementing mixed reality for supermarkets like every other system supermarket has its own challenges and opportunities. Major areas that need to be focused on are user experience, shelf life of products, Offers and discount sections, Navigation.

While developing application for super market we need to consider key performance indicators (KPI’s).There are many KPI’s which are used traditionally few of them are Basket Size, Customer Acquisition Cost (CAC), Customer Lifetime Value(CVC), Inventory levels, Frequency of shopping [2]

II. CHALLENGES

A. Large Number of Markers

According to FMI [3] supermarkets have over 35000 products on average. This is very large number of markers and categorization placed in very dense environment, for Mixed Reality based application makers are most important thing as information needs to be shown on marker or consolidated based on the categorization of the products. Although because of the regulations each product in supermarkets have market printed which uniquely identifies product and is mostly in the form of Bar codes and QR codes this can be used to the advantage to create Hybrid model for Mixed Reality using combination of marker and marker less object detection.

For Hybrid model to work we can use YOLO(You Only Look Once) [4] marker less object detection algorithm with combination of Typical Bar code reading algorithms like Bayesian algorithm.[5] Also marker less objects should be added in database with there in door location so that after user filters or searches with queries application can guide user to the product and exact location

B. Indoor Location

One of the main problems that needs to be addressed while developing MR application for supermarket is Indoor location application needs to know exact location of the user to recommend them correct products and navigate to the product shelf. As user will be using their device it is very hard for an application to get exact indoor location but there are some algorithms with hardware setups which can give indoor location for AR/MR application.

One of the most efficient and accurate setup is to place stationary Bluetooth beacons in store which are geotagged and based on proximity and triangulation we can find exact indoor position. This also requires users permission to use Bluetooth

???????C. Data Updates and Synchronization

All the products in the super market are stored in inventory database. As these are existing and robust services which has been used over years we leverage these services to get latest information about the products and trigger events which can help in getting more customers.

All these operations are read heavy and are updated less frequently for this type of application we need to use horizontal scaled microservices having multiple replica of database and deployed on cloud for robust ness. Many public clouds like amazon (AWS), Google (GCP), Microsoft (Azure)  provide these services.

???????D. Recommendation System

When it comes to shopping experience to get maximum customer satisfaction and Objectives we need great recommendation system which will guide user in Mixed Reality environment.

There are multiple ways a recommendation system can be implemented but as there are already services which provide recommendation are well tested , robust and effective. Its better approach to use existing recommendation system rather than creating new system from the scratch. From the output of recommendation system combined with indoor positioning and MR content complete user experience can be given to the user.

???????E. Mobile Eco-systnes

Smartphones come in wide variety of shapes, sizes, and operating systems it is hard to create applications which will run on each platform and work fine. YOLO Algorithm can be implemented using TF lite library which supports all major mobile operating systems and is light weight so that wide range of smartphones can be supported also for some latest phones which support ARCore different application can be build to support AR functionality

III. IMPROVEMENTS

A. Personalization

Various personalization’s can be applied from the data collected from MR application which can be then shown on the app or can be used by the organizations to get insights and change location of the items based on the analytics. Each user connected to the system should have his profile and history maintained to create personalized shopping experience. 

???????B. Localization

As Supermarkets can have multiple products which have their price and information written in the standard language which might not be suitable for many people MR application can be used to get localized translated version of the information which can be seen on top of product creating seamless and intuitive user interface.

IV. SYSTEM ARCHITECTURE

System Architecture for proposed system shown in figure bellow consists of different component from High Level View common practices for application like authentication and web services are omitted from the architecture as study focuses more on the Mixed Reality implementation. All the component in the architecture are already discussed in the challenges part

???????

Conclusion

From the case study and analysis we can conclude that Mixed Realty application can be developed for the super market using YOLO for marker less object detection along with Barcode readers and AR core features. Also we discussed how KPI’s for super market can be improved using MR application.

References

[1] Fairhurst, M. (2023) Grocery retail Kpis for executives to drive a successful omnichannel strategy in 2023, Mercatus. Available at: https://www.mercatus.com/blog/navigating-the-storm-are-old-school-kpis-enough-to-guide-grocery-executives/ (Accessed: 11 May 2023). [2] Huang, R., Pedoeem, J. and Chen, C. (2018) ‘Yolo-Lite: A real-time object detection algorithm optimized for Non-GPU Computers’, 2018 IEEE International Conference on Big Data (Big Data) [Preprint]. doi:10.1109/bigdata.2018.8621865. [3] Ng, X.H. and Lim, W.N. (2020) ‘Design of a mobile augmented reality-based Indoor Navigation System’, 2020 4th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT) [Preprint]. doi:10.1109/ismsit50672.2020.9255121. [4] Sabanoglu, T. (no date) Topic: Retail market worldwide, Statista. Available at: https://www.statista.com/topics/5922/retail-market-worldwide/ (Accessed: 11 October 2022). [5] Supermarket facts (no date) FMI. Available at: https://www.fmi.org/our-research/supermarket-facts (Accessed: 11 May 2023). [6] Tekin, E. and Coughlan, J. (2009) ‘A bayesian algorithm for reading 1D barcodes’, 2009 Canadian Conference on Computer and Robot Vision [Preprint]. doi:10.1109/crv.2009.31

Copyright

Copyright © 2023 Rutuja Balasaheb Chavan, Dr. Sulochana Sonkamble, Dr. Nisha Auti. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.