Billing System using Machine Learning

Abstract

The system of using pre-made bar codes to identify a product during its billing process is time-consuming and labour intensive. The relatively unique barcode needs to be first produced, then it must be manually attached to the product. This requires a lot of pre-processing work on the products to make them ready for identification and classification. This paper presents an alternate system that works on the principle of using the products’ natural characteristics like its discrete and distinguishable looks to identify and classify them during the billing process. It’s mimicking the human way of identifying and distinguishing the products. To implement this system, we need to move away from conventional methods of programming and use a different paradigm for designing software systems based on an artificial intelligence concept i.e., machine learning. We use machine learning techniques to design the working philosophy of this system. The algorithms in Deep Neural Networks which is a part of Artificial Neural Networks, help in creating a model to base our software system’s operational logic. Especially the models based on Convolutional Neural Networks have been proven to be efficient in providing a model for image classification. This paper discusses the abstracted software system from the base billing process without worrying about the hardware environment. We choose Python and its web framework Django to design the UI to implement this system over a distributed network within any establishment that needs to incorporate this process so that each node that has to process billing need not have to adhere to the hardware requirements imposed on them to run the various CNN models which are reliant on the GPU-based tensor architecture of TensorFlow. The system also provides mechanisms for inventory management over distributed networks and simple data analytics based on local sales.

Introduction

I. INTRODUCTION

The barcode system is widely used to identify the products when automating the billing process. While this reduces the time needed to bill an item relatively compared to the manual method of billing, it is still not as efficient, because the products need to be individually affixed with barcodes every time.

This process is not only tedious but also labour intensive. Items with variable quantities also exist in most stores, these items cannot be billed using barcodes as they are packaged during checkout and cannot be affixed with barcodes without creating new barcodes, this is again a time-consuming redundant process.

This system of barcodes is so widely used because it makes the identification of an entity by a machine easy as the machine can understand and easily differentiate the barcodes compared to other natural characteristic data like the physical appearance of an entity. Machines have a tough time recognizing images and other visual data using conventional technologies, thus we must look for a new paradigm of design techniques that change the way machines work.

We can find that in the field of Artificial Intelligence as it deals with designing machines that mimic human behaviour, the design philosophy we are looking for.

The technology we require can be found in Machine Learning. It uses a different working philosophy compared to conventional programming. In conventional programming techniques, programmers create rules for machines that they can later use to change the input to the user’s desired output, but in machine learning, programmers give the computer enough input and output so that it can create its own rules based on the input to output associations and then use those rules to work on data which is similar and homogeneous to its original input-output data.[1] It gave rise to software systems that create rules for programs by studying the data and evolving. Machine learning again has further advancements in its subfields, especially Deep Learning. Deep earning is a subfield of machine learning that deals with algorithms stimulated through the structure and working of the human brain called Artificial Neural Networks [3].

We will discuss some of the terminologies and mechanisms that can be used to create a billing system using the various technologies and sub-fields of machine learning in this paper.

II. RELATED WORK

Wo Chang et .al [2] remarked that never in history have we humans created so much digital data and the collective digital footprint of humanity continues to grow almost exponentially. Even though Machine Learning had a relatively early start in 1952 by Arthur Samuel [18], it did not take off due to the lack of data needed to effectively create systems that can be trained with machine learning. Machine learning is seeing a lot of traction and development in current times mainly because of the advancements in Big Data [2] and various other data analysis and data science fields and mainly the availability of data for the machines to be trained. All these factors lead to an unprecedented advancement drive in machine learning using all the data we currently hold to train and create models that can again learn and grow.

M. I. Jordan et .al [1] suggested that Machine learning addresses the question of how to build computers that improve automatically through experience. Machine Learning can be broadly classified into three categories, they are, reinforcement learning, unsupervised learning, and the most widely used supervised learning [1]. Our desired system requires supervised ways of learning so we can implement computer vision and object detection. Supervised learning is widely used in various classification tasks and since image recognition is also a way of classification, we need to use techniques based on supervised learning. Machine Learning is currently being extensively used to create systems in the fields of web search, computational biology, finance, E-commerce, space exploration, robotics, Information extraction, social networks and system debugging [4].

Yann LeCunn et .al [5] suggested that deep neural networks let in computational techniques which might be composed of multiple processing layers to study representations of statistics with multiple ranges of abstraction. These new strategies have substantially progressed the development and improvements in speech recognition, visual object recognition, object detection [4]. and many different domains which include drug discovery and genomics

Ajeet Ram Pathak et .al [4] suggests that Object detection is the first step in any visual recognition activity, it is the methodology of deciding the example of the class to which the object has a place and assessing the area of the object by yielding the bounding box around the object. Identifying a single example of class from the image is called single-class object detection while identifying the classes of all objects present in the image is known as multi-class object detection. Various difficulties like fractional/full impediment, differing light circumstances, presents, scale, and so forth are required to have been taken care of while playing out the object detection. After successfully detecting the image, the next step involves classifying the image.

Deepika et .al [6] suggests that image classification is the name given to the technique where a computer can break down an image and distinguish the group the image falls under. These groups can later be used to classify a new image.

One of the most famous deep neural networks is the Convolutional Neural Network (CNN) [3]. It is named so after the mathematical linear operation among matrices called as convolution. It takes this call from a mathematical linear operation among matrixes called convolution. The CNN has a top-notch performance in device gaining knowledge of problems. Especially the applications that deal with picture data, including the biggest image dataset (Image Net), computer vision, object detection and natural language processing and the outcomes completed have been very splendid [3]. The results of ILSVRC 2010 shows us that systems built on convolutional neural networks are marginally better than any past classification system by achieving the lowest recorded error rate of 17% [3].

Athanasios Voulodimos et .al [7] has concluded that convolutional neural networks can be used as they provide efficient mechanisms for object detection as they support feature learning, translation invariance and generalization. CNNs are widely used in other fields that also require object detection, like facial detection [7]. Google’s FaceNet [22] and Facebook’s Deep-Face [23] use CNNs to implement computer vision.

III. ANALYSIS AND INTERPRETATION

Hardware specification for this system is done by abstracting most of the specific hardware and only stating the core functionality needed in the hardware. This system requires four basic mechanisms, A network of computer(s) for running and using the system, A video capture device to get the image data of the system, a weighing machine to record the weight of the product and a printer for printing the bill. Figure 1 shows the block diagram for the proposed system.

Software analysis is the main focus of this paper and after analysis of available techniques in our literature review, we begin the design of the software system with three basic functionalities as shown in figure 2, namely:

1. Classification of the products
2. Inventory Management
3. Data analytics

A. Classification of products

This is the core functionality of our system. From our literature survey, we found out that a model of our required software system can be created using an implementation of an algorithm based on deep neural networks.

Most deep learning strategies utilize neural network architectures, which is the reason deep learning models are regularly alluded to as deep neural networks. The expression "deep" ordinarily alludes to the quantity of stowed away layers in the brain organization. While deep neural networks can have upwards of 150 secret layers, normal neural networks usually don’t exceed four. Deep learning models are prepared by utilizing enormous arrangements of marked information and natural brain neural designs that create features straightforwardly from the information without the requirement for manual feature extraction.

A CNN convolves learned highlights with input information, and utilizations 2D convolutional layers, making this design appropriate to handling 2D information, for example, images [20]. CNNs wipe out the requirement for manual component extraction, so you don't have to distinguish highlights used to characterize pictures. Various sub-modules under this use case are discussed below.

1. CNN Architecture: The common architecture of a CNN is called VGG16 [8] The CNN works by removing highlights straightforwardly from pictures. Prior training is not done on the required and distinguishable features, they are instead trained by the model while it works on the datasets. This automatic mechanism of deep learning makes it exceptionally precise for the computer vision task that we require. Object recognition calculations like YOLO [24] can perceive and find the item in a scene and can find different articles inside the picture. The system’s operational model is created using convolutional neural networks in order to efficiently implement object detection [7]. Utilizing GPU speed increase can accelerate the interaction fundamentally. Utilizing MATLAB [25] with a GPU lessens the time expected to prepare an organization and can cut the preparation time for a picture arrangement issue from days down to hours. MATLAB has levels of encapsulation that allows automatically utilizes the GPU depending on its availability to reduce programmer intervention [8].
2. Frameworks: We make use of an extensive software library for implementing object classification called TensorFlow [21]. Created and maintained by Google, it has the most robust set of classes and methods we can use for our system [19]. It uses dataflow graphs to model a neural network and various other machine learning algorithms. Although the main tools are written in C++ and CUDA(Compute Unified Device Architecture) [26], its python-based API is the most commonly used version as python is a lot more versatile compared to other languages.
3. Dataset: We use a dataset called Grocery Store created by Klasson et al. in [12], it has image data of fruits, vegetables, dairy and juice products from 18 different grocery stores. This dataset has 5125 images belonging to discrete 81 classes, with each class containing 30 to 138 images. They also have 46 coarse scaled classes. The model will be trained and tested using this dataset during development. After production, the model will be trained using product image data available to the individual establishments that decide to implement this system. This creates a customized model for each establishment’s unique needs.
4. High-level APIs: We make use of two high level APIs namely Keras [27] and Estimator [28] in order to implement the models. Keras has many modules which be used to implement different functionalities in the model and take care of the graph construction. Estimators simplify the model slightly by using the model’s hyperparameters. Custom models can be created using Estimator functions and applying Keras modules to construct the main overall structure for data processing.
5. Localization: localization is usually implemented using the sliding-window technique when working with convolutional neural networks, but due to its poor efficiency and slow speed, so we use a new cascading architecture created by Jonathan Thompson et .al [9]. It is not only efficient but also compensates for the accuracy loss in pooling by combining both the scales of convolutional networks.

Figure 3 shows the block diagram for the operational process of the trained convolutional neural network implemented. The CNN model returns class data to Django after receiving image input from the Django handler. The billing app then retrieves the item data from the item database like its full name and price to sends it to the main Django MVC controller for displaying the product details and effectively recording it. Thus, the process of object detection and classification can be implemented using this architecture.

???????B. Inventory Management

The users need to be able to create new items, read existing item data, update existing item data and delete item data. These use cases can be implemented using an MVC capable web-based framework for distributed computing capabilities. Thus, we use Django. A basic interface for users to manage the establishment’s inventory can be made using HTML5, CSS and various styling frameworks like Bootstrap and jQuery. We use Django as middleware and MySQL as the backend to store user, item data. Each of the main use cases is deployed as individual Django apps to adhere to the DRY principle. Like most MVC frameworks, Django has support for simultaneously distributed computing and thus there is no danger of data asynchronization.

We chose MySQL for this system as it is capable of rapid parallel processing and robust relational models. All three subroutines listed here require database access which makes nodes make multiple types of requests to the database. MySQL is reliable and is used as an industry-standard in many production systems.

???????C. Data Analytics

We can implement the required data analytics using python. Various graphical representations of data can be made with periodical analysis to implement a basic level of business intelligence. We also provide situational business analytics related to the trending item sales on each node. This enables the establishment managers to plan the placement of the most sold items in areas with the most traffic in order to maximize sales nearby billing nodes. Another implementation of business analytics is related to item-item relationships. There are certain items that are dependent on the sales of another item. This system provides analytics to maximize the sales of these related items by recommending discounts when one of the items in the relationship starts trending.

After a thorough analysis of the available technologies that can be used to replace the current system of using bar codes and RFIDs, we have proposed a system that uses Machine Learning and Convolutional Neural Networks to establish product classification instead of the existing one. The identification of products will be done using a live feed of the product. The product need not be changed or altered in order to be identified as all of the processing work that is needed to identify the product is done once and can be used for the product till it changes its recognizable image. This system is relatively less labour and time-intensive and reduces the cost of maintenance and operation for small to medium scale retail outlets. Many grocery shops sell non-pre-packaged products and now incorporate them into the billing process, which could not have been possible with the current system. 

Conclusion

The continuous and rapid development of powerful computing equipment along with huge advancements in data analytics and big data made object detection technology based on deep learning possible, this helps people to implement efficient versions of systems we already have. One of the areas we are focusing on is the system of semi-autonomous billing. After reviewing some of the commonly used detector algorithms, the available technologies needed for our proposed mechanism and related work, we have successfully discussed the design of a system that has a distributed architecture to support multiple billing counters in large establishments like supermarkets and hypermarkets, can bill products without the requirement of the tedious and labor intensive task of creating barcodes and affixing them by using the product’s visual characteristics and can perform business analytics to implement business intelligence. Through the use of convolutional neural networks, we have created the design to a system that can improve itself by updating its model based on various reinforcement techniques. All these factors help in concluding that the proposed system is marginally efficient than the system currently being used.

References

[1] Jordan, M.I., Mitchel, T.M (2015), “Machine learning: trends, perspectives, and prospects”. In Science (vol. 349, issue 6245). [2] Chang, Wo L (2018), “NIST Big Data Interoperability Framework: Volume 3, Use Cases and General Requirements”. In National Institute of Standards and Technology (NIST) Special Publication 1500-3r1 [3] A. K. Jain, Jianchang Mao and K. M. Mohiuddin (1996) \"Artificial neural networks: a tutorial\". in Computer, (vol. 29, no. 3, pages 31-44). [4] Ajeet Ram Pathak, Manjusha Pandey, Siddharth Rautaray (2018), “Application of Deep Learning for Object Detection”. In Procedia Computer Science (vol. 132, pages 1706-1717). [5] Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E Hinton. (2012). “Imagenet Classification with Deep Convolutional Neural Networks”. In Advances in Neural Information Processing Systems, (pages 1097–1105). [6] LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton (2015), \"Deep learning\" In Nature (vol. 521, pages 436-444). [7] Athanasios Voulodimos, Nikolaos Doulamis, Anastasios Doulamis, and Eftychios Protopapadakis (2018), “Deep Learning for Computer Vision: A Brief Review”. In Computational Intelligence and Neuroscience (vol. 2018) [8] Zhao, Zhong-Qiu & Zheng, Peng & Xu, Shou-Tao & Wu, Xindong (2019), “Object Detection with Deep Learning: A Review”. In IEEE Transactions on Neural Networks and Learning Systems (pages 1-21) [9] Tompson, Jonathan and Goroshin, Ross and Jain, Arjun and LeCun, Yann and Bregler, Christoph (2015), “Efficient Object Localization Using Convolutional Networks”. In IEEE Conference on Computer Vision and Pattern Recognition. [10] Jaswal, Deepika & Vishvanathan, Sowmya & Kp, Soman (2014), “Image Classification Using Convolutional Neural Networks”. In International Journal of Scientific and Engineering Research. (vol. 5, pages 661-1668) [11] Meiyin Wu and Li Chen (2015), \"Image recognition based on deep learning\". In 2015 Chinese Automation Congress (CAC) (pages 542-546). [12] Klasson, Marcus & Zhang, Cheng & Kjellström, Hedvig. (2019), “A Hierarchical Grocery Store Image Dataset with Visual and Semantic Labels”. In IEEE Winter Conference on Applications of Computer Vision, (vol. 491-500) [13] Sowmya V., P., S. K., and Deepika, J., “Image Classification Using Convolutional Neural Networks”, International Journal of Scientific & Engineering Research, vol. 5, no. 6, p. 06/2014, [14] R. Girshick (2015), “Fast R-CNN,”. In Proc. IEEE Int. Conf. Comput. Vis. (ICCV), (pages 14401448). [15] S. Ren, K. He, R. Girshick (2017), and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks”. In IEEE Trans. Pattern Anal. Mach. Intell(vol. 39, no. 6, pages 1137-1149). [16] K. He, G. Gkioxari, P. Dollár, and R. Girshick (2017), Mask R-CNN”. In Proc. IEEE ICCV, (pages 2980-2988). [17] G. Ghiasi, T.-Y. Lin, R. Pang, and Q. V. Le (2019), “NAS-FPN: Learning scalable feature pyramid architecture for object detection”, (pages 7029-7038). [18] Kucak, D[anijel]; Juricic, V[edran] & Dambic, G[oran] (2018). “Machine Learning in Education - a Survey of Current Research Trends “. In Proceedings of the 29th DAAAM International Symposium Published by DAAAM International, (pages 0406-0410) ISBN 978-3-902734-20-4, ISSN726-9679, Vienna, Austria [19] Pang, B., Nijkamp, E., & Wu, Y. N. (2019), “Deep Learning with TensorFlow: A Review”. In Journal of Educational and Behavioral Statistics, (vol. 45 , pages 227–248) [20] Anwar Hossain, Md. Shahair Alam Sajib, Md. (2019), “Classification of Image using Convolutional Neural Network (CNN)”. In Global Journal of Computer Science and Technology, ISSN 0975-4172. [21] TensorFlow website, [Online]. Available at: https://www.tensorflow.org/learn [22] FaceNet documentation, [Online]. Available at: https://openbase.com/js/facenet/documentation#dependencies [23] DeepFace: Closing the Gap to Human-Level Performance in Face Verification, [Online]. Available at:https://research.facebook.com/publications/deepface-closing-the-gap-to-human-level-performance-in-face-verification/ [24] YOLO: Real-Time Object Detection, [Online]. Available at: https://pjreddie.com/darknet/yolo/ [25] MATLAB - Mathworks, [Online]. Available at: https://in.mathworks.com/products/matlab.html [26] CUDA Zone | NVIDIA Developer, [Online]. Available at: https://developer.nvidia.com/cuda-zone [27] Keras – The python deep learning API [Online]. Available at: https://www.tensorflow.org/learn [28] Estimator – TensorFlow Core, [Online]. Available at: https://www.tensorflow.org/guide/estimator

Copyright

Copyright © 2022 Nitish Lokesh, Dr. Pawan Kumar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.