Glass Virtual Try-On

Abstract

Virtual try-on is a technology that allows people to virtually check the appearance of accessories, makeup, hairstyle, hair color, clothes and potentially more on themselves. The virtual try-on presents many advantages over real try-on, it speeds up the process providing the possibility to test hundreds of products without the need to reach a physical store. The try-on process is performed on a 3D face reconstructed from the input image allowing the user to see the virtual face and glasses from different viewpoints. The try-on process is fully automated and does not require the user to provide anything else but the picture and name of the glasses frames to test.

Introduction

I. INTRODUCTION

Visual experimentation is a new way to shop online. With the advancement of e-commerce, it expands the external marketing channels of retailers to improve interaction between buyers and sellers. The need for beauty and accessories integration is high, while traditional e-commerce sites are shown, methods do not meet the need. Visual aids fill the void in an intangible asset in common online shopping.

During the process of trying on visible mirrors, consumers can choose the glasses they like by comparing the actual wear effect of different glasses on online shopping. The key to researching the system for trying visual glasses is the rapid success of the online shopping experience.

AR calculates the location and angle of the camera image in real time while adding the corresponding images. The virtual world is set at the top of the screen in the real world with real-time interaction. The virtual world is set at the top of the screen in the real world with real-time interaction. The virtual reality test system includes computer vision, virtual reality for taxpayers we see, and image processing technology.

Our application relies solely on standard and open technology such as HTML5, WebGL or WebRTC. This guarantees that it will not expire soon. The deep learning neural network includes a cropped image from the video camera feed. It emits simultaneously whether it is face or not, face shape and light parameters. Then we present the 3D model mirrors over the video server using a home-made 3D engine.

II. PROBLEM – SOLVING

1. Provides virtual try glasses for users.
2. It can be easily integrated into a website or mobile web application.
3. The experience is real-time: the user sees his face as a mirror, but wearing glasses.

III. FEATURES

1. Real-time web based glasses tried.
2. Light reconstruction (ambient + directing).
3. It is very strong in lighting conditions.
4. Mobile is friendly.
5. Advanced 3D Engine.
6. Body-based delivery (PBR).
7. Sequential shadows.
8. Default blur

IV. METHODS OF FACE RECOGNITION

It is necessary to integrate the material into the real world in order to use AR technology in the visual experimental system, where face recognition is a prerequisite for the visual experimental visual system. During the trial process, it is necessary to find the face of each video frame. However, problems with standing, light, and closure can increase eyebrows and false positives for face detection. Real-time detection is an important indicator of system function to improve the sensory experience of people.

Typical facial recognition procedures include facial recognition, tracking, feature removal, size reduction, and similar visibility.

V. LITERATURE SURVEY

Visual reflection of the mirrors offers some advantages over physical exertion in some cases. In supermarkets, there are hundreds of models available to choose from, and it is difficult and time consuming to choose a model that looks good on a person's face. The virtual experimental system can be used to narrow down the selection to a few designs and sizes effectively and collaboratively, and perhaps to recommendations from an AI-based expert system.

Another problem encountered while trying to physically wear sunglasses is that facial expressions do not look right, as the colour of the lens makes the whole look darker. In addition, it is difficult for visually impaired users to check their appearance with new (non-lens) glasses. They need to take off their real glasses in order to try on new glasses, and that makes the look less attractive. Another problem is that customers cannot compare two or more mirrors at the same time. They should memorize their previous appearance after each attempt. Virtual try-on system can solve all these problems on a large scale. Since users are not actually wearing new glasses, and also do not have to remove real glasses that help to see, if they are wearing them, they can see a clear facial expression with new glasses. In addition, using a visual system, users can have multiple shots with multiple visual mirrors at the same time to compare with each other to get help in choosing a design.

The installation of mirrors on the face can be divided into two common categories. The first step is to redesign the 3D model of the user's face / face using user images and then simulate the 3D visual model in this redesigned 3D model / face model. The advantage of this method is its positive effect. However building a good quality face model is still a challenging task and often requires manual editing to have a realistic model.

The second method is to visualize the visual image model in the sequence of a live face image using augmented reality-related techniques. The sequence of real-world imagery and mirrors looks very realistic, because faces come from real camera images. However, the optimal quality will depend largely on the tracking accuracy and scaling algorithms

VI. GLASSES TRY-ON

In the system, face data is captured by the camera for further processing. First, the user's face is located 30-50 cm right in front of the mobile camera. The face is slightly rotated, without exiting the camera area. In addition, the user should not turn back on the light during the test process because the light affects the camera's camera effect. If the camera does not capture face data, the face shape will be adjusted by command. As the user moves in front of the camera, engine re-sees facial information. A quick try button is clicked to enter a quick try page. The user can place the phone 30-50 cm in front of him. Then, her face appears on the cell phone screen. In the meantime, the system automatically detects the user's facial data to include mirrors. Product details are viewed by sliding glasses or by clicking the details button on the left. In the experimental process, external company open frameworks are used to modify the classroom library according to the required requirements. Section 3 introduces face recognition, alignment, tracking registration and head positioning, and methods for producing a visual model. Combined with these methods, the optical connector is packaged to increase system stability, reducing dependence on external company controls. The embedded section in open frameworks begins with the main function.

VII. OPTIMIZATION

If you experience any performance issues, please start:

1. Check that you are using the latest keyword (/dist/jeelizNNCwidget.js),                                                            
2. Check that your browser has been updated (Chrome is recommended), check that your picture drivers have been updated,                                                                                                                                                             
3. Install the chrome: // gpu in the URL bar and check that there are no major performance alerts, that hardware acceleration is enabled, that your GPU is detected correctly

Performance is flexible. We make several loops for each supply until we reach the maximum number (7). If we can reach this value, the GPU will be 100% operational. The closer we get to this number, the less notice we will see. It is therefore common for a GPU to operate at 100%. But it may irritate some parts of the application because DOM may be slow to update and CSS animation may be difficult.

VIII. MODULE DESCRIPTION

We provide a high-performance experimental solution for head-mounted devices. Works everywhere without include anything:

1. On a website browsed by a desktop or mobile devices
2. In the mobile app.
3. In a display mirror.

Mirrors are displayed over the user’s head in real time, using a video feed from his camera. If the device is too old or if the user refuse to share his or her camera, you may still be able to upload an image. Glass models are stored on a central website called GlassesDB. We’ve built a smooth and advanced workflow to add and quickly adapt new mirror models to this database. Model metadata can be automatically updated with information from external websites such as prices or customer ratings. We can integrate the Virtual glass try-on glass module directly into website or web application.

IX. LIMITATIONS

There are these limitations:                                                                                                                                     

1. The 3D input model needs to be properly configured, adopted and rated                                                           
2. Supported only standard map and colour map drawings,                                                                                   
3. The format needs to be included in the installation file. It is not possible to change, remove or add texture after importing                                                                                                                                                        
4. We use the THREE.js GLTF vendor to analyze the 3D input model. Some important parameters may be lost. You need to restore them by adjusting the slides for the material.

X. 3-D GLASSES POSE

The model should focus on the X axis (red, horizontal). The vertical axis should be the Z axis (blue). Learners should have an X (red) axis, i.e their vertical connection should be 0:

Branches should be aligned. The internal diameter of the mirrors should be 2 units around the world. The brightest part of the eyeballs should be the tangent on the X (red) axis, i.e., Y = 0 and Z = 0:

All visible changes (scale, rotation, location) should be applied to the matches.

Conclusion

This paper proposes a method of experimenting with visual mirrors based on large-scale image measurement, which focuses mainly on the simulation process. The proposed method can ensure accuracy and stereoscopy, equal to a large facial rotation, and does not require sophisticated equipment such as a deep camera. Real-time 3D virtual reality is very much needed in many different programs today in the world of entertainment, commercial enterprises and engineering fields. Our goal was to create a new system for experimenting with 3D visual effects. In this project we have introduced a visual experimental program for using a glass model 3D user reconstructed with a user image. By conveying the actual head movement that does not correspond to the movement of the head model, the user can see certain ideas of how the glasses fit the head which can not be seen directly through the mirror. This program can be used using a computer with a webcam, which is why it is less expensive.

References

[1] YEN, HT, ON G, EP, NI SWAR, A., H UA NG, Z., AND RA-HA RD JA, S. 2009. Automatic 3D Face Synthesis with real time. with 8 International Conference Processes -Virtual Reality Continuum and its Applications in the Industry (VRCAI), 103–106 [2] L. P. MORENCY, A. RAHIMI AND NO-T. DARRELL, ADAPIVE APPREANCE ESED MODEL, IEEE COMPUTER VISION AND PATHTERN RECOGNITION, 2003 [3] https://www.lenskart.com/compare-looks [4] https://www.zennioptical.com/tryon [5] http://www.biyao.com/home/index.html [6] P. M. PALLETT, S. LINK AND K. LEE, New \"gold\" measurements of facial beauty, Vision Research, 50 (2), 149-154, 2010. [7] D. OSCAR, C. MODESTO, L. JAVIER AND A LIUS, Computer Vision Eyewear Selector based, J Zhejiang University Science, 2010 11 (2): 79-91. [8] NIKOS KOMODAKIS, GEORGIOS TZIRITAS, Image Completion Effective Distribution of Belief through Essential Planning and Effective Performance Pruning, IEEE Transactions on Image Processing, 16 (11), 2649 - 2661, 2007. [9] Liu F, Zeng D, Zhao Q, et al. Integrated Face Coordination and 3D Face Reconstruction [C] // European Computer Vision Conference. Springer International Publishing, 2016: 545-560. [10] Huang W Y, Hsieh C H, Yeh J S. Visual vision eye system based on vision [C] // Consumer Electronics (ISCE), 2013 IEEE 17th International Symposium is open. IEEE, 2013: 45-46. [11] Niswar A, Khan I R, Farbiz F. Virtual try-on eyeglass using 3D model of head [C] // Proceedings of the 10th International Conference on the Progress of Real Estate and Its Applications in the Industry. ACM, 2011: 435-438.

Copyright

Copyright © 2022 Piyush Anand, Raj Gupta, Nikhil Kumar Tiwari , Shilpy Gupta. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.