Virtual Reality (VR) and Augmented Reality (AR) Based 360 Degree Panoramic Environment Generator for Virtual Tour

Abstract

We are entering an era of digital wonders and some of the most fascinating fields to have emerged from the technological revolution are virtual-reality and other related technologies, which are playing an increasingly important role in connecting the whole world like never before. The proposed platform will enable the rapid origination of virtual interactive environments using panoramic photos, videos with 360-degree orientation and audio along with the capability to integrate live audio and video in the virtual environment. A prototype for this platform has been designed by using the open-source Marzipano tool that uses the Apache 2.0 license and is hosted on an Amazon Elastic Compute Cloud-server. To create a virtual model, environment suitable panoramic images with 360-degree orientation are supplied and then interconnected together to form the virtual experience. In the present day and age, the need for such a platform has become increasingly vital as such a platform can be implemented to avoid social interactions as and when necessary, viz., in a situation where physically visiting a place is not possible due to the enactment of lockdown measures. The platform can also be used to very rapidly design and implement virtual-tours and campaigns. This brings with it the promise to ensure that people don’t feel isolated in case a physical visit to a location is not possible due to unforeseen circumstances such as pandemics and it can also be used by organizations and businesses for product marketing purposes by quickly creating interactive virtual environments showcasing their products to their prospective clients or by creating interactive virtual models which will enable their clients to gain an idea about their services in a manner similar to that of kinaesthetic-learning.

Introduction

I. INTROUCTION

The 21st century has witnessed the advent of some wondrous technologies such as the social media platforms, autonomous aerial, land and water vehicles and also the smart phone devices; however the emergence of virtual reality and other related technologies is bound to take the technological revolution to new heights. Virtual reality is the mimicking of the real world using digital and electronic means, although experiences may be created that are vastly different from that of the real world, while in case of augmented reality technology, we have the integration of the real-world experience with elements generated by a computer and in VR photography and videos the user experiences panoramic photographs or videos with 360-degree orientation using VR technology. These technologies have the potential to bring great benefits to the general populace due to the fact that real world or other experiences and environments can be showcased through these technologies, which also holds the promise to enrich and enhance the online and offline user experience. Due to this, such technologies find applications in several fields such as medicine and medical research, scientific research as well as sectors such as the entertainment industry, education, business etc.These things find a unique application in a platform which can be used to create interactive VR environments using panoramic photographs and videos with 360-degree orientation and audio recordings which can be embedded in selected regions of the virtual environment and it would also be possible to restrict the orientation of the panoramic photographs and videos to some specific angle for a specific region of the virtual environment depending upon the discretion of the creator. The use of panoramic photos, videos and audio recordings in modelling the virtual environment would ensure the rapid creation of such environments which would increase the productivity as well as make it possible for any user to create such virtual environments without the need for any special technical training. While creating the virtual environment using the platform, the photos and videos can be interlinked in such a manner that the experience should be able to make a smooth transition from one virtual setting to another just by clicking on some nodal point or hovering the mouse over some specific region or point in the virtual space, which can also be highlighted for more intuitive experience.

Finally, the platform would also allow the integration of live videos which can be restricted to some bordered location of the virtual space or live audio which can be embedded in some region of the virtual environment and thus this platform would enable the creation of virtual realms with several regions for the experience to explore at his or her discretion. An immediate application of the platform would be the creation of virtual campaigns to promote environmental awareness among the populace and also the creation of virtual environments for the purpose of virtual tourism, which would be of great benefit to the aged and handicapped people, as it would enable them to virtually tour a tourist destination from the comfort and safety of their homes. Furthermore, the platform would also enable rapid creation of virtual tours for institutions such as schools, colleges and also business organizations.

II. OUR PROPOSED WORK

A. Softwares and Services Used

1. Marzipano Tool for VR 360 Panoramic Image Platform (open source - Apache 2.0 license).
2. Relevant web technologies for designing a web page portal to access the platform and its different modules.
3. Relevant technologies/ methods for taking and/or collecting 360 panoramic images for the platform.
4. Hosting using physical server or cloud-based server (AWS/Google Cloud etc).
5. Designing smart phone applications for mobile access of the platform using relevant technologies.
6. The platform is hosted on a virtual server in the cloud and can be accessed using the Internet.  For this Elastic Compute Cloud of Amazon Web Services is used as it one of the leading cloud service providers.
7. To make the server accessible from the Internet its IP address must be linked to a domain name using a Domain Name System provider. For this Amazon Route 53 is used.
8. For rendering the panoramic image into a virtual-environment module project the Marzipano tool is used. The scenes inside the virtual reality project thus can also be interlinked with one another to create a transitory experience.

B. General Idea

This platform will help the user to create virtual environments which can be thought of as virtual realms with different regions, which can be modelled using the tools available in the platform to showcase different themes and scenarios. Users interacting with different modules created using the platform will get an immersive experience due to the use of virtual reality and it will also be possible to connect the elements within the virtual world of the platform to the real world using augmented reality. For example, some element in the virtual realm created using the platform can be showcased over a real-world marker using augmented reality technology. The platform uses panoramic images and videos with 360-degree space orientation to create the virtual environment. Although it will be possible to restrict the orientation to some specific angle. Furthermore, it will be possible to integrate live video and audio feedback into some selected bordered regions of the platform. Furthermore, it will be possible to interlink the scenes in the virtual environment created using the platform so that the user can easily transition from one viewpoint to the other.

C. User Platform Interaction Process of our Proposed work

Administrator Users who intend to create the virtual modules will use the platform package to integrate selected panoramic images and/or videos with 36- degree orientation to create a project file. The project file which contains the virtual environment can then be further customized using the tools provided by the platform to add details to the virtual environment for example including live audio and video in some bordered region of the virtual space. After this the modules are integrated into the cloud virtual servers and the links of the virtual servers are integrated into the platform portal website.

Any user who wants to interact with the modules will have to follow the below mentioned steps:

1. Step-1: The user starts by visiting platform portal continuing the module links
2. Step-2: Clicking on a module link loads the module
3. Step-3: After the module loads the user can look around the initial scene of the virtual environment
4. Step-4: The user can shift from scene to another using the marked nodal points in the VR module space
5. Step-5: The nodal also comes with info tags describing the module location/scene
6. Step-6: The user my watch the audio or video associated with a bordered region of the VR space
7. Step-7: The user can also save a scene and can jump scenes by scrolling and viewing the scene navigator in scrolling navigator
8. Step-8: After interacting with the VR of a particular module the user can either log out of the platform or return to the home page of the portal to continue browsing the modules.

D. Platform Configuration Steps

A prototype of the platform is designed using the Marzipano tool and the steps to configure the tool to integrate it in to a virtual cloud server are mentioned below :

1. On local machine the Marzipano tool is used to generate the VR environment package from collected panoramic images
2. On the AWS platform the EC2 server is configured and launched :

a. Ubuntu 18.04 LTS is selected as the base OS

b. Appropriate storage space is selected.

3. The EC2 instance is accessed from the local machine and the EC2 instance running Ubuntu 18.04 server is configured to be an Apache HTTP server:

sudo apt install apache2

4. Next a new directory is created

sudo mkdir -p /var/www/websitename.com

5. The VR environment package generated in the local machine is extracted and the appropriate contents are copied in to the directory created in previous step in the Ubuntu 18.04 running on the EC2 instance

        cd /var/www/test.gnitdigiconclave.com

6. Now do cd /etc/apache2/sites-available/ in Ubuntu 18.04 running on the Ec2 instance and then do

sudo nano websitename.com.conf

7. enter the following code block :

<VirtualHost *:80>

    ServerAdmin carla@localhost

    DocumentRoot /var/www/test.gnitdigiconclave.com

    ServerName test.gnitdigiconclave.com

    ErrorLog ${APACHE_LOG_DIR}/error.log

    CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>

8. Then the enable the platform using the following commands :

sudo a2ensite test.gnitdigiconclave.com.conf

        service apache2 reload

9. Then link the EC2 IP address with the website name in a DNS service provider

III.  OUTPUT AND RESULT ANALOGY

Conclusion

The speed at which technologies such as Virtual Reality, Augmented Reality and 360 Degree Panoramic Image and Video are being developed, and the already existing versions being updated, it can be concluded that, in the near future, these technologies may find more use in the domain of medical research- for finding cures for diseases, by studying the underlying genetic structure through models created with the help of these technologies; observing some real world events captured through devices, drones for example, specifically designed to provide data for the VR, AR and other related environments, to get a better understanding of the ground situation etc.However, it is also true that there are several other challenges that are needed to be overcome for that to happen. Some of them are a.Simulation sickness, b.Health effects of VR, AR and other related technologies, c. High cost for developing VR, AR equipment and d. Lack of content. Thus, if these challenges are overcome, technologies such as VR, AR and 360 Degree Panoramic Image and Video can become hugely popular. In future, we would make further modifications on this project by Introducing VR Goggles with human gestures and Raising awareness about VR and other related technologies. Finding suitable use cases to showcase the technology using the platform viz. Environmental Awareness Campaign, Virtual College Tour, Virtual tourism for handicapped and aged people.

References

[1] Towards a contextualized access to the cultural heritage world using 360 Panoramic Images - Pietro Mazzoleni, Stefano Valtolina, Stefano Franzoni, Piero Mussio, Elisa Bertino [SEKE, 416-419, 2006] [2] Application of Panoramic Image Technology in Distance Teaching System. - Yanqun He [International Journal of Emerging Technologies in Learning 10 (6), 2015] [3] Understanding virtual reality - William R Sherman, Alan B Craig [San Francisco, CA: Morgan Kauffman, 2003] [4] Virtual reality-history, applications, technology and future - Tomasz Mazuryk, Michael Gervautz [5] Augmented reality technologies, systems and applications - Julie Carmigniani, BorkoFurht, Marco Anisetti, Paolo Ceravolo, Ernesto Damiani, Misa Ivkovic [Multimedia tools and applications 51 (1), 341-377, 2011] [6] Augmented reality: an overview - Julie Carmigniani, BorkoFurht [Handbook of augmented reality, 3-46, 2011] [7] Improving the skills and knowledge of future designers in the field of ecodesign using virtual reality technologies - Damian Grajewski, Jacek Diakun, Rados?awWichniarek, EwaDostatni, Pawe?Bu?, Filip Górski, Anna Karwasz [Procedia Computer Science 75, 348-358, 2015] [8] Using product popularity to stimulate choice for light products in supermarkets: An examination in virtual reality - Robert PG Goedegebure, Erica van Herpen, Hans CM van Trijp [Food Quality and Preference 79, 103786, 2020]

Copyright

Copyright © 2022 Sourish Mitra, Subhaditya Banerji, Reshmi Das, Sayan Sarkar, Pallabi Das, Bidyutmala Saha, Rafiqul Islam, Rupak Chakraborty. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.