ORB-SLAM for Collaborative Augmented Reality

Abstract

Robotic systems and augmented reality applications have greatly benefited from recent developments in Simultaneous Localization and Mapping (SLAM) methods. With an emphasis on ORB-based techniques and their uses, this literature review examines several SLAM technological advancements and approaches. The publications in the survey cover a wide range of subjects, including depth image analysis, dynamic environment mapping, markerless augmented reality, and real-time SLAM implementations. Methods for combining deep learning models, visual inertial fusion, and ORB features with depth sensors are also looked at. By improving accuracy, stability, and efficiency in mapping and localization tasks, these research aid in the improvement of SLAM algorithms. The survey\'s insights provide insightful advice for researchers and practitioners looking to apply cutting-edge SLAM methods in their fields.

Introduction

I. INTRODUCTION

Augmented reality, or AR, has grown in popularity across a number of industries and fields, including gaming, entertainment, and education. Accurate and reliable environment mapping and localization are crucial for AR applications because they enable the seamless overlay of virtual material on the actual world. Simultaneous Localization and Mapping (SLAM) techniques are essential in this setting because they allow devices to perceive and respond to their environment instantly. The well-known ORB-SLAM (Oriented FAST and Rotated BRIEF SLAM) SLAM technique is one that has attracted a lot of interest. Using keypoint detection and description, ORB-SLAM is a feature-based SLAM technique that tracks camera movements while concurrently creating an environment map. It is a well-liked option for augmented reality applications because to its effectiveness, flexibility, and compatibility with multiple sensor modalities. The scope of standard augmented reality is expanded by Collaborative Augmented Reality (CAR), which allows numerous users to interact with the same augmented world at once. The collaborative element poses further difficulties for localization and mapping, since the system needs to accurately and consistently synchronize and integrate data from several devices in real-time. In order to solve the challenges, a number of studies have looked into integrating ORB-SLAM into collaborative AR systems. Through the utilization of ORB-SLAM's advantages, namely its scalability and real-time performance, researchers want to create CAR systems that can offer collaborative and immersive AR experiences in a variety of applications.

In this literature survey, we will examine the state-of-the-art techniques and advancements in ORB-SLAM for Collaborative Augmented Reality. We are going to review the ways in which researchers have expanded and modified ORB-SLAM to facilitate cooperative AR applications, as well as the difficulties they have faced and suggested ways to resolve them. This survey attempts to provide insights into the current state of ORB-SLAM in collaborative AR by integrating existing research and to recommend areas for further research and development.

II. SCOPE AND FOCUS

ORB-SLAM for Collaborative Augmented Reality aims to enhance the user experience by integrating ORB-SLAM, a robust and efficient SLAM algorithm, into collaborative AR applications. This involves developing technologies that enable correct spatial alignment and seamless connection of virtual and real-world elements, allowing numerous users to interact with virtual information in a shared physical environment. The scope includes research and development work aimed at creating cooperative augmented reality systems that use ORB-SLAM for real-time mapping and localization, allowing consistent and coordinated AR experiences across various devices. The major goal is to deal with the difficulties associated with collaborative AR, including preserving users' consistent spatial awareness, managing occlusions, and guaranteeing seamless transitions between virtual and real-world material. In particular, the emphasis is on utilizing ORB-SLAM's capabilities to offer reliable and precise mapping and localization in dynamic, cooperative AR settings. In order to promote smooth collaboration and boost user engagement in augmented reality experiences, this entails enhancing ORB-SLAM for real-time performance, scalability, and reliability in multi-user settings.

Additionally, unique interaction mechanisms and user interfaces are explored. Furthermore, efforts might be focused on investigating uses of collaborative AR outside of entertainment, such training, education, remote help, and collaborative design, demonstrating the potential influence of ORB-SLAM technology in a variety of collaborative AR domains.

III. LITERATURE REVIEW

In [1] 2019 Zhou et al. presented an enhanced SLAM technique that makes use of RGB-D data. The system's performance in localization and mapping tasks is improved with the integration of depth information. The flaws of conventional RGB-only SLAM systems are addressed by this method, especially in low-texture or poorly lit situations. They hope to improve SLAM's accuracy and robustness in a variety of situations by including RGB-D data. Applications like robotics and augmented reality that need accurate 3D mapping and localization may find value in this work.

In [2] Ruan et al. (2024) - "DN-SLAM: A Visual SLAM With ORB Features and NeRF Mapping in Dynamic

Environments" ,presents that combining ORB features with NeRF (Neural Radiance Fields) mapping—a technique created especially to handle dynamic environments—DN-SLAM offers a fresh approach to visual SLAM. This points to a major development in SLAM technology to handle difficult situations when the environment is constantly changing.

In [3] "Real-Time 6-DOF Monocular Visual SLAM based on ORB-SLAM2" Huang et al. (2019) proposes that an ORB-SLAM2-based real-time monocular visual SLAM system with six degrees of freedom (6-DOF) localization and mapping capabilities. This is perhaps an improvement or optimization of the ORB-SLAM2 framework for better accuracy and efficiency in real-time applications. With real-time operation, the suggested system seeks to deliver strong performance in real-world scenarios by utilizing

ORB-SLAM2, a cutting-edge SLAM framework. Applications in robots, autonomous cars, and augmented reality may benefit from this research.

In [4] "ORB-SLAM map initialization improvement using depth" by Fujimoto et al. (2016) focuses on enhancing ORB-SLAM's map initialization procedure through the integration of depth data. In SLAM systems, map initialization is an essential step where using depth data can improve robustness and accuracy, particularly in the early mapping stages. They want to improve the accuracy of pose estimation and feature matching at the initialization stage by including depth data. The state-of-the-art in SLAM initialization techniques is advanced by this research, which benefits a variety of localization and mapping applications.

In [5] "Improvement and Experimental Evaluation Based on ORB-SLAM-VI Algorithm" Wang et al. (2022) present that the ORB-SLAM-VI method has been improved, and the authors give experimental evaluations to support their claims. Performance assessments in a range of situations are probably part of this to show how well the suggested improvements work.

In [6] "Stereo vision SLAM system based on ORB" by Jiao and Wang (2019) introduces a stereo vision SLAM system based on ORB characteristics is presented in this article. Stereo vision, which uses two cameras, naturally offers depth information, which can increase the precision and resilience of SLAM systems. The efficiency and dependability of the system are probably increased when ORB capabilities are integrated with stereo vision.

In [7] Target detection and ORB-SLAM2 are integrated in this research by Wang and Hu to propose an improved technique for AR projection. The term "augmented reality" refers to a technology that projects digital data onto the physical world, usually as seen through AR glasses or a smartphone. The ORB (Oriented FAST and Rotated BRIEF) feature descriptor is used by the visual SLAM (Simultaneous Localization and Mapping) system ORB-SLAM2 to provide reliable localization and mapping. By increasing target detection accuracy and combining it with ORB-SLAM2 for more accurate localization, the authors suggest a way to improve AR projections. The difficulties of precise AR projection are probably covered in the study, along with how integrating target detection with SLAM can help to overcome these difficulties and possibly provide better user experiences in AR applications.

In [8] A technique for semi-dense mapping with a monocular camera and ORB-SLAM is explained by Wang et al. Robotics and autonomous systems depend on SLAM technology to map and navigate their environments in real time. Popular SLAM algorithm ORB-SLAM has become known for its flexibility and efficiency. In comparison to conventional sparse maps, the semi-dense maps produced by the authors are more precise representations of the surroundings since they utilize an extension of ORB-SLAM. This work probably addresses the advantages of semi-dense mapping, including enhanced obstacle recognition and scene understanding, and how ORB-SLAM might be modified to achieve this. This mapping technology breakthrough may have applications in intelligent transportation systems, such drones or autonomous vehicles.

In [9] , Shao offers a monocular SLAM system based on ORB characteristics. Monocular SLAM, or simultaneous localization and mapping with a single camera, is crucial for applications like mobile phones and lightweight robots where only one camera is available. Because of their robustness and efficiency, ORB features are a kind of feature descriptor that are frequently employed in SLAM algorithms.

The design and implementation of a monocular SLAM system that makes use of ORB features for real-time mapping and localization is probably what the study explains. It might also go into the drawbacks and difficulties of monocular SLAM in comparison to systems with several sensors, as well as how ORB features work to overcome these difficulties.

In [10] Wang et al.'s presentation focuses on semi-dense mapping utilizing ORB-SLAM with a monocular camera. It probably goes through how to modify the popular SLAM algorithm ORB-SLAM to produce semi-dense maps, which offer more precise depictions of the surroundings than conventional sparse maps. The benefits of semi-dense mapping for intelligent transportation systems, such better obstacle identification and path planning, might be covered in the study. It might also discuss and suggest ways to get over the technical difficulties in achieving semi-dense mapping using a monocular camera.

In [11] A paper on visual-inertial fusion for indoor autonomous quadrotor navigation using ORB-SLAM is presented by Haddadi and Castelan. Drones, also known as quadrotors, are being used more and more for indoor navigation among other uses. In order to increase navigation accuracy and durability, visual-inertial fusion refers to the combination of visual data from cameras with inertial measurements from sensors like accelerometers and gyroscopes. The fusion system uses ORB-SLAM as its visual component, taking advantage of its real-time SLAM capabilities through the use of visual characteristics. The difficulties quadrotors have when navigating indoors, like GPS signal loss, will probably be covered in the study along with how visual-inertial fusion with ORB-SLAM overcomes these difficulties to provide dependable autonomous navigation in indoor environments.

In [12] The monocular SLAM system based on ORB features presented in Shao's work is probably intended to be used for system implementation and assessment. When a single camera is available for localization and mapping tasks, monocular SLAM becomes crucial. Because of their resistance to changes in viewpoint and lighting and their computational efficiency, ORB characteristics are often used in SLAM algorithms. The use of ORB characteristics in the SLAM system to achieve real-time localization and mapping with a single camera is probably covered in the study. It may also compare the system's performance to that of other SLAM techniques, assessing the system's accuracy, resilience, and computing efficiency. The study may also go over possible uses for the monocular SLAM system in industries including robotics, augmented reality, and autonomous vehicles.

IV. FINDINGS

The twelve papers' findings cover a wide range of developments in ORB-based SLAM (Simultaneous Localization and Mapping) systems. In order to improve localization accuracy and mapping robustness, researchers are increasingly combining more sensor data, such as RGB-D information and depth data. Novel approaches such as DN-SLAM illustrate how SLAM methods may be adjusted to dynamic situations, while real-time monocular SLAM solutions highlight how important effective algorithms are for real-world use. Further optimization efforts are shown by enhancing map initialization and algorithm efficiency. Uses ranging from stereo vision SLAM to augmented reality demonstrate how versatile ORB-based techniques are. The flexibility of navigation is further improved by integration with various sensor modalities, such as inertial sensors, especially in difficult environments. All of these results lead to a dynamic research landscape where the common objective is to achieve improved accuracy, robustness, and usability in real-world scenarios while improving ORB-based SLAM systems for a variety of applications. The twelve papers' literature reviews collectively tell an integrated narrative of the latest advances in ORB-based SLAM (Simultaneous Localization and Mapping) systems for a range of applications. As demonstrated by Zhou et al.'s improved ORB-based SLAM using RGB-D data, researchers have been integrating richer sensor data—such as RGB-D information—to improve localization and mapping accuracy. Furthermore, the creation of DN-SLAM by Ruan et al. represents a significant advancement in managing dynamic environments by combining NeRF mapping methods with ORB features. Huang et al.'s proposal for real-time monocular SLAM systems emphasizes the importance of effective algorithms like ORB-SLAM2 for a range of real-world uses. Furthermore, in order to ensure the reliability and robust performance of SLAM systems, attempts have been made to improve map initialization, such as the depth-based initialization enhancements by Fujimoto et al.As highlighted by Wang et al.'s work, algorithmic advancements and extensive experimental assessments highlight the continuous quest to develop SLAM methods in order to satisfy changing demands. Jiao and Wang's stereo vision SLAM system demonstrates how flexible ORB-based techniques may be in a variety of sensor modalities and circumstances. Combining SLAM techniques with other computer vision tasks, like Wang and Hu's work on improving the accuracy of augmented reality predictions, shows how well they work together. The creation of monocular SLAM systems and improvements in semi-dense mapping by different researchers also help to increase the possibilities of SLAM technology. Lastly, Haddadi and Castelan's investigation of visual-inertial fusion for navigational purposes emphasizes the integration of several sensor modalities to improve navigation robustness, especially in difficult environments.

These findings collectively illustrate a dynamic research landscape focused on advancing ORB-based SLAM systems for diverse applications, with a common goal of achieving higher accuracy, robustness, and applicability in real-world scenarios.

V. ACKNOWLEDGMENT

This literature review would be incomplete without dedicating our gratitude to a few people who have contributed a lot towards the victorious completion for this paper. We would like to thank Thiru L.Gopalakrishnan, Managing Trustee, PSG & Sons Charities, for providing us with the prospect and surroundings that made the work possible. We take this opportunity to express our deep sense of gratitude to Dr T.Kannaian, Secretary of PSG College of Arts & Science, Coimbatore for permitting and doing the needful towards the successful completion of this paper. We express our deep sense of gratitude and sincere thanks to our Principal Dr D.Brindha M.Sc., M.Phil., Ph.D., MA (Yoga)., for her valuable advice and concern for students. We are very thankful to Dr A. Anguraj, M.Sc., M.Phil., Ph.D., Vice Principal (Academics), Dr Jayanthi M M.Com., MBA., M.Phil., Ph.D., Vice Principal (Student Affairs), Dr M. Umarani MBA, M.Phil., Faculty-In-Charge (Student Affairs), for their support towards this paper. We owe our deepest gratitude to Dr. K. V. RUKMANI., M.C.A., M.E., Ph.D., Associate Professor & Head, Department of Software Systems, for her encouragement to pursue new goals and ideas.

I convey my heartiest thankfulness to Lt.Dr.D.Antony Arul Raj, M.Sc.,(CS).,M.Phil., PGDCE., Ph.D., Assistant Professor, Department of Software Systems, for his timely suggestion which had enabled us in completing the paper successfully.

Conclusion

ORB-SLAM for collaborative augmented reality demonstrates its significance in advancing collaborative experiences by providing robust spatial awareness and interaction capabilities. The ORB-SLAM for collaborative augmented reality (AR) highlights the potential of integrating simultaneous localization and mapping (SLAM) techniques with AR systems to create immersive and interactive collaborative environments. Numerous methods and enhancements have been suggested by researchers to take use of ORB-SLAM in order to improve the precision, dependability, and usability of collaborative AR applications. The ability of ORB-SLAM to enable real-time localization and mapping using visual features is one of its main advantages in the context of collaborative AR. This makes it appropriate for dynamic situations where several users interact with virtual objects at the same time. Researchers want to facilitate shared experiences among users in collaborative AR situations by seamlessly integrating virtual content with the real-world environment through the integration of ORB-SLAM with AR frameworks. Based on the twelve articles provided, it\'s evident that there is a significant focus on utilizing ORB (Oriented FAST and Rotated BRIEF) features in SLAM (Simultaneous Localization and Mapping) systems. Overall, the literature review highlights the significance of ORB features in advancing SLAM technology and highlights the ongoing efforts to enhance ORB-based SLAM algorithms for a broad range of practical applications. Future research directions may involve further optimizations, integration with other sensor modalities, and innovation of novel algorithms to address remaining challenges and expand the capabilities of ORB-based SLAM systems.

References

[1] Y. Zhou, B. Li, Y. Li and D. Wang, \"Improved-ORB-based SLAM Using RGB-D Data,\" 2019 Chinese Control And Decision Conference (CCDC), Nanchang, China, 2019, pp. 4944-4949, doi: 10.1109/CCDC.2019.8832804. [2] C. Ruan, Q. Zang, K. Zhang and K. Huang, \"DN-SLAM: A Visual SLAM With ORB Features and NeRF Mapping in Dynamic Environments,\" in IEEE Sensors Journal, vol. 24, no. 4, pp. 5279-5287, 15 Feb.15, 2024, doi: 10.1109/JSEN.2023.3345877. [3] W. Huang, Y. Li and F. Hu, \"Real-Time 6-DOF Monocular Visual SLAM based on ORB-SLAM2,\" 2019 Chinese Control And Decision Conference (CCDC), Nanchang, China, 2019, pp. 2929-2932, doi: 10.1109/CCDC.2019.8833084. [4] S. Fujimoto, Z. Hu, R. Chapuis and R. Aufrère, \"ORB-SLAM map initialization improvement using depth,\" 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, 2016, pp. 261-265, doi: 10.1109/ICIP.2016.7532359. [5] R. Wang, G. Wu and Y. Deng, \"Improvement and Experimental Evaluation Based on ORB-SLAM-VI Algorithm,\" 2022 5th International Conference on Artificial Intelligence and Big Data (ICAIBD), Chengdu, China, 2022, pp. 229-234, doi: 10.1109/ICAIBD55127.2022.9820290. [6] Y. Jiao and Z. Wang, \"Stereo vision SLAM system based on ORB,\" 2019 IEEE 4th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chengdu, China, 2019, pp.418-421, doi: 10.1109/IAEAC47372.2019.8997808. [7] C. Wang and Y. Hu, \"An AR Projection Improvement Strategy via the Integration of Target Detection and ORB-SLAM2,\" 2021 IEEE 3rd International Conference on Circuits and Systems (ICCS), Chengdu, China, 2021, pp. 277-282, doi: 10.1109/ICCS52645.2021.9697207. [8] B. Wang, H. Wang, Y. Yu and L. Zong, \"ORB-SLAM based semi-dense mapping with monocular camera,\" 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), Yokohama, Japan, 2017, pp. 1-5, doi: 10.1109/ITSC.2017.8317640. [9] S. Shao, \"A Monocular SLAM System Based on the ORB Features,\"2023 IEEE 3rd International Conference on Power, Electronics and Computer Applications (ICPECA), Shenyang, China, 2023, pp. 1221-1231, doi: 10.1109/ICPECA56706.2023.10075945. [10] B. Wang, H. Wang, Y. Yu and L. Zong, \"ORB-SLAM based semi-dense mapping with monocular camera,\" 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), Yokohama, Japan, 2017, pp. 1-5, doi: 10.1109/ITSC.2017.8317640. [11] S. J. Haddadi and E. B. Castelan, \"Visual-Inertial Fusion for Indoor Autonomous Navigation of a Quadrotor Using ORB-SLAM,\" 2018 Latin American Robotic Symposium, 2018 Brazilian Symposium on Robotics (SBR) and 2018 Workshop on Robotics in Education (WRE), João Pessoa, Brazil, 2018, pp. 106-111, doi: 10.1109/LARS/SBR/WRE.2018.00028. [12] S. Shao, \"A Monocular SLAM System Based on the ORB Features,\" 2023 IEEE 3rd International Conference on Power, Electronics and Computer Applications (ICPECA), Shenyang, China, 2023, pp. 1221-1231, doi: 10.1109/ICPECA56706.2023.10075945.

Copyright

Copyright © 2024 Lt. Dr. D. Antony Arul Raj, Dr. K. V. Rukmani, Ms. P. R. Kalyani, Ms. B. Shreyaswini, Mr. Shyam Haldurai. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.