Virtual Reality in Manufacturing Planning and Operations

Abstract

Today’s business environment is dominated by unpredictable changes and uncertainty, there is a need for reducing cost and improving quality within the realm of manufacturing. These challenges have forced companies to turn to new and emerging technologies such as virtual reality (VR). Virtual reality is being used in manufacturing planning and operations as a helpful technology to achieve rapid consolidation of information and decision-making through visualization and experience. In this paper, 23 articles relevant to past and current virtual reality’s application to manufacturing were discussed and analyzed. Future virtual reality ventures and applications for manufacturing are discussed.

Introduction

I. INTRODUCTION

Virtual reality (VR) is a popular information technology area that provides an indirect experience by creating a virtual space that interacts with the human sensory systems and overcomes spatial and physical constraints of the real world. VR technology can be categorized as follows: expression technology for stimulating human sensory systems, interaction technology for interfacing reality with VR, authoring technology for developing VR content, and collaboration technology that networks multiple participants within VR. VR technology has various applications, from industry to educational and entertainment. Additionally, many manufacturing companies have applied VR technologies in their line of work.

Within manufacturing, various efforts and research are being conducted to develop smart manufacturing (SM) systems, to minimize human intervention and allow systems to control the sites intelligently. The introduction of VR has had a predominant effect on manufacturing in terms of development time and creating a safe efficient environment for both people and machines.

A. Manufacturing System

When examining the whole manufacturing system, it can be divided into several hierarchical levels. Production is a networked process where more production units are included to facilitate proper investigation of a manufacturing system. Six production units are identified and organized in a top down fashion. A direct link of simulations to a virtual environment enables user interaction and during simulation processes and allows higher occurrence for exploring complex interactions within objects, users, and operations. Hence, virtual reality is an enabling platform for tools within production planning, product design or technical specifications. There are sub areas in software applications for the field of manufacturing system: factory-level prototyping, virtual assembly environments and virtual prototyping of lower-level activities. At the factory-level VR is used primarily to support modification and simulation of existing shop floors, as well as, improvement in design layouts. Various issues are looked into such as the foundation of the layout to ensure it is well designed to prevent idle time and bottlenecks in a manufacturing system. Additionally, improvement of parts flow through the factory can be address. While on the workplace level, VR is used to analyze single cells or assembly process within a workstation.  A VR framework is divided into several phases, as follows:

Figure 1: Manufacturing System

Modelling, application and adaption; this framework facilitates the digital enterprise tools for manufacturing system design and enables an integrated use of them within a virtual environment. The modeling phase, the data basis, usually contains geometric modeling and then creates a VR model. Within the geometric model, various objects in a manufacturing system are included, such as people, machines, parts, materials, etc. Furthermore, different VR platforms are visualized and manipulated to create the virtual environment where the applications are implemented. The application phase includes two core components: simulation and visualization. Based on the modeled virtual world, simulation rebuilds the manufacturing processes and provides essential information for visualization and VR. Users have a more realistic perception in a virtual environment. Based on the analysis results from applications, the manufacturing system will be adapted. A VR supported Continuous Improvement Process (CIP) Workshop is used to implement such adaption. A CIP workshop is performed in the following order: detection of problems, analyzing problems, developing improvement measures, realizing the measures, and evaluation of results.

B. Virtual Reality Technologies

Virtual reality (VR) is a scientific method and technology created in a virtual space. The founder of VPL Co., Jaron Lanier, presented the phrase “Virtual Reality”, which has been accepted by academia and has become the title of this scientific technology field. [8] Virtual reality (VR) is a popular information technology (IT). VR provides an indirect experience through a virtual space, which interacts with the human sensory systems and overcomes spatial and physical constraints of the real world. [6] VR systems have come a long way since the 1960s when it was first proposed as a “system that can display information to all senses of the user with an equal or bigger resolution than the one that can be achieved in a natural way so that the user cannot say that the artificial world is not real.” [7]

C. VR Classification

VR systems have been categorized in various forms. One of which, the ETRI report classified VR systems into four categories: expression technology, interaction technology, authoring technology, and collaboration technology.  [6]

1. Expression technology: This category of VR technology includes technology related to human sensory systems, including visual, auditory, haptic, olfactory, and taste. This technology has been able to create VR with eighty percent resemblance.
2.  Interaction technology: This refers to technology used to interface humans and computers by using motion, symbols and bio-signals.
3. Authoring technology. This refers to technology, which generates VR content and its supporting database.
4. Collaboration technology. This refers to networking of multiple participants in a VR space.

VR systems have been classified as hardware based or computer based. A hardware-based VR system relies on specific VR hardware such as head mounted display (HMD), VR glove, optical tracking sensor, etc. A computer-based VR system uses software on personal computers and uses PC peripherals as inputs and output tools.

II. LITERATURE REVIEW

In recent times, VR has had a strong effect on manufacturing in terms of development time and creating a safe efficient environment for both people and machines. In “An approach for exoskeleton integration in manufacturing lines using Virtual Reality techniques”, a VR based decision tool for the exoskeleton integration in industrial lines which will assist in the identification of the optimal areas and tasks for application based on simulation results and effectively training workers into the correct use of different exoskeletons. [12] A virtual factory is a system that predicts, improves and controls problems with overall production tasks by association to the sites within the virtual space. Industry 4.0 and “Factory of the Future” is a new trend in automating and data exchange in manufacturing industries. Research to develop more practical use of exoskeleton in industrial industries is being carried out and much research on specific tools for the integration. Operator 4.0 is an extension from Industry 4.0, which considers technology augmented workers. Wearable exoskeletons have been recently introduced as an alternative workplace intervention. An exoskeleton is a wearable, external mechanical structure that is designed to work in sync with the human to provide support or enhance their capabilities. The study aids in integration of exoskeletons in the manufacturing industry through Virtual Reality-based Exoskeleton Integration Framework (VREIF). The combination of virtual reality with exoskeleton technologies facilitates field of application to different use cases.  

Various studies which focused on VR applications in production systems and operations were analyzed. In “Improving Manufacturing Process Change by 3D Visualization Support: A Pilot Study on Truck Production” a VR tool in supporting the factory layout plan decision making process is explored.

 [9] Through point cloud data of the existing factory environment and CAD models of the potential equipment to be installed. Various layout solutions were presented through the desktop VR tool and analyzed by stakeholders in different function groups. 3D laser scanning was used to capture spatial data of the factory environment. The captured point cloud data was processed and formatted for later integration in creating the desktop VR tool. 3D CAD models were gathered and imported to the VR tool. A graphic interface is developed to create alternative layouts for further evaluation. Semi-structured one-to-one interviews were conducted with five stakeholders from different functional groups in the company. Each interview was carried out at the stakeholder’s office and lasted around 45 minutes. The interview consists of three parts. It begins with questions regarding the interviewee’s role and responsibilities in the current manufacturing process change as well as their comments and reflections about the current practice. A demonstration of the collaborative tool and work process is followed, which presents the 3D visualization of the three layout designs for assessment. Finally, it is the discussion concerning how the desktop VR tool and work process could help improve the current work practice and what are the limitations.

In “A novel VR tool for collaborative planning of manufacturing process change using point cloud data”, proposed a working procedure for the development of virtual factory consisted with point cloud data and 3D CAD models. [10] The virtual factory then can be used to test out alternative layouts by all the stakeholders. The immersive HMD and trackable controllers are used to provide near-to-life visualization and interaction. The proposed approach was implemented and tested by the stakeholders. Feedback were gathered through questionnaire and interview and later analyzed. A truck manufacturing plant in United States was selected to demonstrate the immersive VR for the planning of manufacturing process change. The proposed working procedure starts with the data preparation for the virtual factory. 3D laser scanner captures the point cloud data set of the real factory, which provide the most important 3D contextual data of the virtual model. Another data source is the CAD models of any machines that are intended to be implemented during the layout change. The data can be further enhanced with the connection to the existing ICT systems, so that the virtual model not only shows the realistic factory environment, but also has the possibility to augment additional machine specific information in the virtual model for the better support of the decision-making in new layout design. Depending on the specific requirement of each layout planning scenario, the above data can be prepared and feed into the Unity development tool for synchronization, so that the virtual tool is ready for later layout creation and evaluation. After the virtual factory is prepared and deployed to the server, all the stakeholders can access the virtual factory anywhere in the world provided that an immersive VR HMD is available. In the virtual factory, each stakeholder is provided with the information they need to create new layouts or assess and leave comment to existing layouts based on their own expertise. Thus, different layout proposals and feedback are gathered in the system for the synchronization which will either reach the idea solution for implementation or repeat the same process until the idea solution is reached.

In “Development of Virtual Reality Support to Factory Layout Planning”, aims at developing a general guidance for employing point cloud-based VR to support FLP decision-making. [11] Through three industrial cases which incorporated point cloud data into modelling virtual factory for FLP, the general guidance is extracted and refined. Three case studies that adopted the point cloud based virtual factory modelling approach were conducted. They are all industrial cases where layout changes were needed in existing production sites either for improving productivity or adaption of new products. The cases vary in the areas of manufacturing and the scale of the layout change. Desktop VR was implemented in the first case and immersive VR system was chosen for the latter two cases as the technology has becoming ever mature with much lower price and improved performance. With the 110- degree FOV and 90 frame per seconds (FPS) image rendering, it enables nature visualization that users feel as if they were present in the real environment and intuitive interactions within the virtual model. The cases and implemented functions are summarized in the Table below.

The study categorized different forms of data that can be used in the virtual factory modelling process as well as their characteristics and requirements. The merits and limitations of desktop and immersive VR systems are discussed in relation to how well they can support FLP. As the final step in answering how CR can support FLP task, the extracted guidance provided a systematic methodology to employing the state-of-the-art VR technology in supporting FLP.

A. Planning and Operations Application in VR

The desktop VR approach to support FLP decision-making is one addition to the state-of-the-art VR applications in production area. The idea of incorporating point cloud data of existing environment into the modelling of virtual factory and presenting the virtual factory through desktop VR systems to evaluate alternative layout options, is the first step towards how VR can support FLP tasks. “Improving Manufacturing Process Change by 3D Visualization Support: A Pilot Study on Truck Production” demonstrated the need and benefits of having realistic 3D contextual data in the process of decision-making on alternative layouts, it also pointed out direction for later studies to how VR can support FLP task.

The working procedure for point cloud-based VR approach proposed in “A novel VR tool for collaborative planning of manufacturing process change using point cloud data” was exemplified and refined with three industrial cases. It categorized different forms of data that can be used in the virtual factory modelling process as well as their characteristics and requirements. The merits and limitations of desktop and immersive VR systems are discussed in relation to how well they can support FLP. The extracted guidance provided a systematic methodology to employing the state-of-the-art VR technology in supporting FLP.

Companies can achieve and be benefited by minimizing product life cycle, cost and meeting delivery schedule. This paper also presents simplified models that can be modified and used by different companies based on the companies’ objective and requirements

B. Challenges

Challenges in the adoption of virtualization in manufacturing processes will vary. In some cases, the lack of a comprehensive approach towards a collaborative factory planning to take full advantage of immersive systems will be the main roadblock. In addition, manufacturers will require significant capital investments in retrofitting existing facilities, training, and retaining resources.

Even though hardware is becoming more available, “VR is still not a turnkey system” [17] These systems still require multiple roles to operate (See Exhibit 2). In addition, there are many steps involved to model a reality. In their extensive study, Berg, L and Vance, J. noted the complexity in the execution, with manufacturers often following different implementation techniques, further exacerbating the challenge they face. [17] (See Exhibit 3).

While VR has a sizable footprint in manufacturing processes, augmented reality (AR) will change the landscape of virtualization even further. The challenge with AR will be more around the hardware and user experience [15]. Headsets, for now, suffer from a “lack of effective ways to interact with holographic images”, not to mention ongoing issues with “optical field of view, contrast, size, weight, [and] batter life” [15].

C. Future Direction

VR could substantially accelerate time to market while cutting costs and increasing efficiencies throughout a manufacturing process. Specifically, virtual prototypes can identify defects, simulate testing, and lead to faster decision-making. [14]  In the assembly phase, virtual assembly (VA) can lead to enhanced work workspaces, better safety measures, and better oversight over part integration and interaction. [15] Manufacturers are investing heavily in VR technology and adoption. For example, Jaguar Land Rover has established a world-class Virtual Innovation Center in the UK. These centers are allowing, for example, for human factor testing of interactive components of systems. CAD software are also being increasingly used to create digital twins for near-real renderings, helping in the “preparation of environmental impact reports”.

In the consumer-facing domain, innovation in marketing is evident in use of virtual models/products that stand out from a physical brochure; wearing specialized headsets, a customer could flip through a brochure and interact virtually with a product, for example, from the convenience of their home.

In the AR domain, the technology is “poised for significant breakthroughs in the coming years” 15]. In one example, the Radius Innovation and Development center at Jabil, a leading manufacturing technology company, have seen very promising examples where AR is transforming collaboration and curtailing time to market. In one instance, a part had 19 design iterations in the same time it would have taken one iteration in traditional methods.

The [future] benefits of AR can be summarized as follows 15]:

1. Bringing products to life, leading to rapid prototyping and shorted design and development cycles.
2. In-context visualization allows for experimenting within real settings.
3. Long-distance collaboration allows for tapping into centers of excellence around the globe.
4. With sensor technology, spatial sound, and advanced optics, AR takes objects off the screen, allowing more sensory interactions, leading to more effective processes.

Paper interviews were conducted with 11 engineers and employees of allied disciplines from an automotive manufacturer about their current physical and virtual properties and processes. The results guided a review of research findings and scientific advances from the academic literature, which formed the basis of recommendations for future developments of VR technologies and applications. These include: develop a greater range of virtual contexts; use multi-sensory simulation; address perceived differences between virtual and real cars; improve motion capture capabilities; implement networked 3D technology; and use VR for market research.

Conclusion

Until the widespread of AR-based games, there was not a clear validation of how AR can be mainstream. With the same devices, smartphones, available to businesses and consumers alike, AR has been gaining momentum. In a recent study by Jabil, 69% of surveyed businesses see AR becoming mainstream in their processes [16] (See Exhibit 1). Although headsets such as Meta, ODG, HoloLens, coupled with smartphones, are enabling manufacturers to adopt some level of VR in their respective industries, constructing a VR facility is a very cumbersome task: first, it requires a significant capital investment and the need to garner by-in from internal stakeholders; second, technology adoption and standardization is often hard without executive sponsorship and focus on change management [17]. This could explain why 51% of surveyed manufacturers by Jabil have not developed an initial plan for VR adoption. As such, manufacturers may want to consider partnering with early leaders so they could focus on their core capabilities and delivering value. Lastly, with VR becoming more affordable for consumers, with powerful software behind it, manufacturing businesses will benefit from diminishing costs, availability of VR-aware talent, and consumer-driven advancements.[17]

References

[1] Belay, AM (2009) Design for Manufacturability & Concurrent Engineering for Product Development. W.-K. Chen, Linear Networks and Systems (Book style). Belmont, CA: Wadsworth, 1993, pp. 123–135. [2] Doshi, A., Smith, •.T., Thomas, •.H., & Bouras, •. (2016). Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing. [3] Gusikhin, O., Rychtyckyj, N., & Filev, D. (2007). Intelligent systems in the automotive industry: applications and trends. Knowledge and Information Systems, 12(2), 147-168. [4] Menck, N, Yang, X, Weidig, C. (2012) Collaborative factory planning in virtual reality. Procedia CIRP 3: 317–322. [5] Quevedo, W. X., Sánchez, J. S., Arteaga, O., Álvarez, M., Zambrano, V. D., Sánchez, C. R., & Andaluz, V. H. (2017, June). Virtual reality system for training in automotive mechanics. In International Conference on Augmented Reality, Virtual Reality and Computer Graphics (pp. 185-198). Springer, Cham. [6] Electronics and Telecommunications Research Institute (ETRI) (2001) Virtual Reality Technology/Market Report. Daejon, 30 December pp. 12–29. [7] Che, W. (2017). Virtual Reality. In International Encyclopedia of Geography: People, the Earth, Environment and Technology (eds D. Richardson, N. Castree, M. F. Goodchild, A. Kobayashi, W. Liu and R. A. Marston). [8] Wu, Y., Liu, X., Zhao, H. and Wang, M., Research on Locating Video Format and Its Retrieving Method. ed. Information Science and Engineering (ICISE), 2009 1st International Conference on, 26-28 Dec. 2009 2009, 2153-2156. [9] Gong, L., Berglund, J., Wang, Z., Larborn, J., Skoogh, A., & Johansson, B. (2016). Improving manufacturing process change by 3D visualization support: a pilot study on truck production. Procedia CIRP, 57, 298-302. [10] Gong, L., Berglund, J., Saluäär, D., & Johansson, B. (2017). A novel VR tool for collaborative planning of manufacturing process change using point cloud data. Procedia CIRP, 63, 336-341. [11] Gong, L., Berglund, J., Fast-Berglund, Å., Johansson, B., Wang, Z., & Börjesson, T. (2019). Development of virtual reality support to factory layout planning. International Journal on Interactive Design and Manufacturing (IJIDeM), 1-11. [12] Karvouniari, A., Michalos, G., Dimitropoulos, N., & Makris, S. (2018). An approach for exoskeleton integration in manufacturing lines using Virtual Reality techniques. Procedia CIRP, 78, 103-108. [13] Romero, D., Bernus, P., Noran, O., Stahre, J., Fast-Berglund, Å., 2016. The Operator 4.0: human cyber-physical systems & adaptive automation towards human-automation symbiosis work systems, In IFIP International Conference on Advances in Production Management Systems, Springer, Cham, pp. 677-686. [14] Lawson, G., Salanitri, D., & Waterfield, B. (2016). Future directions for the development of virtual reality within an automotive manufacturer. Applied ergonomics, 53, 323-330. [15] Holtrof, J, “Five Benefits of Using Augmented Reality in Product Design”. Retrieved 4/1/2019 [16] https://www.jabil.com/insights/blog-main/augmented-reality-in-product-design.html [17] (2019) Ahranov, A. What is the Future of Augmented and Virtual Reality? Retrieved 3/29/2019 [18] https://www.jabil.com/insights/blog-main/future-of-augmented-and-virtual-reality-technology.html [19] Berg, L. P., & Vance, J. M. (2017). Industry use of virtual reality in product design and manufacturing: a survey. Virtual reality, 21(1), 1-17. [20] W.B. Lee, C.F. Cheung, J.G. Li, Applications of virtual manufacturing in materials processing, J. Mater. Process. Technol. 113 (2001) 416–423. [21] Matthew W. Rohrer, Seeing is believing: The Importance of Visualization in Manufacturing Simulation, Winter Simulation Conference,2000. [22] Nicholas M. Putman, Francisco Maturana, Kira Barton, Dawn M. Tilbury. (2017) Virtual fusion: a hybrid environment for improved commissioning in manufacturing systems. International Journal of Production Research 55:21, pages 6254-6265. [23] Christopher J. Turner, Windo Hutabarat, John Oyekan, Ashutosh Tiwari, \"Discrete Event Simulation and Virtual Reality Use in Industry: New Opportunities and Future Trends\", Human-Machine Systems IEEE Transactions on, vol. 46, no. 6, pp. 882-894, 2016.

Copyright

Copyright © 2025 Emad Tanbour, M. C. Greenfield, Hawra Alzuad. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.