Multicloud Orchestration using Terraform

Abstract

In the past ten years, innovations in the field of cloud computational infrastructure have made a major impact and became an essential part of every organization. Most of the cloud service providers are improvising their service and they are making the user experience better. Even though the providers have provided many services, they are facing the situations like vendor lock-in, unreliability, lower performance and increased costs. A new methodology has been introduced to help overcome these problems being faced. Multi-cloud technology provides the utility of using various cloud functionalities provided by different cloud vendors that gives organizations a great number of options to optimize performance, reduced financial overhead and chose the best cloud technologies available without creating any platform complexities. There are different tools available that are used for cloud provisioning, orchestration and management like AWS CloudFormation, Ansible, Terraform, Cloudify etc. This paper summarizes the analysis of the evolution of multi-cloud strategy and also shows the role of terraform in cloud orchestration with a small comparison with one of the other cloud orchestration tools.

Introduction

I. NTRODUCTION

Cloud computing has gained nearly universal adoption. The way most of the enterprises are adopting the multi-cloud environments is increasing. Many organizations already have their applications running on different cloud platforms. The multi-cloud approach provides with different service providers like IaaS, PaaS or SaaS manage organization’s work overload. Making multiple clouds to be combined and make it work together would be a bit complex task but it will help in financial benefits by reducing the management cost and greater flexibility by allowing to handle multiple operations on different clouds rather than a single cloud.

As per the studies, 89% of organizations stated that they are using a multi-cloud strategy, 80% are using a combination of both private and public clouds which is called as hybrid cloud approach. As per the current scenario the studies also state that till 2023 multi-cloud trends will focus less on the initial adoption of multi-cloud environments instead it will focus on the continued growth, efforts to control costs, security, and matching the right applications to cloud services.

The organizations are adopting the public cloud strategy in such a way that the financial spend on these have rapidly got increased. 37 percent of organizations have stated that they have spent more that 12 dollars in the last year and 80 percent have stated that their yearly spend was more than 1.2 million dollars. The medium and small business organizations run lesser workloads, so their spend on cloud will also me less. But still, 53 percent of medium and small business organizations annual spend has got increased by 1.2 million dollars.

The demand for automating the management and deployment of applications is increased as there has been an rapid increase in the multiple cloud adoption. Due to this increased complexity, the cloud orchestrators which can automate the management of applications on multiple clouds has also been increased.

A.  Need of Cloud Orchestration

The best way to handle the multiple cloud infrastructure is multi-cloud orchestration. In this method, all the work load operations on a cloud could be managed virtually. The tasks like deploying the infrastructure, balancing the load, solving the issues in networking and handling the errors occurred will all come under the process of this management. The cloud orchestrator is used to handle all these operations which will take care of reducing the manual work overloads by automating all the process involved from cloud provisioning till deployment.? The work that was being carried out manually by the IT team members can be reduced by the cloud orchestrator by automating those works which will help in increasing the productivity and making use of the resources for other operations. The errors that were occurring due to manual process will also be reduced. By doing so, it would lead in the faster application deployments and helps the organizations to concentrate on innovating more ideas. This will also make the experience of the customer better.

In order to make the process more simple, choosing the proper tool for orchestration plays a major role. The tool should be able to manage all the tasks that were being carried out by an IT staff and make sure in the proper execution of tasks. Multiple cloud orchestration tools are present in the market which are even open-source which provides a very good community support and proper guide to use the tool as well. Terraform is one of such tools which can act as a cloud orchestrator. It can manage and orchestrate multiple cloud providers, cloud services by using a single configuration file among multiple platforms.

II. LITERATURE SURVEY

In the present day, multi-cloud techniques has to be used by the organizations in order to improve the performance and provide better user experience. Making use of the correct approach to implement the multi-cloud also plays an important role in improving the performance, availability, reducing the redundant data usage. But it can also lead to a problem if the demand in multi-cloud increases more as more organizations are adopting it[1]

Most of the organizations would need multi-cloud or hybrid cloud infrastructure as considering the growth of the organizations whether small or large. Management and monitoring the work continuously is very important in controlling the operations like recovering the errors occurred, cloud bursting. The multicloud would help to handle these operations effectively. Still there would be a problem of setting up the environment for multicloud[2]

Multiclouds are made up by a group of public clouds. Where as the hybrid clouds can be combination of both public and private clouds. The traffic and the operational overloads can be handled and reduced by using these kind of environments. It would also help the organizations by preventing vendor lock-in and provide reliability, flexibility, cost and performance efficiency[3]

As the name multi cloud which refers to making use of different cloud platforms or services, all the organizations should start adopting the multiple cloud strategies and should not be only bounded by the use of single cloud. By making use of multi cloud, the organizations can free up the resources being over consumed in order to handle the works load that was being handled on a single cloud[4]

Using the cloud services from different vendors could be called as multi-cloud. These vendors or the providers could be anything like AWS, Google cloud platform, IBM, Azure which would allow to run any kind of applications on any cloud platform like SaaS, IaaS or PaaS[5]

In the orchestration, the cloud operations will be handled virtually. This management can handle all the operations on cloud like deploying the infrastructure, handling network issues, balancing the work overhead, handling the errors. All these management activities are handled by cloud orchestrator across different cloud platforms. Through this all the service requests will be automated and accessed[6]

Not only the automation or management of clouds, the tools used for management process, the APIs being used for cloud management in an organization are also handled by cloud orchestrator end to end. In order to make the application run properly end to end, the automation process is given with proper resources, protocols in terms of security will be maintained and it will be made sure that the deployment is also done without any issues. This process will be handled by the cloud orchestrator[7]

While providing the services to any multiple cloud application or providers, the organizations would look for minimum errors to occur in their service. Along with their service they will also expect that the spend on the service they are providing to be less and possibility of vendor lock in should not be there. To make all this happen, terraform does the provisioning and management of the infrastructure by using single configuration file and also using versioning for easy save and rollbacks[9]

III. DISCUSSION

Infrastructure provisioning across multiple clouds reduces the faults and helps to overcome from the outages in the vendors. All the provides will be following their own workflows and will be having different interfaces. Though the providers have different workflows, they can be managed by making it follow the single workflow by using terraform. It will help in handling the dependencies with multiple clouds and also manage the large scale orchestration as well.

Terraform is one of the popular tool which is widely known as infrastructure as code tool used for provisioning the cloud infrastructure irrespective of the providers or vendors. It provisions the required resources for any cloud vendor or cloud service which can make use of a single configuration file to follow the same workflow.

Terraform provides different approaches to handle the servers by configuring them and make it merge with the configurations that are being used previously. The diagram below demonstrates how the cloud infrastructure provisioning was before and how the process is after using terraform respectively.

There are number of cloud orchestration tools available from which some are open source and some are managed orchestration services. Some of the powerful orchestration tools can be listed as AWS CloudFormation, IBM Cloud Orchestrator, Ansible, Terraform, Cloudify, Microsoft Azure Automation, Morpheus etc. These tools can be integrable with different cloud providers. Terraform has around 130 providers which it supports officially.

IV.  METHODOLOGY

The infrastructures with only a single-vendor are not extensible to the other vendors or external services. This kind of organizations or teams use the tools and the configurations that will align only for a single vendor. This method would lead to multiple challenges like increased costs, vendor lock-in and result in reduced productivity since it fails to handle multiple workflows. It will also lead to increased risk in order to secure, govern, and audit for the organizations which has multiple workflows.

In order to reduce the risk, optimize the cost and increase the productivity, Terraform proposes with a consistent approach to multi-cloud provisioning. It makes use of the single configuration files to provision different cloud platforms that can be even for multiple vendors which enables the organizations to use a consistent workflow between multiple clouds in terms of provisioning, security, governing and auditing any infrastructure.

Terraform workflow can be divided into two parts. The first part is to allocate the resources that are required for the particular infrastructure and the second step is to apply the configuration to see what and where it has been provisioned. Then all the resources will be allocated based on the other requirements and will be provisioned. It reduces the redundancy of provisioning each vendor separately and uses a single configuration file to provision multiple vendors as shown in the figure.

Conclusion

As per one of the experiments conducted to compare different aspects of performance between cloudify and terraform, it has been proven that terraform turned out to be a better performer than cloudify where cloudify consumed more resources and took more time for the orchestration.Most of the organizations are trying to make use of multi-cloud infrastructure in addition to the private clouds which are existing to improve their performance, reduce the cost at large scale and meet their requirements in an efficient way. It is also found that the implementation of multi-cloud helps the organizations to improve their overall performance and prevent vendor lock-in situations with increased performance. Overall, it has been noticed that terraform is one of the best IaC tool which has many popular features that makes it possible to be used as a cloud orchestrator too. Even though terraform is not fully recognized for cloud orchestration as it’s recognized for infrastructure provisioning, terraform is still able to compete with the other orchestration tools which is able to provide justice by meeting the expected requirements.

References

[1] Coredge Marketing, \"What is multicloud orchestration? why do we need it?\", https://community.nasscom.in/index.php/communities/cloud-computing/what-multicloud-orchestration-why-do-we-need-it, January 28, 2022 [2] D. Sitaram et al., \"Orchestration Based Hybrid or Multi Clouds and Interoperability Standardization,\" 2018 IEEE International Conference on Cloud Computing in Emerging Markets (CCEM), 2018 [3] NS1, “Migrating to the cloud. What to consider”, https://ns1.com/multi-cloud#:~:text=Multi%2Dcloud%20is%20a%20strategy,and%20intelligent %20DNS%20orchestration%20layer ,January 30, 2017. [4] H. A. Imran et al., \"Multi-Cloud: A Comprehensive Review,\" 2020 IEEE 23rd International Multitopic Conference (INMIC), 2020 [5] IBM Cloud Education, “What is Multicloud – IBM”, https://www.ibm.com/ cloud/learn/multicloud ,9 August 2021 [6] Brent Smolinski, “Tips for getting multi-cloud orchestration right”, https://www.accenture.com/us-en/blogs/cloud-computing/tips-for-getting-multi-cloud-orchestration-right2, May 10, 2021 [7] CloudZero.Inc, “What Is Cloud Orchestration? 8 Tools To Get You Started”, https://www.cloudzero.com/blog/cloud orchestration#, 2021 [8] ClouIq, “Terraform for Multi-Cloud Orchestration Strategy”, https://www.cloudiqtech.com/terraform-for-multi-cloud-orchestration-strategy/, 2022 [9] Hashicorp Terraform, “Multi-Cloud Provisioning with HashiCorp Terraform”, https://www.hashicorp.com/resources/enabling-multi-cloud-with-hashicorp-terraform, 2018 [10] Nitheesh Poojary, “Terraform: Orchestrate Multi-Platform Cloud Deployments”, https://cloudacademy.com/blog/terraform-multi-cloud-deployments/, September 25 2015 [11] Hashicorp Terraform, “Unlocking the Cloud Operating Model: Provisioning”, https://www.hashicorp.com/resources/unlocking-the-cloud-operating-model-provisioning? utm_source=terraformcomplianceCTA, Jul 23, 2019 [12] Fraction, “Multi-Cloud Trends: The Latest Trends of 2022”, https://www.factioninc.com/blog/hybrid-multi-cloud/multi-cloud-trends/, January 4 2022 [13] Brian Adler, “Cloud Computing Trends: Flexera 2022 State of the Cloud Report”, https://cloudacademy.com/blog/terraform-multi-cloud-deployments/, September 25 2015 [14] L. R. de Carvalho and A. Patricia Favacho de Araujo, \"Performance Comparison of Terraform and Cloudify as Multicloud Orchestrators,\" 2020 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID), 2020

Copyright

Copyright © 2022 Krishnamoorti Bhat, Prof. Prashanth K. . This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.