Explore how Pulumi offers a solution to Terraform's dynamic provider config challenge, enabling multi-region and multi-account infrastructure provisioning.
If you have worked with Terraform for a while, you most likely have tried, and failed, to implement a dynamic provider configuration. The first time I remember hitting this roadblock was many years ago when I was attempting to create Kubernetes clusters in Terraform and then apply Helm charts to the new clusters during a single Terraform apply.
Another common use case for dynamic provider configurations is to provision multi-region AWS resources or to create multiple AWS accounts and provision identical resources in those accounts in a single terraform apply.
If you have tried something similar, you quickly learned that the provider configurations for Terraform must be defined before running Terraform apply, and a single resource must be linked to a particular provider. This is due to the architecture of several subsystems in Terraform, which would require breaking changes to support dynamic provider configurations.
To understand why this is difficult to solve in Terraform requires understanding the difference between a provider and a provider configuration. A provider is the plugin that enables Terraform to interface with a 3rd party, such as the AWS Cloud Control API. A provider configuration, on the other hand, refers to the settings for that provider, e.g. the region or the IAM role to assume.
In Terraform, provider configurations are implicitly linked to the provider. In other words, Terraform can infer which provider you want to use (hashicorp/aws) based on your provider configuration. Additionally, if you look at the state file for a resource, you will see that each resource is tied to exactly one provider configuration:
"resources": [
{
"mode": "managed",
"type": "aws_s3_bucket",
"name": "my_bucket",
"provider": "provider[\"registry.terraform.io/hashicorp/aws\"].east",
"instances": [...]
Notice how the provider configuration (in this case specifying the aws.east alias) is associated with the resource, and instances of the resource are an array at the same level. This illustrates that provider configurations are defined at a per-resource scope instead of at a per-resource-instance scope.
Therefore, to solve the issue architecturally, Terraform would need to separate providers from provider configurations. This is one reason why providers must be statically configured. See this explanation for more details.
Before moving forward, I’d like to clarify the difference between runtime provider configuration and dynamic provider configuration. Terraform does indeed support runtime provider configuration, meaning that theoretically it should be possible to create an IAM Role in Terraform and define a provider that assumes the role in a single terraform apply. I say “theoretically” because as of March 2024 there is an open issue that prevents exactly this use case due to the eventually consistent nature of IAM Role creation.
Dynamic provider configuration, on the other hand, refers to using the for_each or similar meta-arguments to create multiple resource instances in a loop with different provider configurations.
Let’s attempt dynamic provider configurations in Terraform. The use case is that we have defined two regions, and we want to loop through the regions and create a bucket in each one.
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
}
}
}
variable "bucket_name" {
default = "tf-test-bucket-23948"
}
provider "aws" {
alias = "east"
region = "us-east-1"
}
provider "aws" {
alias = "west"
region = "us-west-2"
}
resource "aws_s3_bucket" "my_bucket" {
// Invalid Syntax
for_each = toset([aws.east, aws.west])
provider = each.value
bucket = "${var.bucket_name}-${each.value}"
}
We quickly learn that Terraform does not support using for_each to set provider configurations, and we get back a somewhat confusing error message:
- provider registry.terraform.io/hashicorp/each: required by this configuration but no version is selected
One significant feature that sets Pulumi apart from Terraform is that Pulumi has created their own deployment engine. AWS CDK and Terraform CDK are similar to Pulumi in that you can use a modern programming language to define your infrastructure. However, AWS CDK is converted to CloudFormation before execution, and Terraform CDK synthesizes code into JSON files that are passed as configuration to Terraform. Pulumi, on the other hand, does not have this “synthesis” step. The code is passed from the Language Host directly to the Pulumi Deployment Engine which uses providers and state to reconcile the infrastructure.
The Pulumi Deployment Engine is one of the strongest differentiators of Pulumi. The choice to develop their own deployment engine is what enables asynchronous infrastructure provisioning, and allows us to solve our dynamic provider configuration problem.
Let’s try it out:
import * as aws from "@pulumi/aws";
const providers = [
{
name: "east",
provider: new aws.Provider("east", {
region: "us-east-1",
}),
},
{
name: "west",
provider: new aws.Provider("west", {
region: "us-west-2",
}),
},
];
providers.forEach((providerConfig, index) => {
new aws.s3.Bucket(
`myBucket-${providerConfig.name}`,
{
bucket: `provider-bucket-${providerConfig.name}-${index}`,
},
{ provider: providerConfig.provider }
);
});
Pulumi has no issues executing the above code. Both buckets are created in a single “pulumi up” operation and exist in both configured regions.
Do you want to perform this update? yes
Updating (s3dev):
Type Name Status
+ pulumi:pulumi:Stack dynamic-providers-pulumi-s3dev created (2s)
+ ├─ pulumi:providers:aws west created (0.32s)
+ ├─ pulumi:providers:aws east created (0.62s)
+ ├─ aws:s3:Bucket myBucket-west created (3s)
+ └─ aws:s3:Bucket myBucket-east created (1s)
Resources:
+ 5 created
We can then verify the resource configuration:
$ aws s3api get-bucket-location --bucket provider-bucket-east-0
{
"LocationConstraint": null
}
# aws s3api get-bucket-location --bucket provider-bucket-west-1
{
"LocationConstraint": "us-west-2"
}
Dynamic provider configuration has been a long-standing challenge for Terraform users. The architecture of Terraform, which tightly couples providers with their configurations, makes it difficult to implement dynamic provider configurations without breaking changes. This limitation becomes apparent when attempting to create resources across multiple regions or accounts in a single terraform apply.
Pulumi's custom deployment engine and state schema allow for dynamic provider configurations, setting it apart from other infrastructure-as-code tools like AWS CDK and Terraform CDK, which rely on synthesizing code into intermediate formats before execution. This architectural difference makes Pulumi a compelling choice for users who require the flexibility of dynamic provider configurations in their infrastructure provisioning workflows.
StratusGrid is a certified AWS partner dedicated to helping businesses leverage the full potential of infrastructure-as-code. Our team of experts can help you migrate your existing Terraform configurations to Pulumi, implement dynamic provider configurations, and streamline your multi-cloud and multi-account deployments.
Contact StratusGrid today to learn more about how we can help you achieve infrastructure agility.
Enhance your cloud management with Multiaccount Application Pipeline Terraform Module. Effortlessly deploy and manage applications across multiple...
Explore the step-by-step process of building and deploying a private Terraform Provider Registry in AWS S3. Overcome S3 limitations with practical...
Easily import VMs into AWS with StratusGrid's IAM Role VMImport Terraform module, ensuring secure, efficient migration and role management.