Terraform 1.15 Alpha: Development overrides
I work with the AWSCC provider a lot. What does that really mean ? I am either testing new versions, or testing inflight schemas before a release is published or in some cases testing my own fixes to the provider before pushing to the upstream repositories. Now, the first example doesn’t really require much than upgrading your provider version to the latest one or leaving the version constraint out completely. In cases where you are in a directory with an existing .terraform.provider.lock file, you need an terraform init -upgrade to get the latest version. The other two scenarios are based on unreleased local builds of the provider.
Using a local version of the provider ( binary) requires you to inform terraform that you need to use your local binary than the version publicly available. How do you do this ?
Dev overrides
Reference: development overrides.
TLDR; you configure
dev_overridesin your.terraformrcfile. Terraform looks at the user directory by default and uses any added overrides.
# ~/.terraformrc
provider_installation {
dev_overrides {
"namespace/provider_name" = <path_to_binary>
}
direct {}
}
Lets test this. What Terraform version am I on ?
tf --version
Terraform v1.14.5
on darwin_arm64
Base configuration
# main.tf
terraform {
required_providers {
random = {
source = "hashicorp/random"
version = "~> 3.0"
}
}
}
resource "random_pet" "test" {
length = 2
}
resource "awscc_s3_bucket" "before_tf115" {
}
Run terraform init
terraform init
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/random versions matching "~> 3.0"...
- Finding latest version of hashicorp/awscc...
- Installing hashicorp/random v3.8.1...
- Installed hashicorp/random v3.8.1 (signed by HashiCorp)
- Installing hashicorp/awscc v1.72.0...
- Installed hashicorp/awscc v1.72.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
╷
│ Warning: Provider development overrides are in effect
│
│ The following provider development overrides are set in the CLI configuration:
│ - hashicorp/awscc in /Users/manuchn/.local/bin
│
│ Skip terraform init when using provider development overrides. It is not necessary and may error unexpectedly.
╵
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Look at these lines..
- Finding latest version of hashicorp/awscc...
- Installing hashicorp/random v3.8.1...
- Installed hashicorp/random v3.8.1 (signed by HashiCorp)
- Installing hashicorp/awscc v1.72.0...
- Installed hashicorp/awscc v1.72.0 (signed by HashiCorp)
Even after specifying the dev overrides, the latest provider gets pulled in for my local run. Reviewing the local .terraform directory confirms that the provider version 1.72.0 gets downloaded.
tree .terraform/
.terraform/
└── providers
└── registry.terraform.io
└── hashicorp
├── awscc
│ └── 1.72.0
│ └── darwin_arm64 -> /Users/manuchn/.terraform.d/plugin-cache/registry.terraform.io/hashicorp/awscc/1.72.0/darwin_arm64
└── random
└── 3.8.1
└── darwin_arm64 -> /Users/manuchn/.terraform.d/plugin-cache/registry.terraform.io/hashicorp/random/3.8.1/darwin_arm64
lock file
cat .terraform.lock.hcl
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
provider "registry.terraform.io/hashicorp/awscc" {
version = "1.72.0"
hashes = [
"h1:hhwZ3kR8iJcCVDKpCe13kr+xkeg8x3PUPR7yd4aHADQ=",
"zh:0d104a115655e730d610ad293fde399fbdcb18b8fb0d2759b7f7686d51cd7404",
"zh:11e9e67b68e8d4d6738d0d7355b34b2617cbd0f857c7e69e59f829c1fa493d56",
"zh:17219beea499bc8e7c0af32c8df8106d62075e5c410cc102d18cdae951ee7ab7",
"zh:3a859627bac0c8abab251502d54ebd9e551bcad7066b98d44f304728541d9f7a",
"zh:45f831d2dbe24ddea941705d0611261785ee0127bf17f3d09c67e48d54c976f2",
"zh:553aec846b43cb719ecc0a3db41b202bb09190bf7fb1f884df1496baaacfca39",
"zh:5c9d1cc8e5c7c6d26de90dc61960d0a56290dcb4f2387581ab86b7b4383857ad",
"zh:6392677132bff1cba10cf289f01037f9c3e33422da2fda8db48ce51e896168ab",
"zh:a3fd1e1ef3fe68859e9e707f4f78b3f0b75ce421547b47c710a0389670974408",
"zh:ab53728650fad7c062e56cc31154ed56662352c2c6ee19b20d700a1d118520da",
"zh:bc09e0d912213b7259beab55299aa59d07bee7741a03611ae915596981d62090",
"zh:bfec079186bec6dcb6e67869bdda6c3b3bd16fabfa2d5fa95d03c1f891649fac",
"zh:c21e391d2268d804e23d26fa499edddfd3466da616dbaa964657890af18fefb3",
"zh:ebcc673cab45e6c7c5460fd81f012f8f2227400de344f51236145f3f04884433",
"zh:f809ab383cca0a5f83072981c64208cbd7fa67e986a86ee02dd2c82333221e32",
]
}
provider "registry.terraform.io/hashicorp/random" {
version = "3.8.1"
constraints = "~> 3.0"
hashes = [
"h1:u8AKlWVDTH5r9YLSeswoVEjiY72Rt4/ch7U+61ZDkiQ=",
"zh:08dd03b918c7b55713026037c5400c48af5b9f468f483463321bd18e17b907b4",
"zh:0eee654a5542dc1d41920bbf2419032d6f0d5625b03bd81339e5b33394a3e0ae",
"zh:229665ddf060aa0ed315597908483eee5b818a17d09b6417a0f52fd9405c4f57",
"zh:2469d2e48f28076254a2a3fc327f184914566d9e40c5780b8d96ebf7205f8bc0",
"zh:37d7eb334d9561f335e748280f5535a384a88675af9a9eac439d4cfd663bcb66",
"zh:741101426a2f2c52dee37122f0f4a2f2d6af6d852cb1db634480a86398fa3511",
"zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3",
"zh:a902473f08ef8df62cfe6116bd6c157070a93f66622384300de235a533e9d4a9",
"zh:b85c511a23e57a2147355932b3b6dce2a11e856b941165793a0c3d7578d94d05",
"zh:c5172226d18eaac95b1daac80172287b69d4ce32750c82ad77fa0768be4ea4b8",
"zh:dab4434dba34aad569b0bc243c2d3f3ff86dd7740def373f2a49816bd2ff819b",
"zh:f49fd62aa8c5525a5c17abd51e27ca5e213881d58882fd42fec4a545b53c9699",
]
}
Let’s check what the size of these providers are.
ls -al .terraform/providers/registry.terraform.io/hashicorp/awscc/1.72.0/darwin_arm64/
total 164008
drwxr-xr-x@ 4 manuchn staff 128 Feb 22 20:43 .
drwxr-xr-x@ 3 manuchn staff 96 Feb 22 20:43 ..
-rw-rw-r--@ 1 manuchn staff 16755 Feb 22 23:12 LICENSE.txt
-rwxr-xr-x@ 1 manuchn staff 71368720 Feb 22 23:12 terraform-provider-awscc_v1.72.0 -> # Around 70MB.
For a beefier provider like aws, this would be around 700MB. Now, I don’t want all of that spread around my filesystem. For normal terraform module authoring and non-provider triaging work, I would normally go for setting up a pluign cache which would use a global set of plugins (or providers) by the version available. In my case, it doesn’t work as I have to keep changing this setting to ignore the cache for my local builds. Maybe there is a better way to do this.
Lets stick to the dev overrides and provider development use case for now.
TF version 1.15 alpha
The latest alpha version of Terraform at the time of writing this had this line item.
init: skip dependencies declared in development override. This allows you to use terraform init with developer overrides and install dependencies that are not declared in the override file.
And the description in the PR from Daniel , the author, captures my frustration to the dot.
Scratching my own itch here, I found this quite annoying when developing a provider recently. Each time I added another provider to my demo I had to remove my code for the overwritten provider, run init, re-add my code. Quite tedious. With this change init runs successfully with a development override and only installs what is not in the development override.
Lets retest this with the alpha version and see how this behaves. The configuration and override files remain the same (except for the slight change in the id).
# main.tf
terraform {
required_providers {
random = {
source = "hashicorp/random"
version = "~> 3.0"
}
}
}
resource "random_pet" "test" {
length = 2
}
resource "awscc_s3_bucket" "after_tf115" {
}
Using tfenv switch to the alpha version.
tfenv use 1.15.0-alpha20260218
Switching default version to v1.15.0-alpha20260218
Default version (when not overridden by .terraform-version or TFENV_TERRAFORM_VERSION) is now: 1.15.0-alpha20260218
> terraform --version
Terraform v1.15.0-alpha20260218
on darwin_arm64
Run terraform init. No trace of the call to the registry to get the version of awscc provider as terraform recognizes the overrides in ~/.terraformrc.
terraform init
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/random versions matching "~> 3.0"...
- Installing hashicorp/random v3.8.1...
- Installed hashicorp/random v3.8.1 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
╷
│ Warning: Provider development overrides are in effect
│
│ The following provider development overrides are set in the CLI configuration:
│ - hashicorp/awscc in /Users/manuchn/.local/bin
│
│ These providers are not installed as part of init since they were overwritten. If this is unintentional please re-run without the development overrides
│ set.
╵
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
What does the .terraform directory look like ?
tree .terraform/
.terraform/
└── providers
└── registry.terraform.io
└── hashicorp
└── random
└── 3.8.1
└── darwin_arm64 -> /Users/manuchn/.terraform.d/plugin-cache/registry.terraform.io/hashicorp/random/3.8.1/darwin_arm64
7 directories, 0 files
Terraform lock file
As expected the lock file has no entry for the aws provider here.
cat .terraform.lock.hcl
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
provider "registry.terraform.io/hashicorp/random" {
version = "3.8.1"
constraints = "~> 3.0"
hashes = [
"h1:u8AKlWVDTH5r9YLSeswoVEjiY72Rt4/ch7U+61ZDkiQ=",
"zh:08dd03b918c7b55713026037c5400c48af5b9f468f483463321bd18e17b907b4",
"zh:0eee654a5542dc1d41920bbf2419032d6f0d5625b03bd81339e5b33394a3e0ae",
"zh:229665ddf060aa0ed315597908483eee5b818a17d09b6417a0f52fd9405c4f57",
"zh:2469d2e48f28076254a2a3fc327f184914566d9e40c5780b8d96ebf7205f8bc0",
"zh:37d7eb334d9561f335e748280f5535a384a88675af9a9eac439d4cfd663bcb66",
"zh:741101426a2f2c52dee37122f0f4a2f2d6af6d852cb1db634480a86398fa3511",
"zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3",
"zh:a902473f08ef8df62cfe6116bd6c157070a93f66622384300de235a533e9d4a9",
"zh:b85c511a23e57a2147355932b3b6dce2a11e856b941165793a0c3d7578d94d05",
"zh:c5172226d18eaac95b1daac80172287b69d4ce32750c82ad77fa0768be4ea4b8",
"zh:dab4434dba34aad569b0bc243c2d3f3ff86dd7740def373f2a49816bd2ff819b",
"zh:f49fd62aa8c5525a5c17abd51e27ca5e213881d58882fd42fec4a545b53c9699",
]
}
Let’s verify that this actually provisions and destroys resources as expected. I do have some aliases for terraform apply and destroy.
Apply :
> tfy
╷
│ Warning: Provider development overrides are in effect
│
│ The following provider development overrides are set in the CLI configuration:
│ - hashicorp/awscc in /Users/manuchn/.local/bin
│
│ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with
│ published releases.
╵
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# awscc_s3_bucket.after_tf115 will be created
+ resource "awscc_s3_bucket" "after_tf115" {
+ accelerate_configuration = (known after apply)
+ access_control = (known after apply)
+ analytics_configurations = (known after apply)
+ arn = (known after apply)
+ bucket_encryption = (known after apply)
+ bucket_name = (known after apply)
+ cors_configuration = (known after apply)
+ domain_name = (known after apply)
+ dual_stack_domain_name = (known after apply)
+ id = (known after apply)
+ intelligent_tiering_configurations = (known after apply)
+ inventory_configurations = (known after apply)
+ lifecycle_configuration = (known after apply)
+ logging_configuration = (known after apply)
+ metadata_table_configuration = (known after apply)
+ metrics_configurations = (known after apply)
+ notification_configuration = (known after apply)
+ object_lock_configuration = (known after apply)
+ object_lock_enabled = (known after apply)
+ ownership_controls = (known after apply)
+ public_access_block_configuration = (known after apply)
+ regional_domain_name = (known after apply)
+ replication_configuration = (known after apply)
+ tags = (known after apply)
+ versioning_configuration = (known after apply)
+ website_configuration = (known after apply)
+ website_url = (known after apply)
}
# random_pet.test will be created
+ resource "random_pet" "test" {
+ id = (known after apply)
+ length = 2
+ separator = "-"
}
Plan: 2 to add, 0 to change, 0 to destroy.
random_pet.test: Creating...
random_pet.test: Creation complete after 0s [id=pro-falcon]
awscc_s3_bucket.after_tf115: Creating...
awscc_s3_bucket.after_tf115: Still creating... [00m10s elapsed]
awscc_s3_bucket.after_tf115: Creation complete after 13s [id=4j7dmsn9yf1gms5ytdyq5kezk-g0lzliefelhz]
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
Destroy:
experiments/tf_validate/test_dev_overrides
❯ tfd
╷
│ Warning: Provider development overrides are in effect
│
│ The following provider development overrides are set in the CLI configuration:
│ - hashicorp/awscc in /Users/manuchn/.local/bin
│
│ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with
│ published releases.
╵
random_pet.test: Refreshing state... [id=pro-falcon]
awscc_s3_bucket.after_tf115: Refreshing state... [id=4j7dmsn9yf1gms5ytdyq5kezk-g0lzliefelhz]
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
- destroy
Terraform will perform the following actions:
# awscc_s3_bucket.after_tf115 will be destroyed
- resource "awscc_s3_bucket" "after_tf115" {
- arn = "arn:aws:s3:::4j7dmsn9yf1gms5ytdyq5kezk-g0lzliefelhz" -> null
- bucket_encryption = {
- server_side_encryption_configuration = [
- {
- bucket_key_enabled = false -> null
- server_side_encryption_by_default = {
- sse_algorithm = "AES256" -> null
} -> null
},
] -> null
} -> null
- bucket_name = "4j7dmsn9yf1gms5ytdyq5kezk-g0lzliefelhz" -> null
- domain_name = "4j7dmsn9yf1gms5ytdyq5kezk-g0lzliefelhz.s3.amazonaws.com" -> null
- dual_stack_domain_name = "4j7dmsn9yf1gms5ytdyq5kezk-g0lzliefelhz.s3.dualstack.us-east-1.amazonaws.com" -> null
- id = "4j7dmsn9yf1gms5ytdyq5kezk-g0lzliefelhz" -> null
- ownership_controls = {
- rules = [
- {
- object_ownership = "BucketOwnerEnforced" -> null
},
] -> null
} -> null
- public_access_block_configuration = {
- block_public_acls = true -> null
- block_public_policy = true -> null
- ignore_public_acls = true -> null
- restrict_public_buckets = true -> null
} -> null
- regional_domain_name = "4j7dmsn9yf1gms5ytdyq5kezk-g0lzliefelhz.s3.us-east-1.amazonaws.com" -> null
- website_url = "http://4j7dmsn9yf1gms5ytdyq5kezk-g0lzliefelhz.s3-website-us-east-1.amazonaws.com" -> null
}
# random_pet.test will be destroyed
- resource "random_pet" "test" {
- id = "pro-falcon" -> null
- length = 2 -> null
- separator = "-" -> null
}
Plan: 0 to add, 0 to change, 2 to destroy.
random_pet.test: Destroying... [id=pro-falcon]
random_pet.test: Destruction complete after 0s
awscc_s3_bucket.after_tf115: Destroying... [id=4j7dmsn9yf1gms5ytdyq5kezk-g0lzliefelhz]
awscc_s3_bucket.after_tf115: Destruction complete after 6s
Destroy complete! Resources: 2 destroyed.
So what did it solve ?
If you work on Terraform providers of your own or contribute to some established providers, this would help reduce the number of downloads you are doing for a provider with dev overrides specified. And, you can be sure that it did indeed your local version than the downloaded version when you are fixing things. In short, terraform init respects the provider development overrides.
References
Provider Development Overrides:
- PR #37884 - init: skip dependencies declared in development override
- Issue #27459 - Original issue (January 2021)
- Provider Development Documentation