Overview
These days Terraform is the industry’s go-to tool for Infrastructure automation. Terraform allows you to write infrastructure as a code, which you can manage via source control and one of many benefits is that you can keep track of the changes of your infrastructure (which is a nightmare for any organization).
How Terraform keep track of the changes in your environment? it creates a terraform.tfstate file on a local filesystem. TF state file is simply a small database of the state of your environment. Whenever you run terraform plan, apply or destroy commands it reads the current state from terraform.tfstate file and applies changes to it.
Problem
The problem arrives when you are working in a team. Since terraform.tfstate file is created on your local file system the other developer does not have visibility to it. When any other developer executes the same scripts terraform will create a new terraform.tfstate file which would be different from the current state.
Common solutions to this issue could be to store terraform.tfstate in a source control, that might work in a small team where one person is working at a time or where you have the option of having a different account for each developer. One issue with that is also the .tfstate file could have some sensitive information (such as RDS passwords) that you don’t want to upload to source control systems like GitHub.
Another solution is to use Terraform enterprise solution which comes with all bells and whistles.
In this post, I will show you how you can solve this problem using Remote backends, how can you setup Terraform to use S3 buckets to keep the state of your environment.
Remote Backends
There are many types of remote backends you can use with Terraform but in this post, we will cover the popular solution of using S3 buckets.
Following are some benefits of using remote backends
- Team Development – when working in a team, remote backends can keep the state of infrastructure at a centralized location
- Sensitive Information – with remote backends your sensitive information would not be stored on local disk
- Remote Operations – Infrastructure build could be a time-consuming task, some remote backends supports remote execution of the tasks. You can then turn off your computer and your operation will still complete. Paired with remote state storage and locking above, this also helps in team environments.
I hope that gives you enough info on remote backends, let’s dive into the solution.
Pre-requisites
- Make sure the IAM user has Full Access to S3 (attach proper IAM policies)
Note: Best practice is to assign policies to a group in my case it is called Developers
- S3 bucket in your account (the name is globally unique) in my case is it datanextapps, create necessary sub-folders if you like in my case it is terraform/dev to keep my *.tfstate file separately for each environment
- Access Keys and Secret Keys are setup in your Terraform code or AWS profile on the local machine (not covered in this post)
Solution
Note: Download example Terraform code from our GitHub repo
The solution to this problem is quite simple you need to add the following code in your .tf file
terraform {
backend "s3" {
bucket="datanextapps"
key="terraform/dev/terraform_dev.tfstate"
region="us-east-1"
}
}
bucket – the name of the S3 bucket in your account
key – the name of .tfstate file in my example I am keeping the file under terraform/dev/ folders inside my datanextapps bucket
region – region of your s3 bucket
To see the running example download this code
Setup AWS access keys and secret keys using aws configure command, optionally replace the values in the code
Execute the following commands from the folder where your main terraform file exists
$ terraform init
$ terraform plan
$ terraform apply
$ terraform show
If all goes well you will see terraform.tfstate file in your S3 bucket as shown in the slide
Hope you find this post useful, please like share or clap
About DataNext
DataNext Solutions is US based system integrator, specialized in Cloud, Big Data, DevOps technologies. As a registered AWS partner, our services comprise of any Cloud Migration, Cost optimization, Integration, Security and Managed Services. Click here and Book Free assessment call with our experts today or visit our website www.datanextsolutions.com for more info.