Table of Contents
Introduction
When I started my first Terraform project within GitLab I wondered where should I store my state file. One option could be to store it in my Git repository but is there a better way?!
The answer is yes there is! This article is about how I set up my pipeline in order to use the GitLab server as my state file backend. There are a couple things that need to be considered.
Read more about it here: https://docs.gitlab.com/ee/user/infrastructure/iac/terraform_state.html
In the following article, I will configure an ACI environment and use GitLab CE 15.8 on Ubuntu 22.04 as my state file backend.
Create an environment, API key & variables
First, create a separate environment. In the GitLab WebUI go to Deployments > Environments > New environment
Next create an access token, which is needed to communicate with the GitLab server via API and create/update the state file after changes.
Go to Settings > Access Tokens and enter the following input for the project (as an example).
Token name: terraform
Role: Maintainer
Select api
Copy the token and save it in a temp file.
The token can not be viewed again and you need it later!
Create the following variables in Settings > CI/CD > Expand Variables:
- TF_USERNAME: root (use your username)
- TF_PASSWORD: (use your token that you copied, tik also Mask Variable)
Make sure that you create the variables and choose the right environment 😉.
Create a custom script
Create a file called init_file.sh which will run in your pipeline to initialize the Terraform environment to use your GitLab server with all the required credentials (user / API key).
#! /bin/bash
TF_USERNAME=${TF_USERNAME} \
TF_PASSWORD=${TF_PASSWORD} \
TF_ADDRESS="${CI_SERVER_URL}/api/v4/projects/${CI_PROJECT_ID}/terraform/state/terraform_statefile" \
terraform init \
-backend-config=address=${TF_ADDRESS} \
-backend-config=lock_address=${TF_ADDRESS}/lock \
-backend-config=unlock_address=${TF_ADDRESS}/lock \
-backend-config=username=${TF_USERNAME} \
-backend-config=password=${TF_PASSWORD} \
-backend-config=lock_method=POST \
-backend-config=unlock_method=DELETE \
-backend-config=retry_wait_min=5
Adjust the main.tf
Add the following lines to your main.tf which will tell Terraform to not store the state file locally.
More about possible backends can be found here: https://developer.hashicorp.com/terraform/language/settings/backends/configuration
terraform {
backend "http" {
}
}
Adjust the pipeline
As an example, I attached the entire pipeline content that I created in my lab to make changes in my ACI environment.
The pipeline consists of the following stages
- build
I am building my own Docker container with the required tools and storing it in my GitLab container registry. - validate
Validate the Terraform config files for error. - plan
The Terraform plan command helps to identify exactly which resources will be created, replaced, changed, or destroyed without executing. - apply
Terraform executes the necessary changes to the infrastructure.
variables:
IMAGE_NAME_ACI: $CI_REGISTRY_IMAGE/aci
IMAGE_TAG_ACI: "1.0"
stages:
- build
- validate
- plan
- apply
build_image:
stage: build
tags:
- shell-runner
script:
- docker build -t $IMAGE_NAME_ACI:$IMAGE_TAG_ACI docker/aci/.
push_image:
stage: build
needs:
- build_image
tags:
- shell-runner
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
script:
- docker push $IMAGE_NAME_ACI:$IMAGE_TAG_ACI
validate:
needs:
- push_image
stage: validate
image: $IMAGE_NAME_ACI:$IMAGE_TAG_ACI
environment: ACI
tags:
- docker-runner
before_script:
- cd terraform
script:
- chmod +x init_file.sh
- ./init_file.sh
- terraform validate
plan:
needs:
- validate
stage: plan
image: $IMAGE_NAME_ACI:$IMAGE_TAG_ACI
environment: ACI
tags:
- docker-runner
before_script:
- cd terraform
script:
- chmod +x init_file.sh
- ./init_file.sh
- terraform plan
apply:
needs:
- plan
stage: apply
image: $IMAGE_NAME_ACI:$IMAGE_TAG_ACI
environment: ACI
tags:
- docker-runner
before_script:
- cd terraform
script:
- chmod +x init_file.sh
- ./init_file.sh
- terraform apply --auto-approve
when: manual
Run the pipeline and check the state file
Let’s make changes and to trigger the pipeline.
In the GitLab WebUI, go to Infrastructure > Terraform and download the state file (JSON) and make yourself familiar with the content.
That’s it 👍 ! The state file will be stored in your GitLab server locally. There is also an option to use cloud storage like Amazon which I will not discuss in that article.
