GitLab as a Terraform state file backend 📝

Introduction

When I started my first Terraform project within GitLab I wondered where should I store my state file. One option could be to store it in my Git repository but is there a better way?!

The answer is yes there is! This article is about how I set up my pipeline in order to use the GitLab server as my state file backend. There are a couple things that need to be considered.

Read more about it here: https://docs.gitlab.com/ee/user/infrastructure/iac/terraform_state.html

In the following article, I will configure an ACI environment and use GitLab CE 15.8 on Ubuntu 22.04 as my state file backend.

Create an environment, API key & variables

First, create a separate environment. In the GitLab WebUI go to Deployments > Environments > New environment

Next create an access token, which is needed to communicate with the GitLab server via API and create/update the state file after changes.

Go to **Settings > Access Tokens **and enter the following input for the project (as an example).

Token name: terraform
Role: Maintainer
Select api

Copy the token and save it in a temp file.
The token can not be viewed again and you need it later!

Create the following variables in Settings > CI/CD > Expand Variables:

• TF_USERNAME: root (use your username)
• TF_PASSWORD: (use your token that you copied, tik also Mask Var****iable)

Make sure that you create the variables and choose the right environment 😉.

Create a custom script

Create a file called init_file.sh which will run in your pipeline to initialize the Terraform environment to use your GitLab server with all the required credentials (user / API key).

				
					#! /bin/bash
TF_USERNAME=${TF_USERNAME} \
TF_PASSWORD=${TF_PASSWORD} \
TF_ADDRESS="${CI_SERVER_URL}/api/v4/projects/${CI_PROJECT_ID}/terraform/state/terraform_statefile" \

terraform init \
  -backend-config=address=${TF_ADDRESS} \
  -backend-config=lock_address=${TF_ADDRESS}/lock \
  -backend-config=unlock_address=${TF_ADDRESS}/lock \
  -backend-config=username=${TF_USERNAME} \
  -backend-config=password=${TF_PASSWORD} \
  -backend-config=lock_method=POST \
  -backend-config=unlock_method=DELETE \
  -backend-config=retry_wait_min=5 
				
			

Adjust the main.tf

Add the following lines to your main.tf which will tell Terraform to not store the state file locally.

More about possible backends can be found here: https://developer.hashicorp.com/terraform/language/settings/backends/configuration

				
					terraform {
  backend "http" {
  }
}
				
			

Adjust the pipeline

As an example, I attached the entire pipeline content that I created in my lab to make changes in my ACI environment.

The pipeline consists of the following stages

• build
  I am building my own Docker container with the required tools and storing it in my GitLab container registry.
• validate
  Validate the Terraform config files for error.
• plan
  The Terraform plan command helps to identify exactly which resources will be created, replaced, changed, or destroyed without executing.
• apply
  Terraform executes the necessary changes to the infrastructure.

				
					variables:
  IMAGE_NAME_ACI: $CI_REGISTRY_IMAGE/aci  
  IMAGE_TAG_ACI: "1.0"

stages:
  - build
  - validate
  - plan
  - apply

build_image:
  stage: build
  tags:
    - shell-runner
  script:
    - docker build -t $IMAGE_NAME_ACI:$IMAGE_TAG_ACI docker/aci/.

push_image:
  stage: build
  needs:
    - build_image
  tags:
    - shell-runner
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker push $IMAGE_NAME_ACI:$IMAGE_TAG_ACI

validate:
  needs:
    - push_image
  stage: validate
  image: $IMAGE_NAME_ACI:$IMAGE_TAG_ACI
  environment: ACI
  tags:
    - docker-runner
  before_script:
    - cd terraform
  script:
      - chmod +x init_file.sh
      - ./init_file.sh
      - terraform validate

plan:
  needs:
    - validate
  stage: plan
  image: $IMAGE_NAME_ACI:$IMAGE_TAG_ACI
  environment: ACI
  tags:
    - docker-runner
  before_script:
    - cd terraform
  script:
      - chmod +x init_file.sh
      - ./init_file.sh
      - terraform plan

apply:
  needs:
    - plan
  stage: apply
  image: $IMAGE_NAME_ACI:$IMAGE_TAG_ACI
  environment: ACI
  tags:
    - docker-runner
  before_script:
    - cd terraform
  script:
      - chmod +x init_file.sh
      - ./init_file.sh     
      - terraform apply --auto-approve
  when: manual
				
			

Run the pipeline and check the state file

Let’s make changes and to trigger the pipeline.

In the GitLab WebUI, go to Infrastructure > Terraform and download the state file (JSON) and make yourself familiar with the content.

That’s it 👍 ! The state file will be stored in your GitLab server locally. There is also an option to use cloud storage like Amazon which I will not discuss in that article.