GitLab as a Terraform state file backend 📝

Chris Beye

Chris Beye

Systems Architect @ Cisco Systems

Table of Contents

Introduction

When I started my first Terraform project within GitLab I wondered where should I store my state file. One option could be to store it in my Git repository but is there a better way?! 

The answer is yes there is! This article is about how I set up my pipeline in order to use the GitLab server as my state file backend. There are a couple things that need to be considered.  

Read more about it here: https://docs.gitlab.com/ee/user/infrastructure/iac/terraform_state.html

In the following article, I will configure an ACI environment and use GitLab CE 15.8 on Ubuntu 22.04 as my state file backend.

Create an environment, API key & variables

First, create a separate environment. In the GitLab WebUI go to Deployments > Environments > New environment

Next create an access token, which is needed to communicate with the GitLab server via API and create/update the state file after changes. 

Go to Settings > Access Tokens and enter the following input for the project (as an example).

Token name: terraform
Role: Maintainer
Select api 

Copy the token and save it in a temp file.
The token can not be viewed again and you need it later!

Create the following variables in Settings > CI/CD > Expand Variables:

  • TF_USERNAME: root (use your username)
  • TF_PASSWORD: (use your token that you copied, tik also Mask Variable)

 

Make sure that you create the variables and choose the right environment 😉.

Create a custom script

Create a file called init_file.sh which will run in your pipeline to initialize the Terraform environment to use your GitLab server with all the required credentials (user / API key).

				
					#! /bin/bash
TF_USERNAME=${TF_USERNAME} \
TF_PASSWORD=${TF_PASSWORD} \
TF_ADDRESS="${CI_SERVER_URL}/api/v4/projects/${CI_PROJECT_ID}/terraform/state/terraform_statefile" \

terraform init \
  -backend-config=address=${TF_ADDRESS} \
  -backend-config=lock_address=${TF_ADDRESS}/lock \
  -backend-config=unlock_address=${TF_ADDRESS}/lock \
  -backend-config=username=${TF_USERNAME} \
  -backend-config=password=${TF_PASSWORD} \
  -backend-config=lock_method=POST \
  -backend-config=unlock_method=DELETE \
  -backend-config=retry_wait_min=5
Adjust the main.tf

Add the following lines to your main.tf which will tell Terraform to not store the state file locally.

More about possible backends can be found here: https://developer.hashicorp.com/terraform/language/settings/backends/configuration

				
					terraform {
  backend "http" {
  }
}
Adjust the pipeline

As an example, I attached the entire pipeline content that I created in my lab to make changes in my ACI environment.  

The pipeline consists of the following stages

  • build
    I am building my own Docker container with the required tools and storing it in my GitLab container registry.
  • validate
    Validate the Terraform config files for error.
  • plan
    The Terraform plan command helps to identify exactly which resources will be created, replaced, changed, or destroyed without executing. 
  • apply
    Terraform executes the necessary changes to the infrastructure. 
				
					variables:
  IMAGE_NAME_ACI: $CI_REGISTRY_IMAGE/aci  
  IMAGE_TAG_ACI: "1.0"

stages:
  - build
  - validate
  - plan
  - apply

build_image:
  stage: build
  tags:
    - shell-runner
  script:
    - docker build -t $IMAGE_NAME_ACI:$IMAGE_TAG_ACI docker/aci/.

push_image:
  stage: build
  needs:
    - build_image
  tags:
    - shell-runner
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker push $IMAGE_NAME_ACI:$IMAGE_TAG_ACI

validate:
  needs:
    - push_image
  stage: validate
  image: $IMAGE_NAME_ACI:$IMAGE_TAG_ACI
  environment: ACI
  tags:
    - docker-runner
  before_script:
    - cd terraform
  script:
      - chmod +x init_file.sh
      - ./init_file.sh
      - terraform validate

plan:
  needs:
    - validate
  stage: plan
  image: $IMAGE_NAME_ACI:$IMAGE_TAG_ACI
  environment: ACI
  tags:
    - docker-runner
  before_script:
    - cd terraform
  script:
      - chmod +x init_file.sh
      - ./init_file.sh
      - terraform plan

apply:
  needs:
    - plan
  stage: apply
  image: $IMAGE_NAME_ACI:$IMAGE_TAG_ACI
  environment: ACI
  tags:
    - docker-runner
  before_script:
    - cd terraform
  script:
      - chmod +x init_file.sh
      - ./init_file.sh     
      - terraform apply --auto-approve
  when: manual
Run the pipeline and check the state file

Let’s make changes and to trigger the pipeline.

In the GitLab WebUI, go to Infrastructure > Terraform and download the state file (JSON) and make yourself familiar with the content.

That’s it 👍 ! The state file will be stored in your GitLab server locally. There is also an option to use cloud storage like Amazon which I will not discuss in that article.  

More Articles