When I started my first Terraform project within GitLab I wondered where should I store my state file. One option could be to store it in my Git repository but is there a better way?!\u00a0<\/p>
The answer is yes there is! This article is about how I set up my pipeline in order to use the GitLab server as my state file backend. There are a couple things that need to be considered.\u00a0\u00a0<\/p>
Read more about it here: https:\/\/docs.gitlab.com\/ee\/user\/infrastructure\/iac\/terraform_state.html<\/a><\/p>
First, create a separate environment. In the GitLab WebUI go to Deployments > Environments > New environment<\/strong><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
Next create an access token, which is needed to communicate with the GitLab server via API and create\/update the state file after changes.\u00a0<\/p>
Go to\u00a0Settings > Access Tokens\u00a0<\/strong>and enter the following input for the project (as an example).<\/p>
Token name:\u00a0terraform<\/strong>
Role:\u00a0Maintainer<\/strong>
Select\u00a0api\u00a0<\/strong><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
Copy the token and save it in a temp file.
The token can not be viewed again and you need it later!<\/strong><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
Create the following variables in Settings > CI\/CD > Expand Variables<\/strong>:<\/p>
\u00a0<\/p>
Make sure that you create the variables and choose the right environment \ud83d\ude09.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
Create a file called init_file.sh<\/strong> which will run in your pipeline to initialize the Terraform environment to use your GitLab server with all the required credentials (user \/ API key).<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t\n\t\t\t\t\t#! \/bin\/bash\nTF_USERNAME=${TF_USERNAME} \\\nTF_PASSWORD=${TF_PASSWORD} \\\nTF_ADDRESS=\"${CI_SERVER_URL}\/api\/v4\/projects\/${CI_PROJECT_ID}\/terraform\/state\/terraform_statefile\" \\\n\nterraform init \\\n -backend-config=address=${TF_ADDRESS} \\\n -backend-config=lock_address=${TF_ADDRESS}\/lock \\\n -backend-config=unlock_address=${TF_ADDRESS}\/lock \\\n -backend-config=username=${TF_USERNAME} \\\n -backend-config=password=${TF_PASSWORD} \\\n -backend-config=lock_method=POST \\\n -backend-config=unlock_method=DELETE \\\n -backend-config=retry_wait_min=5 <\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\tAdjust the main.tf<\/h5>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\t\tAdd the following lines to your main.tf<\/strong> which will tell Terraform to not store the state file locally.<\/p>More about possible backends can be found here: https:\/\/developer.hashicorp.com\/terraform\/language\/settings\/backends\/configuration<\/a><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\n\t\t\t\n\t\t\t\t\n\t\t\t\t\tterraform {\n backend \"http\" {\n }\n}<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\tAdjust the pipeline<\/h5>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\t\tAs an example, I attached the entire pipeline content that I created in my lab to make changes in my ACI environment.\u00a0\u00a0<\/p>
The pipeline consists of the following stages<\/p>
- build
I am building my own Docker container with the required tools and storing it in my GitLab container registry.<\/li> - validate
Validate the Terraform config files for error.<\/li> - plan
The Terraform plan command helps to identify exactly which resources will be created, replaced, changed, or destroyed without executing.\u00a0<\/li> - apply
Terraform executes the necessary changes to the infrastructure.\u00a0<\/li><\/ul>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\n\n\t\t\t\t\t\tINFORMATION<\/span>\n\t\t\t\n\t\t\t\t\t\tIn line, you specify the environment that you created earlier and will map the variables. Pay attention to lines 41-42. Here is the script will run that you created before to set the variables to reach the GitLab Terraform backend. This needs to run in each job. <\/span>\n\t\t\t\n\t\t\t\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\n\t\t\t\n\t\t\t\t\n\t\t\t\t\tvariables:\n IMAGE_NAME_ACI: $CI_REGISTRY_IMAGE\/aci \n IMAGE_TAG_ACI: \"1.0\"\n\nstages:\n - build\n - validate\n - plan\n - apply\n\nbuild_image:\n stage: build\n tags:\n - shell-runner\n script:\n - docker build -t $IMAGE_NAME_ACI:$IMAGE_TAG_ACI docker\/aci\/.\n\npush_image:\n stage: build\n needs:\n - build_image\n tags:\n - shell-runner\n before_script:\n - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY\n script:\n - docker push $IMAGE_NAME_ACI:$IMAGE_TAG_ACI\n\nvalidate:\n needs:\n - push_image\n stage: validate\n image: $IMAGE_NAME_ACI:$IMAGE_TAG_ACI\n environment: ACI\n tags:\n - docker-runner\n before_script:\n - cd terraform\n script:\n - chmod +x init_file.sh\n - .\/init_file.sh\n - terraform validate\n\nplan:\n needs:\n - validate\n stage: plan\n image: $IMAGE_NAME_ACI:$IMAGE_TAG_ACI\n environment: ACI\n tags:\n - docker-runner\n before_script:\n - cd terraform\n script:\n - chmod +x init_file.sh\n - .\/init_file.sh\n - terraform plan\n\napply:\n needs:\n - plan\n stage: apply\n image: $IMAGE_NAME_ACI:$IMAGE_TAG_ACI\n environment: ACI\n tags:\n - docker-runner\n before_script:\n - cd terraform\n script:\n - chmod +x init_file.sh\n - .\/init_file.sh \n - terraform apply --auto-approve\n when: manual<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\tRun the pipeline and check the state file<\/h5>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\t\tLet’s make changes and to trigger the pipeline.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\t\tIn the GitLab WebUI, go to\u00a0Infrastructure > Terraform<\/strong>\u00a0and download the\u00a0state file (JSON)<\/strong>\u00a0and make yourself familiar with the content.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t