Using Terraform from an Azure DevOps pipeline

There’s a lab out there showing how to use Terraform with Azure DevOps. I found it clunky. Here’s a similar take using YAML pipelines.

Pre-Requisites

A service connection
Azure DevOps extension
A terraform configuration file with an Azure back-end declared

provider "azurerm" { version = "~>1.43" }

Overview

Terraform uses state files to keep track of the desired state. When working in teams, it’s very useful to keep this statefile in a central location so that you and your team members don’t override each others changes. Terraform has many backends that store this state file including Azure.

The lab mentioned above, makes you create the storage account that will persist this state file. With these instructions, the pipeline will do it instead.

Set-up

First define the following variables in your pipeline. It can be useful to apply all these steps to the same pipeline that you use to deploy your code from. Assuming, of course, that you’re using the Terraform config files to provision some infrastructure for applications.

variables: 
 - name: TerraformBackend.ResourceGroup
   value: rg-realworld-staging-001
 - name: TerraformBackend.StorageAccount
   value: strwstagingterraform01
 - name: TerraformBackend.ContainerName
   value: staging
 - group: 'staging'

In the staging variable group, there’s only settings corresponding to the application that will be deployed after the infrastructure is provisioned.

Then, let’s set-up the Azure backend

steps:
- task: AzureCLI@2
  inputs:
    azureSubscription: '[name of your service connection]'
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      az group create --location eastus --name $(TerraformBackend.ResourceGroup)

      az storage account create --name $(TerraformBackend.StorageAccount) --resource-group $(TerraformBackend.ResourceGroup) --location eastus --sku Standard_LRS

      az storage container create --name staging --account-name $(TerraformBackend.StorageAccount)

      STORAGE_ACCOUNT_KEY=$(az storage account keys list -g $(TerraformBackend.ResourceGroup) -n $(TerraformBackend.StorageAccount) | jq ".[0].value" -r)

      echo "setting storage account key variable"
      echo "##vso[task.setvariable variable=ARM_ACCESS_KEY;issecret=true]$STORAGE_ACCOUNT_KEY"

This step will use the Azure CLI to:

Create a storage account, storage container, and resource group for the Terraform state file
Read the account key for the storage account
Assign the account key to the ARM_ACCESS_KEY pipeline variable. It’s important because pipeline variables are also mapped to environment variables

Initializing

Then we’ll want to initialize Terraform. But first, you’ll want to make sure that you have Terraform installed on the build agent.

- task: TerraformInstaller@0
  inputs:
    terraformVersion: '0.12.20'

Now, initialize Terraform.

- task: TerraformTaskV1@0
  displayName: "Terraform Init"
  inputs:
    provider: 'azurerm'
    command: 'init'
    backendServiceArm: '[name of your service connection]'
    backendAzureRmResourceGroupName: $(TerraformBackend.ResourceGroup)
    backendAzureRmStorageAccountName: $(TerraformBackend.StorageAccount)
    backendAzureRmContainerName: '$(TerraformBackend.ContainerName)'
    backendAzureRmKey: 'infrastructure/terraform.tfstate'
    workingDirectory: '$(System.DefaultWorkingDirectory)/iac/staging/'

You’ll want to ensure that your working directory points to the root of your Terraform module. In my case, I’m storing the Terraform configuration under the iac directory in my repository.

The environment variable ARM_ACCESS_KEY will be used by the Terraform Azure back-end provider to connect to the Storage Account. You can read more about it here. I’ve had issues authenticating with just the service principal.

Once is Terraform is initialized, you’ll ready to create infrastructure.

Apply

When you apply, it’s when the magic happens.

- task: TerraformTaskV1@0
  displayName: "Terraform Apply"
  inputs:
    provider: 'azurerm'
    command: 'apply'
    workingDirectory: '$(System.DefaultWorkingDirectory)/iac/staging/'
    environmentServiceNameAzureRM: '[name of your service connection]'
    commandOptions: |
      -var "some_key=$(SomeValue)"

Now your Terraform plan will get executed and your state will be updated.

By the way, using the commandOptions you can also pass in input variables. But a cleaner way would be to use environment variables. Any time you have a pipeline variable on Azure DevOps, it gets mapped to an environment variable. So, if you have a variable group with names that start with TF_VAR then Terraform will pick it up and pass the value to the input variable.

Save. Queue. Run!

Putting it all together

In the end, a CI/CD pipeline for an application/API could have the following steps.

Ensure Terraform is set-up (backend storage account)
Ensure Terraform CLI is installed on the pipeline agent
Ensure Terraform is initialized
Apply the Terraform files
Deploy the application

Using Terraform from an Azure DevOps pipeline

Comments

More from this blog

Here are my "lessons learned" after 5+ years of DevOps consulting

Deploying Terraform at scale with GitHub Actions

Using Terragrunt to deploy to Azure

You don't need secrets to deploy to Azure from GitHub Actions 🤯

Install ArgoCD on an AKS cluster with NGINX

Pre-Requisites

Overview

Set-up

Initializing

Apply

Putting it all together

Command Palette

Comments

More from this blog

Pre-Requisites

Overview

Set-up

Initializing

Apply

Putting it all together