Gitlab CI Terraform use plan and apply in diferent stage

General
1

I want to use terraform plan and terraform apply in different stage, so I use artifacts I build my own docker image with teraform cli.

the structure of my folder

.gitlab-ci.yaml
terraform/main.tf
terrfaorm/variable.tf

$ cat .gitlab-ci.yaml

image: docker_image:0.0

variables:
    PLAN: tfplan
    TF_IN_AUTOMATION: "true"

.terraform-init: &terraform-init
      - terraform -chdir=terraform init -upgrade -backend-config="XX"

stages:
    - validate
    - plan
    - apply

validate:
  stage: validate
  before_script:
    - *terraform-init
  script:
    - terraform -chdir=terraform validate

plan:
  stage: plan
  before_script:
    - *terraform-init
    - terraform -chdir=terraform plan -out $PLAN
  artifacts:
    paths:
      - $PLAN

apply:
  stage: apply
  before_script:
    - *terraform-init
  script:
    - terraform -chdir=terraform  apply -input=false -lock=false -auto-approve $PLAN
  dependencies:
    - plan
  when: manual

I have got this error

Error: Failed to load “tfplan” as a plan file Error: state tfplan: no such ile or directory

I try to looking for with ls command, but I don’t find it on stage apply

2

The issue you are facing is related to the artifacts configuration in GitLab CI. Artifacts must be explicitly preserved between stages in the pipeline, and you must define them properly to ensure that the tfplan file generated in the plan stage is available in the apply stage. Here’s a detailed explanation and a solution.

Why Is This Happening?

  • The terraform plan command generates the plan file ($PLAN) during the plan stage, but unless it is saved and passed as an artifact, the file is not accessible in the apply stage.
  • By default, artifacts are stored in a temporary location for the job that generates them but are not automatically passed to subsequent jobs.

Solution

You need to:

  1. Properly save the tfplan file as an artifact in the plan stage.
  2. Ensure the apply stage uses the artifact from the plan stage.

Here’s the corrected .gitlab-ci.yaml file:

image: docker_image:0.0

variables:
  PLAN: tfplan
  TF_IN_AUTOMATION: "true"

.terraform-init: &terraform-init
  - terraform -chdir=terraform init -upgrade -backend-config="XX"

stages:
  - validate
  - plan
  - apply

validate:
  stage: validate
  before_script:
    - *terraform-init
  script:
    - terraform -chdir=terraform validate

plan:
  stage: plan
  before_script:
    - *terraform-init
  script:
    - terraform -chdir=terraform plan -out $PLAN
  artifacts:
    paths:
      - terraform/$PLAN  # Ensure correct path is specified
    expire_in: 1 hour    # Set the expiration time for the artifact

apply:
  stage: apply
  before_script:
    - *terraform-init
  script:
    - terraform -chdir=terraform apply -input=false -lock=false -auto-approve $PLAN
  dependencies:
    - plan              # Specify dependency to access artifacts from 'plan'
  when: manual          # Allow manual triggering of the apply stage
  artifacts:
    expire_in: 1 hour   # Optionally set expiration for apply artifacts

Key Changes

  1. artifacts in the plan stage:
  • Ensures that the tfplan file is saved and available for later stages.
  • The correct path (terraform/$PLAN) must match the location where terraform plan generates the file.
  • Added expire_in to limit how long artifacts are retained.
  1. dependencies in the apply stage:
  • Ensures the apply stage retrieves the tfplan artifact from the plan stage.
  1. Path to the tfplan file:
  • In the apply stage, ensure that the tfplan file is referenced in the same path as it was saved during the plan stage.

Debugging Tips

To ensure the file exists and is correctly passed:

  • Add a debugging step in the apply stage:
apply:
  stage: apply
  before_script:
    - *terraform-init
    - ls -la terraform  # Check if tfplan exists in the expected location
  script:
    - terraform -chdir=terraform apply -input=false -lock=false -auto-approve $PLAN
  dependencies:
    - plan
  when: manual

This will help verify whether the tfplan file exists and where it is located before applying it.

Final Notes

  1. Artifacts Cleanup: Ensure that artifacts expire after a reasonable time to avoid storage issues in GitLab.
  2. Manual Apply: Keeping apply as a manual stage is a good practice for production deployments to prevent accidental changes.
  3. Path Consistency: Always double-check paths for the tfplan file in both plan and apply stages.

Let me know if you still encounter issues!