The issue you are facing is related to the artifacts configuration in GitLab CI. Artifacts must be explicitly preserved between stages in the pipeline, and you must define them properly to ensure that the tfplan file generated in the plan stage is available in the apply stage. Here’s a detailed explanation and a solution.
Why Is This Happening?
The terraform plan command generates the plan file ($PLAN) during the plan stage, but unless it is saved and passed as an artifact, the file is not accessible in the apply stage.
By default, artifacts are stored in a temporary location for the job that generates them but are not automatically passed to subsequent jobs.
Solution
You need to:
Properly save the tfplan file as an artifact in the plan stage.
Ensure the apply stage uses the artifact from the plan stage.
Here’s the corrected .gitlab-ci.yaml file:
image: docker_image:0.0
variables:
PLAN: tfplan
TF_IN_AUTOMATION: "true"
.terraform-init: &terraform-init
- terraform -chdir=terraform init -upgrade -backend-config="XX"
stages:
- validate
- plan
- apply
validate:
stage: validate
before_script:
- *terraform-init
script:
- terraform -chdir=terraform validate
plan:
stage: plan
before_script:
- *terraform-init
script:
- terraform -chdir=terraform plan -out $PLAN
artifacts:
paths:
- terraform/$PLAN # Ensure correct path is specified
expire_in: 1 hour # Set the expiration time for the artifact
apply:
stage: apply
before_script:
- *terraform-init
script:
- terraform -chdir=terraform apply -input=false -lock=false -auto-approve $PLAN
dependencies:
- plan # Specify dependency to access artifacts from 'plan'
when: manual # Allow manual triggering of the apply stage
artifacts:
expire_in: 1 hour # Optionally set expiration for apply artifacts
Key Changes
artifacts in the plan stage:
Ensures that the tfplan file is saved and available for later stages.
The correct path (terraform/$PLAN) must match the location where terraform plan generates the file.
Added expire_in to limit how long artifacts are retained.
dependencies in the apply stage:
Ensures the apply stage retrieves the tfplan artifact from the plan stage.
Path to the tfplan file:
In the apply stage, ensure that the tfplan file is referenced in the same path as it was saved during the plan stage.
Debugging Tips
To ensure the file exists and is correctly passed:
Add a debugging step in the apply stage:
apply:
stage: apply
before_script:
- *terraform-init
- ls -la terraform # Check if tfplan exists in the expected location
script:
- terraform -chdir=terraform apply -input=false -lock=false -auto-approve $PLAN
dependencies:
- plan
when: manual
This will help verify whether the tfplan file exists and where it is located before applying it.
Final Notes
Artifacts Cleanup: Ensure that artifacts expire after a reasonable time to avoid storage issues in GitLab.
Manual Apply: Keeping apply as a manual stage is a good practice for production deployments to prevent accidental changes.
Path Consistency: Always double-check paths for the tfplan file in both plan and apply stages.