State internals, refresh and drift

Terraform · Expert · 12 min

When something goes wrong with state, you need to know exactly what Terraform stores and how it reconciles.

The state file is JSON with a version (format), terraform_version, a serial (incremented on every write) and a lineage (a UUID fixed when the state was created). Backends refuse to overwrite a state with a different lineage or an older serial, which protects against pushing a stale or foreign copy. Each resource instance stores its attributes, schema version, dependencies and sensitive-attribute markers.

Drift is a difference between state and reality caused by changes outside Terraform. A normal plan refreshes first and reports "Objects have changed outside of Terraform" before proposing to put things back according to your configuration.

  • terraform plan -refresh-only / apply -refresh-only accepts reality into state without changing infrastructure; use it to acknowledge legitimate external changes, then update the code to match.
  • -refresh=false skips reading real objects (faster, riskier), useful in huge states or outages.

State surgery commands, all of which take the lock and write a backup:

  • terraform state rm forgets a resource (prefer a removed block).
  • terraform state mv renames (prefer moved).
  • terraform state pull > backup.tfstate / state push for disaster recovery; push checks lineage and serial unless forced.

Recovering from a corrupted remote state usually means restoring an earlier object version from the bucket's versioning. That's why versioning is non-negotiable.

Example

terraform state pull | jq '{version, serial, lineage, terraform_version,
  resources: [.resources[] | "\(.type).\(.name)"]}'
# { "version": 4, "serial": 42, "lineage": "8f0c2c1e-...",
#   "terraform_version": "1.14.0", "resources": ["aws_vpc.main", ...] }

terraform plan
# Note: Objects have changed outside of Terraform
#   ~ tags = { + "owner" = "ana" }    (added in the console)

terraform apply -refresh-only      # accept the drift into state
terraform state pull > backup-$(date +%F).tfstate

Remember

  • serial increments per write; lineage identifies one state's history
  • -refresh-only accepts outside changes without touching infrastructure
  • Back up with state pull; recover via bucket versioning

Watch out for

  • Using state push -force with an old copy, overwriting newer state and orphaning resources.

Check yourself

  1. Someone changed a tag in the console and the team agrees it should stay. What is the cleanest sequence?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: developer.hashicorp.com/terraform/cli/commands/plan#planning-modes