Locking, and What Happens Without It
The monthly bill has a NAT gateway nobody recognises, in a region the team does not use, that no state file mentions. It was created eleven months ago by an apply that ran at the same time as another apply.
The Problem at Scale
Two applies at once do not produce a merge conflict. There is no merge. State is read, modified and written whole, so concurrency produces a lost update, and the lost update has a specific and permanent consequence.
t0 A reads state, serial 1847
t1 B reads state, serial 1847
t2 A creates a NAT gateway, writes state, serial 1848
t3 B creates a subnet, writes state, serial 1848
B wrote over A's version. B's state has the subnet and
has never heard of the NAT gateway.
the NAT gateway exists
it is in no state file
nothing will ever plan it, change it, or destroy it
it bills forever
That is an orphaned resource, and it is the characteristic damage of running without a lock. It is not loud. Nothing errors, both applies report success, and the artifact of the incident is a line on a bill that somebody notices eleven months later.
Locking is per state file, not per resource and not per account. It prevents two operations from writing the same state, which is the common case and the one worth preventing. It does nothing about two different states that both manage the same real resource, which is the situation you create by importing something twice, and nothing about a person changing the resource in the console. Knowing exactly what the lock covers is what stops you trusting it for the cases it does not.
How It Works
What the lock is
An advisory lock held for the duration of an operation, acquired before reading state and released after the operation finishes. Both plan and apply take it by default, which surprises people who assume a read only operation would not need one, and -lock=false exists and is documented as dangerous.
Note what a plan does not do: it refreshes in memory to build the diff and does not persist that refresh. Writing the refreshed picture back is a separate operation, terraform apply -refresh-only, which is the subject of Module 3. So a plan holds the lock without changing the file, and an apply holds it and writes.
On S3, the modern mechanism is a lock object written next to the state using a conditional write, enabled with use_lockfile. The older approach, a DynamoDB table holding a lock item, is deprecated and will be removed, though both can be configured at once to migrate. A great deal of existing material still describes the DynamoDB table as a requirement, so it is worth checking what your backend actually does rather than what the guide you copied said.
When the lock is held, the failure is informative:
Error: Error acquiring the state lock
Lock Info:
ID: 4f2ac1e8-9b71-44de-a913-8c2f5b1d0e77
Path: org-tfstate-prod/platform/network/terraform.tfstate
Operation: OperationTypeApply
Who: runner@arc-runner-9f2b1
Version: 1.13.2
Created: 2026-09-06 11:04:22.117 +0000 UTC
Every field is useful. Who and Created together answer whether this is a live operation or a corpse.
Stale locks, and force-unlock
A lock outlives the process that took it whenever that process dies without releasing: a CI job cancelled, a runner evicted, a laptop that went to sleep, a network partition mid apply.
terraform force-unlock <ID> removes it. The question is when that is safe, and the answer is narrower than the way it gets used.
Safe when you have positively confirmed the operation is not running. The lock names who took it and when; go and look. A cancelled pipeline job whose run you can see finished is confirmation.
Unsafe when you are inferring from elapsed time. An apply that has been running for forty minutes may be creating a database. Force unlocking it and applying gives you two operations writing the same state, which is the lost update above, except now you caused it deliberately.
The tell is the motive: people force-unlock because they are impatient far more often than because they know. If your reason is that it has been a while, that is not a reason.
The better default is to wait rather than fail:
# Queue behind the current holder instead of erroring immediately.
terraform apply -lock-timeout=10mWhat the lock does not protect
Three gaps, and each has bitten somebody.
Two states managing one resource. Import the same object into two configurations and both will manage it, both hold their own lock, and neither is aware of the other. Each apply happily reverts the other's changes. Locking cannot help, because the states are different files.
Console changes. A person editing the resource directly is not taking any lock. That is drift, and it is Module 3.
A plan you approved and an apply that re-plans. This one is subtle and it matters for pipelines.
terraform plan you read it, you approve it
...someone else applies...
terraform apply re-plans from CURRENT state
and applies THAT
terraform apply without a saved plan file computes a fresh plan and applies it. What you reviewed is not necessarily what runs. Saving the plan closes the gap, because a saved plan records the state it was computed against and Terraform refuses to apply it if the state has moved on:
Error: Saved plan is stale
The given plan file can no longer be applied because the state was
changed by another operation after the plan was created.
That error is the system working. Module 8 builds the pipeline around it; the mechanism belongs here.
Building and Operating It
Turn locking on, and confirm which mechanism you are actually using.
terraform {
backend "s3" {
bucket = "org-tfstate-prod"
key = "platform/network/terraform.tfstate"
region = "eu-west-1"
# Native S3 locking. DynamoDB based locking is deprecated;
# both can be set at once while migrating.
use_lockfile = true
encrypt = true
}
}# Is a lock object actually being written? If this is empty during
# an apply, you are not locked, whatever the config says.
aws s3api list-objects-v2 --bucket org-tfstate-prod \
--prefix platform/network/terraform.tfstate.tflock \
--query 'Contents[].{key:Key,modified:LastModified}'Before force unlocking, look at who holds it.
# The lock names the operation, the holder and the time. Confirm the
# holder is dead before removing it; elapsed time is not confirmation.
gh run list --workflow=terraform-apply.yml --limit 5 \
--json databaseId,status,conclusion,createdAt
# Only then, and only with the ID from the error message.
terraform force-unlock 4f2ac1e8-9b71-44de-a913-8c2f5b1d0e77Detect the damage locking is meant to prevent.
# Resources in the account that no state file mentions. This is the
# orphan check, and it is the only way that eleven month old NAT
# gateway is ever found.
aws resourcegroupstaggingapi get-resources --region eu-west-1 \
--query 'ResourceTagMappingList[].ResourceARN' --output text | tr '\t' '\n' | sort > /tmp/real.txt
for key in $(aws s3api list-objects-v2 --bucket org-tfstate-prod \
--query 'Contents[].Key' --output text); do
aws s3 cp "s3://org-tfstate-prod/$key" - 2>/dev/null \
| jq -r '.resources[].instances[].attributes.arn // empty'
done | sort -u > /tmp/managed.txt
comm -23 /tmp/real.txt /tmp/managed.txt # unmanaged, and billingAnd watch the serial, which tells you about writes you did not make.
# The serial only moves when somebody writes. If it advances between
# your pipeline runs, something applied outside the pipeline.
terraform state pull | jq '{serial, lineage}'A team ran Terraform from laptops with an S3 backend and no locking, because the guide they had followed treated the DynamoDB table as an optional extra step. It worked for two years. The orphans were found during a cost review: a NAT gateway, three EBS volumes and a load balancer, roughly $9,000 of cumulative spend, none of them in any state file and none of them attributable to a change anybody remembered making. The applies that created them had all reported success. Enabling locking took one line and the cleanup took two weeks, most of which was spent establishing that the resources really were unmanaged rather than belonging to a state somebody had forgotten about.
Tradeoffs and Decision Framework
| Situation | Response |
|---|---|
| Lock held by a running operation | Wait. Use -lock-timeout rather than failing |
| Lock held by a confirmed dead process | force-unlock with the ID from the error |
| Lock held and you are unsure | Find the holder. Elapsed time is not evidence |
| Two states managing one resource | Locking cannot help. Remove it from one |
| Someone changed it in the console | Drift, Module 3, not a locking problem |
| Plan reviewed, apply re-planned | Save the plan file and apply that |
Three questions about any estate you inherit. Is a lock object actually being written, which is a different question from whether the config mentions locking. Are there orphans, since that is the accumulated evidence of running without one. And does apply run a saved plan, because otherwise the thing reviewed and the thing applied are not the same thing.
Default: locking enabled and verified by observing the lock object, -lock-timeout rather than immediate failure, force-unlock only after positively identifying a dead holder, and applies that run a saved plan file.
Failure Modes and Common Mistakes
No locking at all. The damage is silent orphans rather than an error, so nothing tells you for months.
Assuming a plan does not need the lock. It takes one by default, and -lock=false is documented as dangerous.
Force unlocking out of impatience. You have deliberately created the concurrent apply the lock existed to prevent.
Treating elapsed time as evidence. A long apply is often a database being created.
Trusting the lock across states. It is per file; two states managing one resource are unprotected.
Applying without a saved plan. What was reviewed and what runs are computed at different moments.
Never checking for orphans. They are the only evidence that locking was missing, and they do not announce themselves.
Two engineers run terraform apply against the same state file at the same time, with no locking configured. Both applies report success. What is the most likely lasting consequence?
What happens if two applies run against the same state at once, and what exactly does the lock protect?