Blue-Green Deployments
Run two identical environments, blue (current) and green (new). Switch traffic all at once, and switch back instantly if something breaks. Fast rollback, at the cost of double infrastructure.
The Concept Explained
A rolling deployment is halfway through a twenty-instance fleet when error rates start climbing. Eight instances are on the new version, twelve on the old, and reversing means running the same slow, health-gated process backward while the broken version keeps serving live traffic.
The problem is structural. The release mechanism and the rollback mechanism are the same mechanism, so rollback is never faster than the deploy that caused it. A twenty-minute rollout implies a twenty-minute undo, spent with a version you already know is bad still in the serving path.
Blue-green removes that coupling by refusing to mutate the running fleet at all. Blue takes production traffic. Green is a second, complete environment on the new version, with its own instances, its own configuration and its own connections to every dependency, at full production capacity and receiving nothing. You deploy to green, smoke test it, and let it sit there costing money.
Then you change one thing. A router in front of both stops sending traffic to blue and starts sending it to green. Every user moves at once. Blue is left running, untouched, still on the previous version, so if green misbehaves you point the router back. The deploy and the rollback are now the same single action, completing in the time a routing change takes.
The cost arrives in two forms. The first is money, since for the length of the overlap you fund two production environments. The second matters more. Traffic moves as a step function, so at the instant of cutover every user is on the new version. A fault that a rolling deployment would have shown to five percent of traffic while a health gate was still deciding is here shown to everyone, for however long it takes an alert or a human to notice.
Blue-green optimizes for how fast you can undo a bad release, not for how few users see it, so it makes a release fast to reverse but not safe to try. Real recovery time is detection time plus the routing change. And the undo covers only the stateless part: blue and green share one datastore, so once green has written data in a shape blue cannot read, flipping back is not a rollback, it is a second uncontrolled migration.
How It Works
Blue-Green Against Rolling, Axis by Axis
Blue-green
Two complete environments, one atomic switch
Rolling
One fleet mutated in place, batch by batch
The axes read like a straight trade, but these are not two points on one scale. Rolling gives you a detector and a slow undo; blue-green gives you a fast undo and no detector at all, since the cutover discovers nothing and only makes correcting cheap. The useful question before a flip is therefore not how fast you can reverse, but how quickly you would know that you need to.
The Switch Is Not One Mechanism
"Flip the router" hides three implementations with very different behavior.
DNS is the slowest and least trustworthy. A TTL is a request, not a contract: resolvers hold entries longer than they should and client runtimes cache lookups for the life of a process. The cutover becomes a long tail, and the reversal inherits it, destroying the property you adopted blue-green to get.
A load balancer target swap is the usual answer. Repoint the listener at green's backend pool and new connections land there within seconds, deterministically, with no client-side cache in the path.
A service mesh or gateway rule is the most precise, since routing is evaluated per request. It also blurs the line usefully: a control plane that can move traffic from 0 to 100 can hold it at 1 percent first.
A green environment that passed every smoke test can still fall over the moment it takes real traffic, because smoke tests warm nothing: caches empty, pools unopened, lazily initialized clients never touched. The trap is what comes next: while green sat idle blue was serving, and after several minutes of a failed cutover blue's own caches and pools have gone cold too, so flipping back does not return the performance you had.
The Datastore Is Shared
Almost nobody duplicates the database. Blue and green point at the same one, so the second environment copies the stateless half of production and nothing else.
Two consequences follow. The schema must satisfy both versions simultaneously, for the whole period blue is retained as a rollback target, so a migration that drops or renames a column at cutover removes your ability to go back. And green writes the moment traffic arrives. If those writes use a new shape, blue reads them after a reversal and behaves unpredictably, making the reversal an unplanned forward migration under incident conditions rather than a rollback.
The pattern that makes schema change compatible with this constraint is expand-contract, which gets its own treatment later. What matters here is simpler: blue-green gives you a code rollback, never a data rollback.
Production Implications
Identical means identical, and that is expensive. Not just double compute: a second copy of configuration, secrets, TLS material, feature flag state, scheduled jobs and every downstream connection, each one a place the two can silently diverge.
Warm green before the flip, not with it. Send synthetic traffic, prime caches and open pools, so the cutover is the first real traffic rather than the first traffic of any kind.
Define the bake period before you start. Name the interval and the metrics that must hold across it, then reclaim blue deliberately. Reclaiming early gives up the rollback; never reclaiming makes a temporary overlap permanent.
Decide the in-flight request policy explicitly. Blue keeps existing connections after the flip and needs a drain window long enough for them to finish. Long-lived connections such as websockets need their own answer; they will not drain on a normal timescale.
Externalize session state first. Sessions in instance memory do not survive a cutover; users are logged out at the flip. This is a prerequisite, not a tuning detail.
Background work is part of the cutover. Queue consumers and schedulers in green run alongside blue's unless you stop one side, which is how a cutover produces duplicated jobs or double-charged records.
Agree the rollback trigger in advance, and rehearse it. Fast rollback is worthless if the decision takes twenty minutes of debate, so fix the metrics, thresholds and the person who calls it.
Answer in this order: two full environments, an atomic traffic switch, and a rollback that is one routing change rather than a second rollout. Then volunteer the cost honestly: doubled infrastructure, and a flip that exposes every user at once, so blue-green is fast to undo but not safe to try. What most candidates miss is the database: blue and green share it, the schema must satisfy both versions for as long as you keep blue alive, and once green has written data in a new shape the flip back is a forward migration rather than a rollback. Naming expand-contract as the resolving pattern separates a rehearsed answer from an operated one.
Tradeoffs and Decision Framework
| Dimension | Blue-green | Rolling |
|---|---|---|
| Rollback | One routing change | A reverse rollout, as slow as the deploy |
| Extra capacity | A second full environment | One batch of surge |
| Exposure to a bad release | Everyone, immediately | A fraction, gated by health checks |
| Mixed versions serving | None at the request layer | The whole rollout window |
| Session and connection impact | Concentrated at the flip | Spread across the rollout |
| Schema coupling | Both versions plus the reversal path | Both versions during the rollout |
| Operational complexity | Environment parity and cutover mechanics | Batch sizing and health gates |
The framework in four questions. How long would it take you to detect a bad release, since blast radius is that number multiplied by full traffic rather than a fraction of it? Can you fund a second complete environment for a defined window, dependencies included? Is this change stateless, or does it write data the previous version cannot read? And what is the switching mechanism, because a DNS-based cutover does not deliver the property the strategy is chosen for.
Default to rolling for routine changes and reach for blue-green when reversal speed is the binding constraint: a risky release, a tight change window, a system where minutes of a known-bad version are unacceptable. When exposure at cutover worries you, put a canary in front of the flip rather than abandoning the topology.
Common Mistakes
Treating fast rollback as low risk. The flip moves every user at once, so a bad release is total until someone notices. Reversal speed does not substitute for a detection strategy.
Assuming the database rolls back too. Blue and green share one datastore, and writes green made in a new shape survive the reversal.
Destructive migrations at cutover. Dropping or renaming a column while blue is still your rollback target removes the rollback.
Using DNS as the switch. TTLs are advisory and clients cache aggressively, so both the cutover and the reversal become long tails you cannot bound.
Flipping into a cold environment. Empty caches and unopened pools mean green fails under real load in ways it never failed under smoke tests.
Reclaiming blue immediately. The rollback exists only while blue runs, so tearing it down at cutover buys back the cost and gives away the benefit.
Environment drift. A green built from different config, secrets or dependency versions is not the same system, and the difference surfaces only after the flip.
Ignoring background workers. Consumers and schedulers running in both environments cause duplicate processing that routing control cannot prevent.
Explain blue-green deployment. What's the main advantage over rolling deployments, what does it cost, and how do you handle database schema changes during the cutover?