Rolling Deployments
Replace instances of the old version with the new one gradually, a few at a time, so there's never full downtime. The default strategy for many systems, with subtle failure modes.
The Concept Explained
Forty instances sit behind a load balancer, all running version N-1. A build of version N is ready and needs to reach all forty of them.
The direct approach is to stop all forty, replace the binary, and start them again. That is the recreate strategy, and it costs an outage window equal to however long forty instances take to start and warm. Acceptable for a nightly batch job, and not for anything with users attached to it.
Follow the alternative. Take four instances out of the load balancer, replace them with version N, put them back, and repeat nine more times. At no point are fewer than thirty-six instances serving traffic. There is no window in which the service is down, only a window in which it runs at slightly reduced capacity while part of it is being replaced.
That is a rolling deployment, and it is the default in nearly every orchestrator because it requires nothing you do not already have. No second environment, no traffic-splitting proxy, no service mesh. If you can add and remove instances from a load balancer, you can roll.
The cost appears in the middle of the rollout. At batch five, twenty instances run N-1 and twenty run N, behind the same load balancer, writing to the same database, calling the same peers. A user's first request lands on N and the retry lands on N-1. A worker on the old version picks up a message written by the new one. This is not an edge case you can engineer away; it is the strategy.
And because moving forward is incremental, moving back is incremental too. There is no switch to flip. A rollback is a second rolling deployment in the opposite direction, and it takes roughly as long as the first one did.
A rolling deployment buys continuous availability by giving up version uniformity. For the whole duration of the rollout, N-1 and N are both live and both talking to the same datastore and the same peers, which means every release must be backward compatible with the one immediately before it. That compatibility requirement, not the mechanics of swapping instances, is the real engineering work. It is also why recovery time equals rollout time: undoing the change is just another pass over the fleet.
How It Works
One Batch Through the Rolling Deployment Cycle
Click each step to explore
The loop looks symmetric, but its cost is not. Almost all wall-clock time sits in starting instances and waiting for readiness, and that cost is paid once per batch. Ten batches at ninety seconds each is a fifteen-minute rollout no matter how fast the pipeline is, and the same fifteen minutes is what a rollback costs when you need it most.
The Two Knobs: Max Surge and Max Unavailable
Every rolling deployment is configured by two numbers, expressed as counts or percentages of the target replica count. Max surge is how many instances above the target may exist at once; max unavailable is how far below the target the serving fleet may drop.
Surge is bought with headroom. Forty instances at 25% max surge means the platform must be able to place fifty, and that is real quota, node capacity, IP addresses, and database connections. Surge never reduces serving capacity, but if the headroom is not there the rollout stalls partway with instances unplaceable and the fleet frozen in a mixed state.
Unavailable is bought with capacity margin. Setting it to 25% means peak traffic is served on thirty instances instead of forty, during the exact window in which unproven code is being introduced. A fleet already at seventy percent utilization has no room for that, and the release becomes a self-inflicted load test.
Setting both to zero means nothing can move. The safe default is max unavailable at zero with a positive surge, accepting the obligation to hold headroom in exchange.
Readiness Is Not Liveness
The rollout advances on a signal, and which signal you choose decides whether the mechanism protects you or hands the whole fleet to a broken build.
A liveness signal answers whether the process is alive. Failing it kills and restarts the instance, which is the right response to a deadlock or a wedged event loop.
A readiness signal answers whether this instance can serve a real request correctly right now. Failing it removes the instance from the load balancer without killing it, which is the right response to a dependency that has not connected yet.
Rollouts gate on readiness. A readiness check that succeeds the moment the HTTP server binds its port proves only that the process started. It does not prove the config parsed, the database pool connected, the caches loaded, or the downstream credentials are valid.
A readiness check that only proves the process is up is the single most common cause of a rollout that carries a broken version to the entire fleet. Every batch reports ready within seconds, every gate passes, the rollout finishes faster than usual, and you are left with no instances capable of serving a request while the tooling reports success. A readiness check must exercise the dependencies the instance actually needs, or it is worse than no check at all, because it manufactures confidence.
Removing an old instance is likewise two ordered operations: the load balancer stops sending it new requests, and only then does it shut down, after finishing what it holds. Deregistration propagates slowly, so the instance must keep serving for seconds after being told to stop.
Production Implications
Every release must be compatible with the one before it. N-1 and N run side by side by design, so a change that only works once the whole fleet has it breaks during the window when half the fleet does not. The real work lands in API and message-format compatibility, and in schema changes, which get their own treatment later.
Batch size sets both blast radius and duration. Small batches limit how much of the fleet a bad version reaches before the gate catches it, at the cost of a longer rollout and a longer mixed-version window. Large batches finish sooner and expose more users faster.
Rollback is a second rolling deployment. Recovery time equals rollout time, and nothing within the strategy changes that. It is rolling's core weakness and the reason blue-green, with its instant cutover, and canary, with its early abort, both exist.
Health gates need a soak window, not an instant. A batch that reports ready and fails two minutes later still passes an instantaneous check, and the rollout carries the failure forward into the next batch.
Put a deadline on the rollout. A rollout that cannot place instances or cannot pass readiness will otherwise sit half-migrated indefinitely. A progress deadline converts a silent stall into a failure someone gets paged about, and a halted rollout leaves both versions running until a human decides.
Draining does not cover long-lived connections. WebSocket sessions, streaming RPCs, and sticky sessions outlive any reasonable drain timeout and need their own client-side reconnect handling.
Lead with the capacity mechanism, since that is what the question literally asks: instances are replaced in batches, and the surge and unavailable settings keep enough of the fleet serving that there is never a full outage. Then name readiness as the gate on the next batch, and say explicitly that a weak readiness check is what lets a broken version reach the whole fleet. Most candidates stop there. The two additions that mark a senior answer are the mixed-version window, meaning both versions share a datastore and every change must be backward compatible with its predecessor, and the fact that rollback is itself a rolling deployment, so recovery is as slow as the release was.
Tradeoffs and Decision Framework
| Dimension | Max surge (add capacity) | Max unavailable (drop capacity) |
|---|---|---|
| Serving capacity during rollout | Never falls below target | Falls by the configured amount |
| What it costs | Headroom: quota, nodes, connections | Margin: your buffer during the riskiest window |
| Money | Extra instances for the rollout duration | Nothing extra |
| Peak-hour rollouts | Safe | Risky, effectively a load test |
| Fixed-size pools or hard quota | Cannot be used | The only option available |
| Failure mode | Rollout stalls with instances unplaceable | Brownout or saturation under load |
| Rollout speed | Limited by instance startup time | Limited by drain and startup time |
The framework in four questions. Do you have real headroom to surge into, since without it max unavailable is not a preference but the only option? What is your utilization at peak, which decides whether dropping instances mid-release is survivable? How long does one instance take to become genuinely ready, since that number times your batch count is both your rollout time and your rollback time? And is the change backward compatible with the version it replaces, because if it is not, no batch size makes rolling safe.
Default to rolling with max unavailable at zero and a modest surge. Leave it when rollback speed matters more than infrastructure cost, which points at blue-green, or when the change is risky enough to want evidence from real traffic first, which points at canary.
Common Mistakes
A readiness check that only proves the process started. The gate passes for every batch, the rollout completes, and the entire fleet is broken while the tooling reports success.
No connection draining. In-flight requests are killed on every batch, producing a burst of errors that gets misattributed to the new version.
Assuming a uniform fleet. Code that expects every instance to be on the same version fails during the mixed window, which every rolling deployment has by construction.
Treating rollback as instant. Recovery takes as long as the rollout did, so an incident starting at batch eight still needs a full pass over the fleet to end.
Shipping a breaking API or schema change in a single release. Both versions are live against the same datastore, so a change that is not backward compatible corrupts data or fails requests during the window.
Max unavailable set high with no capacity margin. The service brownouts under peak load and the new version takes the blame for a capacity planning error.
No progress deadline. A stalled rollout sits half-migrated indefinitely with nobody paged, because nothing has technically failed.
Health evaluation with no soak window. A version that crashes shortly after startup passes an instantaneous check and is carried forward batch after batch.
How does a rolling deployment maintain availability during a release? What's the risk of running two versions simultaneously, and how do you roll back if something goes wrong midway?