UptimeUptime Wiki

Control-plane resilience (HA, multi-AZ)

If a service has a single brain and the server it lives on dies, the whole service loses its coordinator at once -- not just one customer, all of them. That is a single point of failure. It is the cheapest way to run a service and the riskiest, and it is fine right up until the day it is not.

A resilient service runs two or three brains instead of one, ideally in different availability zones so a whole-room failure cannot take them all. If one goes down the others carry on and the service stays up. It costs more -- you are paying to run the brain several times over -- but it buys you the ability to survive a server or even a zone loss without breaking your promises. The bigger and more demanding your customers, the more this stops being optional.

How badly a brain outage hurts -- and how soon you are pushed toward running spares -- tracks the difficulty you chose. On gentler settings a single brain is forgiven for longer; on harder settings an outage bites hard and your most demanding tenants may flat-out require a resilient setup before they will sign.

Detailed explanation

Replicas as the resilience lever

control_plane_replicas: 1 = SPOF, 2-3 = HA (ideally multi-AZ). More replicas cost proportionally more resources but survive a host or AZ loss. Placement matters: co-locating all replicas in one AZ defeats the point. This is the late-game depth layered on top of the SPOF + overload teaching beats.

Difficulty + SLA hooks

The difficulty system scales control-plane outage severity and the rate at which need grows, pushing harder runs toward HA; easier runs forgive a solo control plane. A scenario or SLA-heavy tenant can require a minimum replica count as a hard gate -- the same way compliance certs gate other archetypes.

On this page