v0.1.26 (Early Access)
This one is mostly about alerts that were lying to you, and a load model that punished you for spikes your hardware could easily absorb.
Alerts
- Saturation alerts now name the thing that is actually full. Every chokepoint between your racks and the carrier was being collapsed into one number, so a struggling firewall reported itself as "uplinks saturated, facility-wide" while the internet page showed your lines at a fifth of capacity. Lines, routers and security boxes now each report against their own capacity and name themselves, and security boxes distinguish between forwarding limits and inspection limits. One alert per bottleneck, so it no longer flaps when a different one takes over.
- Individual cables can now report saturation. A link can be pegged while the switch and router at either end have headroom to spare, because the cable, or the slower of the two ports, is the limit. A 100G port on a cable that negotiated down to 10G is doing 10G. The alert names the cable and the network map highlights it.
- Saturation alerts no longer blame customers whose traffic never touched the bottleneck. The affected list was built from "who is slow right now", so a customer short on CPU two rooms away counted as a victim. That also inflated severity: one unrelated customer on a redundant contract pushed a minor link alert to critical. The list is now traced through your actual wiring, and severity follows it.
Racks
- Unracking a router or an internet box frees the U it was sitting in. The slot stayed claimed, and nothing, including the router itself, would go back into it. Servers, switches and patch panels were never affected, which is what made it look like one bad model. Existing saves repair the stuck slots on load.
The network map
- Switch port counts no longer include the power inlets on the back. A 32-port switch read as 34, a 48-port as 53.
- Switch and server speeds read from the links you have actually run, not from whatever happened to be in port one. Two identical switches now report the same speed. Mixed speeds get a "+" after the number, and an unpatched switch says so instead of inventing a figure. The fabric figure beside it was always correct and is unchanged.
Load and performance
- A traffic spike on a server with spare capacity is no longer punished. Load was measured against what a customer had bought rather than what the machine had free, so a spike drew errors and near-two-second response times on a host running at 5% with a hundred idle cores, and buying more changed nothing. Worse, the burst grant was calculated as exactly the shortfall, so a served customer was still recorded at precisely 100% and charged the full penalty. Load is now measured against the CPU genuinely available on the host.
- Spikes grow into whatever the server is not using. The old ceiling of twice contract had no reasoning behind it and customers routinely spike three to eight times. The only limit now is physical.
- Response times scale with how bad the overload is, capped at the request timeout. Every level of overload used to report the same figure, so the number could not tell you whether the hardware you just bought had helped. When the cause is other customers crowding the same machine, the delay is attributed to that and the alert names the machine.
- Burst capacity above contract is now billed at the normal rate. It had been free since the feature shipped, and it was invisible because the invoice matched the contract exactly.
- Scale-up requests now trigger on a customer consistently needing more than they bought. They used to trigger on the customer failing to keep up, which stopped happening once spikes were absorbed properly.
- Existing saves adopt the new load model on load. If you changed either setting yourself, your choice is left alone.
Kubernetes
- When a cluster cannot place all its worker nodes, the game now tells you what actually stopped it. The refusal reported a fixed story about processor and memory no matter what the real cause was, so a cluster blocked on needing separate failure domains told you it was short of compute while most of the pool sat idle. Buying more servers then changed nothing, because servers were never what it was short of. The message now names the constraint the placer actually hit. This does not by itself place a node that could not be placed before; it stops the game sending you after the wrong hardware.
- A cluster no longer takes its owner's other clusters down with it. Re-laying one cluster released every cluster the customer held and rebuilt only one.
Load balancers
- Load balancers are real machines now. A proxy was charged to your fleet as a small management overhead rather than as software running on a server, so a load balancer cost roughly nothing to host no matter how much traffic it carried. Proxies are now sized from the request rate they serve, reserve processor and memory on the machine they sit on like any other workload, and are packed against the rest of your fleet accordingly. A busy load balancer is a real tenant on a real box. Expect load-balancer customers to take up more room than they used to.
Storage
- How many copies of a customer's data you keep is now decided by what that customer bought, and nothing else. There was a fleet-wide replication setting that could raise a tenant above their tier, so a customer who paid for one copy could silently be given three, at triple the storage, without anything on their contract saying so. It only ever affected your cheapest tenants, since anyone on a higher tier was already above it. The setting is gone.
- The storage row on a customer's page states copies, not liveness. A bucket contracted for three copies and running on one reported "1 of 1 live, healthy" while the alert about the same bucket said one of three. Both numbers were real; they were answering different questions and only one of them was the question you were asking. The page and the alert now quote the same pair.
- A content delivery network short of its points of presence says so. One live location out of four reported as one of one.
Databases
- A database that lost its primary machine no longer reads as failing over forever. The failover clock started on any primary loss, but only stopped on the path that requires a live replica and automatic failover switched on. Every other case left the database marked as mid-failover permanently, including after the machine came back, and it was billed as degraded for as long as that lasted.
- A database signed while its service was still locked now gets its automatic backup. The backup schedule was only ever started by the signing path, behind a gate that such a customer never passed, so the database ran without one indefinitely.
- The repair that moves work off machines with the wrong accelerator now sees managed databases. It only looked at plain virtual machines, so a database pinned to a particular accelerator was skipped.
Diagnose
- The fault-isolation row says which workload it means. It is computed from your virtual machines and its fix is to rebalance them, but it said "copies", which is the word the database rows use. A customer whose database was short a replica saw "database has no live replica" and "copies are stacked, migrate them apart" side by side, which reads as a contradiction and sends you to perform a move that cannot exist when only one copy is there.
- Kubernetes, content delivery and load balancers can report a compromised spread at all. Only virtual machines and databases were ever measured, so a cluster with every worker stacked on one machine reported itself perfectly healthy. A single machine failure would have taken the whole cluster and nothing said so.
- Where a service is allowed to spread is now visible on the customer page, and you can set it. It existed as a contract term the game held you to and never showed you. Choosing one tells you immediately what it will cost in copies you cannot separate, rather than leaving you to discover it later.
- The serverless row counts containers that are resident, not requests in flight. It blinked in and out between requests while the container held its memory the whole time.
- A customer failing none of its requests is no longer flagged as failing some. The check warned on any rate above zero while printing it to one decimal place, producing "0.0% of requests are failing". The rate is a rolling average on purpose, so a customer recovers in minutes instead of being pinned to a lifetime figure, but it decays towards zero without arriving: one bad moment kept a customer flagged for half an hour, and a customer whose traffic stopped stayed flagged forever. It now settles to zero below anything the display can show.
- That is also why your site checklist could claim "40 of 70 tenants need attention" while every tenant looked fine. Warnings count toward that tally, so each decaying average added a tenant. The incident evidence panel had the same line and is fixed alongside it.