Sometimes you pay for idle, on purpose

Autoscaling sounds like the end of a conversation. You set a policy, the platform adds capacity when load climbs and removes it when load drops, and you go home feeling like you solved scaling. For the steady state, it works. For the cliff, it quietly lies to you.

I learned that running a live-streaming platform through the kind of event that happens once a year and is not allowed to fail. The traffic didn’t show up as a gentle ramp. It showed up as a wall. A partner would shift their whole audience to us in bursts that looked less like a busy afternoon and more like a failover event, and the sessions were short enough that they were gone before any autoscaler noticed they had arrived. Every test we ran, they promised one number and delivered several times more, on purpose, to see where we tipped over.

So we did the unfashionable thing: we pre-scaled to the full expected peak days ahead, and we paid for all that capacity to sit there idle, waiting for an audience that would arrive in minutes and leave in hours.

Why idle was the cheap option

The instinct is to call that wasteful. It isn’t, once you do the arithmetic on the alternative. The cost of running at peak for a few extra days is a rounding error. The cost of falling over in front of the entire audience, on the one night everyone is watching, is the kind of number that shows up in contract penalties and in the next quarter’s churn. When the downside is that asymmetric, “wasteful” capacity is just insurance with a clear premium.

Pre-scaling also bought us something autoscaling never could: certainty. We weren’t hoping the scale-up would keep pace with a vertical traffic curve. The capacity was already there, already warm, already proven by the time the event started. The night of, the most exciting thing that happened was nothing.

The actual lesson

The mistake isn’t choosing autoscaling. The mistake is not knowing which regime you’re in. Most of your system lives in the steady state, where autoscaling is exactly right and pre-scaling everything would be genuinely wasteful. But every system has a few moments that are not the steady state, where the traffic is a step function and the cost of being a few minutes late is total. For those, you pre-scale, you eat the idle, and you sleep.

Reach for autoscaling by default. Reach for pre-scaling when the curve is a cliff and failure isn’t an option you can buy your way out of after the fact. Knowing the difference is most of the job.