Cloud capacity isn't "infinite" anymore. Let's look at what's happening to compute capacity in 2026, why classical capacity planning is back, and why forecasts, CSP cooperation and some old assumptions need a serious revisit.

This week's The Pragmatic Engineer newsletter covered a subject particularly close to my day job: compute capacity.

I work as an SRE in a Platform Capacity team at Elastic, so instance types, reservations, quotas, forecasts and OutOfCapacity errors are things I spend quite a bit of time thinking about. And capacity has become a surprisingly interesting problem in the AI era.

For a long time, one of the great promises of public cloud was that infrastructure capacity was somebody else's problem - or, in other words, that it was practically infinite. Well... sort of. But those days are over. At significant scale, capacity is very much a real engineering constraint. And this isn't only about GPUs anymore.

So this post explores various vectors for approaching capacity problems. There's no single silver bullet (there never was, lol), so you'll need to bear with me. There's quite a lot to cover, so instead of turning this into one enormous post, I'm splitting it into three digestible parts:

1. Part 1 - Capacity planning is back
   Why compute capacity is becoming constrained, what happened to the assumption of infinite cloud resources, forecasting, working with CSPs, and revisiting business assumptions.
2. Part 2 - Designing for capacity flexibility
   Making capacity part of application design, rightsizing, cooperation between developers and Capacity teams, instance-type fallbacks, and designing for OOC failures.
3. Part 3 - Capacity optimization and automation
   Bin-packing, containers and cgroups, Kubernetes scheduler configuration, quotas, reservations, shared capacity, and the automation needed to manage all of it.

Just to make one thing very clear: everything in this post is based on experience gathered over years of working in the industry, discussions and work with some great engineers, and publicly available information - I'm not sharing any internal details about Elastic. We're a publicly traded company, and there's a line between sharing engineering experience and sharing things I shouldn't.

What I can share are ideas, patterns and lessons learned along the way that can help when designing systems for constrained capacity and handling OOC (Out of Capacity).

Let's start with the least technical part — and probably the most important one: capacity planning.

Ah, and yes - general-purpose compute is becoming constrained too 👻

CPUs are getting harder to get

The article that triggered this post describes engineering organizations increasingly struggling to source ordinary CPU capacity - not only GPUs. Some are seeing longer lead times, difficulties obtaining particular instance types, and much less certainty that capacity will simply appear when an autoscaler asks for it.

This makes sense when looking at what's happening underneath the cloud abstraction. AI infrastructure obviously consumes enormous quantities of GPUs, but GPUs don't operate in isolation. AI agents search, execute programs, operate browsers, call tools, run containers and process data - all of that requires ordinary CPUs as well.

At the same time, the problem doesn't stop at semiconductor manufacturing. A CSP having money to buy another 50,000 servers doesn't mean it can necessarily install those servers in the region where you need them. Datacenters require buildings, networking, cooling and - most importantly - power. In parts of Europe, available grid capacity and the ability to obtain new grid connections are already constraints on datacenter expansion.

So when we talk about "cloud capacity", we're really talking about several interconnected supply chains. Eventually, a constraint somewhere down that chain may surface several abstractions higher as something beautifully concise:

Warning  FailedScaleUp  cluster-autoscaler
scale.up.error.out.of.resources

That's the cloud politely telling the Kubernetes autoscaler: I'd love to add another node, but I don't have one.

Where did all the Spot instances go?

Spot capacity is an especially interesting indicator. The basic idea behind Spot instances is simple: cloud providers have unused machines, so rather than leaving them idle, they sell this spare capacity at a significant discount - with the obvious trade-off that they may reclaim it.

But what happens when there isn't much spare capacity anymore? Spot isn't dead, of course, and availability differs enormously between instance families, availability zones and regions. Still, relying on Spot capacity can be considerably harder for some workloads than it used to be. The same industry reports that describe CPU shortages also describe Spot capacity becoming difficult to obtain for particular shapes and locations.

And that matters beyond infrastructure engineering. If the business case for a service assumes:

"We'll just run most of this on cheap Spot instances."

then Spot availability looks like part of a business model.

Cloud capacity isn't binary

There's another important distinction. Saying:

GCP, AWS or Azure has plenty of CPUs.

doesn't necessarily help. One might need:

• a particular CPU architecture or generation;
• a certain CPU-to-memory ratio;
• local SSDs;
• specific storage or network performance;
• a particular region;
• perhaps even a particular zone;
• and several thousand of those machines rather than three.

Capacity therefore isn't simply:

capacity = available / unavailable

It's closer to:

capacity = instance characteristics × region x zone × quantity × time

At sufficient scale, that distinction matters a lot. Sounds serious? It is. Which is also why I think Capacity is currently one of the most interesting areas of infrastructure engineering to work in. Or, depending on how much you value peace of mind, possibly one of the worst 😄

Capacity planning is back

There's something slightly amusing about all this. Two books that are particularly relevant here are John Allspaw's The Art of Capacity Planning and Neil J. Gunther's Guerrilla Capacity Planning. Neither was written for the Kubernetes-and-AI world. Gunther's book dates back to 2007, and a lot has changed since then. But the underlying problem hasn't.

For a while, public cloud made classical capacity planning look almost unnecessary:

demand increases -> autoscaler -> cloud API -> more servers

But there's a rather important hidden assumption there:

cloud API -> capacity actually exists

An autoscaler allocates capacity. It doesn't create it! And that means some rather old-school capacity-engineering principles are becoming useful again. Measure. Understand. Model. Forecast. Compare the forecast with reality. Adjust the model. Repeat.

One of the useful lessons from classical capacity planning is that a forecast isn't supposed to be prophecy. Its job is to give you enough information, early enough, to make a decision. Gunther's "guerrilla" approach is particularly interesting here because it emphasizes getting useful capacity answers within relatively short planning horizons rather than building enormous forecasting exercises. Do you know the GCAP wheel? Well, this is how it could look like these years:

That feels imo appropriate for cloud infrastructure in 2026. More like a control-loop instead of a spreadsheet.

Btw - in some companies, capacity planning was never abandoned ;-)

Forecasting what services will actually need

Useful capacity planning is not a linear function:

We have 10,000 CPUs. We're growing 20%. Therefore we'll need 12,000 CPUs.

Ideally, you want to understand the relationship between business demand and infrastructure demand:

The exact input depends on the service. It might be customers, requests per second, stored data, shards, jobs, events or something entirely different. The important part is understanding what drives resource consumption. Then forecast it.

And don't create only one number. Depending on the service, it can be useful to model expected growth, faster-than-expected growth, major launches or migrations, and the additional capacity needed to survive failures. Most importantly, compare the forecast with reality.

If the model predicted 15% growth and reality was 24%, the interesting question isn't merely that the forecast was wrong, but why was it wrong? Maybe traffic grew faster. Maybe a software release changed CPU efficiency. Maybe the average customer became larger. Maybe a new workload appeared.

Understand it, update the model and forecast again - that's the loop.

Lead time changes everything

There's another variable that becomes increasingly important when capacity isn't instantly available: lead time.

If you need capacity in December and obtaining it can take months, discovering the requirement in November doesn't help much. Capacity planning therefore needs to work backwards:

This sounds obvious, but becomes considerably less obvious when you're doing it across many services, regions, projects/accounts and machine families. And this is exactly where capacity planning stops being an academic forecasting exercise and starts affecting what you can actually deliver.

Talking to cloud provider

Your CSP account, technical and capacity teams at the provider should know about significant expected ramps. Well, they will probably even ask about it. If you believe you'll need another 20,000 cores of a particular class in a particular region six months from now, the moment your autoscaler starts returning OOC errors is probably not the ideal time to start that conversation.

Share things such as:

• expected growth and ramp rates;
• target regions and zones;
• preferred and acceptable machine families;
• CPU, memory and storage characteristics;
• major migrations;
• product launches;
• reservation requirements;
• and, importantly, uncertainty in the forecast.

This creates another useful feedback loop. You know your expected demand and the CSP knows much more about the supply side than you do. Maybe your preferred machine family will be constrained in a particular region. Maybe another generation is a much safer bet. Maybe capacity can be guaranteed, but only if it is reserved sufficiently early.

Capacity planning therefore becomes partly an engineering discipline and partly a coordination problem. And trust me - this relationship matters.

Revisiting business assumptions

This one is uncomfortable, but necessary. Some assumptions that were perfectly reasonable several years ago might no longer survive contact with 2026:

"We'll run it on Spot."

"This instance family is always available."

"We'll just scale up another 2,000 nodes."

"We can always expand in this region."

"Compute will continue getting cheaper."

"If one zone fills up, we'll use another one."

I remember hearing that kind of sentences many times across past 16+ years (after cloud-computing era started). I'm 100% sure, that many businesses just run on those assumptions.

If losing cheap Spot capacity makes a service economically unviable, that's not something a better Kubernetes scheduler will fix. If the service requires a very specific machine shape that cannot be obtained at the expected growth rate, no amount of capacity forecasting will manufacture it.

Sometimes the solution to a capacity problem is better automation. Sometimes it's a reservation. Sometimes it's a different architecture. And sometimes the original assumption simply doesn't work anymore.

Knowing which one you're dealing with is a large part of capacity engineering.

What's next?

Forecasting and talking to CSPs tell us how much capacity we're likely to need and whether there's a reasonable chance of obtaining it. But there's another question:

What capacity does the application actually need?

Does it really require this exact instance type? Could it run on another CPU generation? What about a different CPU-to-memory ratio? Does it really require local SSD? How much performance do we lose on an alternative shape? What does "degraded" actually mean for the service? And who should answer those questions? The application developer or the capacity / platform engineer? The answer, obviously, is both (I will die on that hill). But that's where things get considerably more technical, and I'm saving it for the next part, which I'll try to publish very shortly: (Part 2 — Engineering for constrained capacity).