Pod Density Per Node Calculator

In Kubernetes you don’t typically fail due to lack of disk space; you break capacity planning. Usually system falls over when you hit a limit. This might be too many pods for the kubelet scheduler to manage without stalling, too many connections and IP addresses for kernel, or who knows what. Memory and CPU are tangible resources that you can measure on a dashboard; it’s understandable that you fixate on them.

Pod density is a complex constraint with software and networking limits that will bite you long before you hit any hardware limit. It’s easy to guess wrong about whether you’re more likely to have a connectivity bottleneck or compute bottleneck, but the calculator above spares you. Just plug in your workload profiles and node specs, and let the calculator do the math.

How to Plan Your Cluster Space

So let’s start with what determines all of this: your average pod request size. Unless your service is really light (e.g., uses only 128 MB of RAM and 50 millicores), it will take up a whole CPU core and chunk of RAM. This means that soon enough, you’ll be out of resources for new pods before you get to far. In other words, the limit here is simple arithmetic.

On the other hand, if your service is truly tiny (e.g., uses only 128 MB of RAM and 50 millicores) then you could potentialy fit hundreds of them on a node. And this is where the hidden limits come into play. Each pod require some space in the kubelet’s tracking structures, a spot in the kernel’s network namespace table, and an IP address. You may not have used even half of machine’s CPU. However, you may have already exceeded the `maxPods` limit or reached maximum number of Pod IPs. This means you cannot schedule any more containers.

There is a quiet cost to density, and that silent tax are DaemonSets. Nearly all clusters deploy monitoring sidecars, security scanners, CNI plugins, logging agents, etc. These are deployed as DaemonSets. They’re pods that begin running on each node before any application code lands. In a tight environment with limited IPs, a handful of system pod could grab half your capacity on a small node.

To compensate, the tool lets you subtract out these fixed costs from your overall allowance. This is an important tweak. If you don’t account for system pods, you’ll over provision nodes that look empty on paper but are actualy stuffed with operational overhead.

The other part of the problem is that network config is crazy-varied by provider, as well. Cloud providers typically use some form of CNI plugin which tie pod IPs to some sort of underlying subnet/subnet/network interface. For example, AWS VPC CNI constrains pods based off the number of secondary IP addresses supported per ENI (Elastic Network Interface) and the max number of ENIs supported for each instance type. Even if you’ve got a huge server with oodles of RAM, if it’s only got two ENIs that support limited numbers of IPs, then your pod count gets capped pretty soon.

Moving to overlay or prefix delegation mode fundamentally changes this calculation, removing the hard limit of network addresses and instead letting pod count be constrained again by things like CPU/memory. The page has nice reference tables that detail all this out, exactly where various CNIs move the bottleneck from network address back to, say, memory/CPU.

Finally, leave some space for chaos. Leave headroom in your cluster for software updates, moving nodes, and unexpected changes in size. Packing your nodes to 100% usage means the scheduler will have to evict other pods under any sort of maintenance activity, which leads to cascading failures. By setting a 20+% HA buffer or higher, you ensure that when a node goes down, its workload can be migrated to nearby nodes without landing in pending state or causing OOM kills. It’s not only about stability, but also maintaining predictable latency even in the face of disruptions.

What I’ve found over time though is it’s largely a question of knowing what those numbers mean. How thin do you slice? Memory and CPU are continuous resources. Pods are discrete slots with an upper bound. IP addresses are finite leases. You want to keep all of them in balance and not have one dictate your scaling behavior too soon. Let the tool tell you which is the binding limit, then set your node pool strategy around that.

Are you bottlenecked by the number of connections or pod IPs? Is your environment such that bigger nodes are cheaper then smaller nodes? More big nodes or lots of little nodes? Capacity planning isn’t a race to maximum density for its own sake. It’s about never getting surprised when the network’s run out of room way before the server is.