Kubernetes High Availability: Why Is a PDB Only Part of the Story?

Node maintenance is a routine part of operating Kubernetes. Pods are evicted, rescheduled, and replaced — and with the right configuration, users should barely notice.

That’s where PodDisruptionBudgets (PDBs) come in. They are designed to ensure that voluntary disruptions, such as node drains, do not take down too many replicas of the same workload at once.

But what happens when the PDB is working exactly as configured, and users still experience an outage?

Let’s say a production service runs with 3 replicas. A Node needs to be patched, so the platform team starts a routine drain. The PDB allows 1 Pod to be evicted, 2 healthy replicas remain, and a replacement Pod starts on another Node.

Everything looks correct. And yet, for a few seconds, the service starts returning 502 Bad Gateway errors.

That question is what led me to take a closer look at PDBs and the wider picture around Kubernetes availability.

This reinforces an important point: keeping the required number of Pods alive is only one part of keeping an application available.

So, what does a PDB actually promise, and where does that promise quietly run out? What happens to traffic when a Pod is leaving? And what else needs to be in place for an application to handle disruptions safely?

Let’s start with a simple maintenance scenario and connect the pieces.

The maintenance scenario

The platform team is patching a batch of worker Nodes as part of the monthly security maintenance cycle. critical-service runs three replicas, protected by:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: critical-service
  namespace: critical
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: critical-service

With minAvailable: 2 , Kubernetes can voluntarily evict one replica — but not a second until the replacement Pod comes back up and is counted as healthy again.

So, The PDB did exactly what it was configured to do: two healthy replicas remained available throughout the drain, yet the service still returned 502 Bad Gateway errors.

To understand why, we need to look at what a PDB actually guarantees — and, just as importantly, what it doesn’t.

Understanding PDB Limits

A PDB defines how many Pods from a workload should remain available when Pods are voluntarily disrupted.

It can be expressed in one of two ways:

• minAvailable — the minimum number (or percentage) of Pods that must stay healthy at all times.
• maxUnavailable — the maximum number (or percentage) of Pods allowed to be down at once.

Kubernetes exposes the current state of the budget through:

kubectl get pdb -n critical

For example:

NAME              MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
critical-service  2               N/A               1                     14d

The important field is disruptionsAllowed: how much room is currently left for voluntary disruption.

With three healthy replicas and minAvailable: 2, one disruption is allowed — evicting a single Pod still leaves two available. But if a replica is already unavailable for any reason, the number of healthy replicas is already two, and disruptionsAllowed drops to 0. The next voluntary eviction has to wait. This is why PDB status is useful operationally — disruptionsAllowed can change as the health of the workload changes, even when no disruption is currently being performed.

A Pod can also disappear because of an involuntary disruption — for example, a Node or VM failure, resource pressure, or Spot instance reclamation. A PDB cannot prevent those events, but the resulting loss of a replica reduces the workload’s available margin.

Say pod-c disappears on its own:

$ kubectl get pods
NAME     READY   STATUS    RESTARTS   AGE
pod-a    1/1     Running   0          2d
pod-b    1/1     Running   0          2d
pod-c    0/1     Unknown   0          2d

We still have two healthy replicas, but now disruptionsAllowed is already at 0.

A drain attempted later will simply be blocked.

There is another important limit to what a PDB controls: voluntary evictions.

For example, deleting a Pod directly:

kubectl delete pod pod-a

does not use the Eviction API, so the PDB is not consulted. The same applies when kubectl drain is explicitly configured to bypass eviction:

kubectl drain <node> --disable-eviction

So a PDB limits how many Pods can be voluntarily evicted at once — it does not prevent every possible Pod deletion.

And even when the PDB is respected, it only controls the number of Pods that can be disrupted. It says nothing about the requests those Pods are serving or what happens to traffic while a Pod is terminating.

That actually leads to the next question:

What Happens When a Pod Terminates

Going back to the 3 replicas, pod-c gets evicted, pod-a and pod-b stay up, and minAvailable: 2 is satisfied the entire time. From the PDB’s perspective, nothing went wrong.

But pod-c might have had in-flight requests or open connections when it was evicted.

A PDB does not guarantee that:

• active requests finish successfully
• the application has stopped accepting new work
• traffic components immediately observe the endpoint change
• existing connections drain cleanly
• the remaining replicas can immediately receive the extra traffic

This highlights the difference between Pod availability and application availability: enough Pods can remain healthy while requests are still failing. And that is where our 502 errors can come from.

When the Pod starts terminating, the traffic path is roughly:

That takes a moment, and during that moment, requests can still land on a Pod that’s already shutting down.

If the application responds to SIGTERM and immediately closes its listener, any request that arrives in that window simply fails. The PDB is still correct. Two healthy replicas still exist. Users still see a 502, a reset connection, or a timeout. The PDB wasn’t the problem here - the Pod’s shutdown behavior was.

This is where graceful shutdown and readiness come in. A shutdown sequence that actually protects traffic looks like this:

The exact implementation depends on the application, but the principle is simple:

Stop taking new traffic, finish existing work, then terminate.

terminationGracePeriodSeconds should give the application enough time to complete that process.

Readiness is also important here because it answers a different question:

Which is different from:

A Pod can be alive but intentionally not ready to serve traffic.

Put together, these three mechanisms complement each other:

Together, they provide much stronger protection during maintenance than a PDB alone.

But there’s still a gap:

The rest of the availability picture

Another Kubernetes mechanism that is sometimes confused with PDBs is Pod Priority and Preemption.

A PriorityClass tells the scheduler how important a Pod is compared with other Pods. When the cluster is full and a higher-priority Pod cannot be scheduled, Kubernetes may preempt lower-priority Pods to make room.

PDBs are considered during this process, but they are not an absolute guarantee. In some situations, preemption can still violate a PDB.

For critical workloads, Pod priority and PDBs therefore need to be considered together.

But there is another side to the problem. A PDB can also be too restrictive. Instead of allowing too much disruption, it can prevent maintenance from progressing when the cluster would otherwise be able to handle it safely.

For example:

minAvailable: 100%

or:

maxUnavailable: 0

Neither of these leaves room for voluntary disruption:

The cluster is behaving correctly. The policy is simply too restrictive for normal maintenance.

The goal is not to make every PDB permissive. It is to make the disruption budget match the operational reality of the workload.

And even a perfectly designed PDB cannot protect a workload whose replicas are badly placed.

Imagine Pods A, B, and C from the same workload are all placed on Node 1. If Node 1 fails, all three replicas can disappear together.

That is not a PDB problem. It is a placement problem.

This is where the bigger picture comes together.

Each layer addresses a different failure mode. High availability comes from the combination, not from a single Kubernetes feature.

The Takeaway

A good PDB should not only look correct in YAML — it should behave correctly under real cluster conditions. If disruptionsAllowed stays at zero, investigate why. A replica may be unhealthy, a Node may have failed, the workload may not have enough replicas, or the PDB may simply be too restrictive. Policies such as Kyverno or OPA/Gatekeeper can help catch obviously problematic configurations before they reach production.

But static validation only goes so far. The best way to test your availability setup is to perform the operation itself: drain a Node under realistic traffic and observe the entire process. Check the PDB status, Pod readiness, EndpointSlice changes, application shutdown behavior, request errors and latency, and how quickly replacement Pods become ready.

A PDB is only one part of availability. What matters is whether the application continues serving traffic successfully while Pods and Nodes are being replaced, restarted, or removed.