Sure shot question 5: Mark a Worker Node as Unschedulable and Reschedule Pods

Question Format

 

Mark the worker node named kworker as unschedulable and reschedule all the pods running on

 

Prerequisites

 

Before proceeding, ensure you have:

  • A running Kubernetes cluster.
  • kubectl access to the cluster.
  • A worker node named kworker that you need to mark as unschedulable.

 

Step 1: Mark kworker as Unschedulable

 

To prevent new pods from being scheduled on kworker, run:

kubectl cordon kworker

This marks kworker as unschedulable, preventing the Kubernetes scheduler from assigning new pods to it.

 

Step 2: Verify the Node Status

Check if kworker is now unschedulable:

kubectl get nodes

You should see the SchedulingDisabled status for kworker:

kworker   Ready, SchedulingDisabled   ...

 

Step 3: Reschedule Existing Pods

To reschedule all running pods on kworker, first drain the node:

kubectl drain kworker --ignore-daemonsets --delete-emptydir-data

This command:

Evicts all running pods from kworker.

Ignores daemonset pods, as they will be recreated automatically.

Deletes emptyDir volumes, ensuring clean rescheduling.

Note: If any pods are using local storage (e.g., emptyDir volumes), you must use --delete-emptydir-datato force their eviction.

 

Step 4: Verify Pod Rescheduling

Check if all pods from kworker have been rescheduled to other available nodes:

kubectl get pods -o wide

Ensure that the pods previously running on kworker are now running on different worker nodes.

 

Step 5: Uncordon kworker (Optional)

Once maintenance is complete, allow scheduling on kworker again:

kubectl uncordon kworker

This marks the node as schedulable, allowing new pods to be assigned to it.