K8S – pods stuck on ‘Terminating’

Kubernetes - Pod Stuck In Terminating State K8S

Hey, today a quick post on a bug that sometimes happens in kubernetes while removing pods. If someone is unlucky, when removing the kubernetes, they can “get stuck” in the Terminating state and hang like that for a few days.

k8s stuck kubernetes terminating
Kubectl delete pods --all -n NAMESPACE_NAME
Kubectl delete pods POD_NAME -n NAMESPACE_NAME
Kubectl delete deployments DEPLOYMENT_NAME -n NAMESPACE_NAME

Deleting PODS in the ‘Terminating’ State

Manual removal of pod from namespace will not help, even if we specify that we want to delete a specific pod and give its exact name. Removing the ‘deployment’ will also do nothing. POD on kubernetes will be stubbornly stuck in the state of ‘Terminating’.

If we want to fix this, we have to use some force. The command given below always helps me, it removes it almost immediately. Of course, if you have a POD in a dedicated namespace, add its name to the command.

kubectl delete pod POD_NAME --grace-period=0 --force
kubectl delete pod POD_NAME --grace-period=0 --force -n NAMESPACE_NAME

There is one more solution, a bit more drastic, that I didn’t have to use. If forcibly removing the pod doesn’t work, you can restart the entire ‘worker’.

A little bit of automation

A bit of automation from bash if someone is bored of removing blocked pods manually:

namespace="enter_the_name"

delpods=$(kubectl get pods -n ${namespace} | grep -i 'Terminated' | awk '{print $1 }')

for i in ${delpods[@]}; do
  kubectl delete pod $i --force=true --wait=false --grace-period=0  -n ${namespace}
done
  • At the beginning you give the name of the namespace from which you want to remove blocked pods.
  • In the next part, the names of all pods in the terminating state are saved in the variable. Thanks to the ‘grep‘ command, I only print those in the terminating state. Thanks to the ‘awk‘ command, I only save the names of pods.
  • Finally, using a simple loop ‘for‘, I delete pods whose names are stored in the variable ‘delpods’.

You can find more information about Pods and Kubernetes in the k8s documentation.

You can find more articles about Kubernetes in the Kubernetes category.

Newsletter blog Lepczynski IT
Tags:

1 thought on “K8S – pods stuck on ‘Terminating’”

Leave a Reply

Your email address will not be published. Required fields are marked *