What is a disruption

the action of preventing something, especially a system, process, or event, from continuing as usual or as expected.

– https://dictionary.cambridge.org/de/worterbuch/englisch/disruption

Kubernetes differentiates between 2 kinds of disruptions:

• involuntary (Outside the control of Kubernetes)
  • hardware failure
  • kernel panic
  • etc.
• voluntary (controlled using Kubernetes mechanisms)
  • draining a node (repair, upgrade, scale down)
  • priority based eviction (to allow other, high priority pods to be scheduled instead)
  • Deployment,StatefulSet,DaemonSet, etc. update
  • deleting a pod

Kubernetes will try to move pods from unhealthy to healthy nodes when encountering involuntary disruptions. For voluntary disruptions, Kubernetes provides APIs to control how many pods for a given selector can be disrupted simultaneously.

Note

This article only talks about the kubelet + CRI based container runtimes. Pre-CRI based implementations (i.e., Docker) are slightly different.

Logging & Kubernetes

Logs can provide helpful insights into an application. Especially when troubleshooting bugs, logs can help us understand the why? In Kubernetes, logs from containers are being handled by the container-runtime (e.g. containerd, cri-o, etc.) & the kubelet, as long as containers write logs to stdout or stderr. By default, logs can be fetched using kubectl -n <namespace> logs <pod> -c <container>. Though, it is not always an option, nor is it especially convenient. There are several situations where Kubernetes will move pods across nodes, which causes the kubelet on the old node to delete the logs. To avoid losing logs and make searching logs easier, they can be shipped to a central location.