Example Question Format:

Given an existing Kubernetes cluster running version 1.26.0, upgrade the master node and worker node to version 1.27.0. Be sure to drain the master and worker node before upgrading it and uncordon it after the upgrade.

Solution

# Solution Guide: Upgrading Kubernetes Cluster to Version 1.27.0

This guide explains how to upgrade a Kubernetes cluster from version 1.26.0 to 1.27.0. It covers the process for both master (control plane) and worker nodes, ensuring proper draining and uncordoning steps.

---

## **Pre-requisites**
1. **Access Requirements**:
  - SSH access to all nodes in the cluster.
  - Administrative privileges on the nodes.
  - `kubectl` and `kubeadm` installed and configured.

2. **Check Current Versions**:
  - Verify the current Kubernetes versions using:
    ```bash
    kubectl version --short
    kubeadm version
    ```

3. **Backup Cluster Data**:
  - Backup etcd (critical for control plane nodes):
    ```bash
    ETCDCTL_API=3 etcdctl snapshot save /path/to/backup.db \
      --endpoints=https://127.0.0.1:2379 \
      --cacert=/etc/kubernetes/pki/etcd/ca.crt \
      --cert=/etc/kubernetes/pki/etcd/server.crt \
      --key=/etc/kubernetes/pki/etcd/server.key
    ```

4. **Confirm Upgrade Path**:
  - Check available versions:
    ```bash
    kubeadm upgrade plan
    ```

5. **Download Updated Binaries**:
  - Ensure Kubernetes binaries for version 1.27.0 are available in your package manager.

---

## **Steps**

### **1. Upgrade Control Plane (Master Node)**

#### Step 1.1: Drain the Master Node
  - Prevent workloads from being scheduled:
    ```bash
    kubectl drain <master-node-name> --ignore-daemonsets --delete-emptydir-data
    ```

#### Step 1.2: Upgrade kubeadm

   - Use madison tool to check which version of the package is available or not

      ` apt-cache madison kubeadm`
  - Update `kubeadm` to version 1.27.0:
    ```bash
    apt update && apt install -y kubeadm=1.27.0-00
    ```
  - Confirm the installation:
    ```bash
    kubeadm version
    ```

#### Step 1.3: Upgrade Control Plane Components
  - Execute the upgrade plan:
    ```bash
    kubeadm upgrade apply v1.27.0
    ```
  - Follow the on-screen prompts and ensure a successful upgrade.

#### Step 1.4: Upgrade kubelet and kubectl
  - Update `kubelet` and `kubectl`:
    ```bash
    apt update && apt install -y kubelet=1.27.0-00 kubectl=1.27.0-00
    ```
  - Restart `kubelet`:
    ```bash
    systemctl daemon-reload
    systemctl restart kubelet
    ```

#### Step 1.5: Uncordon the Master Node
  - Make the master node schedulable:
    ```bash
    kubectl uncordon <master-node-name>
    ```

---

### **2. Upgrade Worker Nodes**

#### Step 2.1: Drain the Worker Node
  - Remove workloads from the worker node:
    ```bash
    kubectl drain <worker-node-name> --ignore-daemonsets --delete-emptydir-data
    ```

#### Step 2.2: Upgrade kubeadm
  - Update `kubeadm`:
    ```bash
    apt update && apt install -y kubeadm=1.27.0-00
    ```
  - Confirm version:
    ```bash
    kubeadm version
    ```

#### Step 2.3: Upgrade the Node Configuration
  - Run the following on the worker node:
    ```bash
    kubeadm upgrade node
    ```

#### Step 2.4: Upgrade kubelet and kubectl
  - Update binaries:
    ```bash
    apt update && apt install -y kubelet=1.27.0-00 kubectl=1.27.0-00
    ```
  - Restart `kubelet`:
    ```bash
    systemctl daemon-reload
    systemctl restart kubelet
    ```

#### Step 2.5: Uncordon the Worker Node
  - Make the node schedulable:
    ```bash
    kubectl uncordon <worker-node-name>
    ```

---

### **3. Verify the Upgrade**

1. **Check Node Status**:
  ```bash
  kubectl get nodes
  ```
  Ensure all nodes are in the `Ready` state with version `1.27.0`.

2. **Check Control Plane Status**:
  ```bash
  kubectl get componentstatuses
  ```

3. **Validate Workload Functionality**:
  - Deploy test workloads and confirm they function correctly.

---

### **Troubleshooting**

- If `kubeadm upgrade apply` fails:
 - Check logs in `/var/log/syslog` or `/var/log/kube-apiserver.log`.

- If `kubelet` fails to restart:
 - Validate its configuration file `/var/lib/kubelet/config.yaml`.

---

### **Summary**
This guide upgrades a Kubernetes cluster from version 1.26.0 to 1.27.0. The process involves upgrading `kubeadm`, applying the upgrade to the control plane and nodes, draining and uncordoning nodes, and verifying the upgrade. Always ensure you back up the etcd database before beginning.