View Certificate Details

1

Identify the certificate file used for the kube-api server.

$ k describe -n kube-system pod kube-apiserver-controlplane | grep -i cert

2

Identify the Certificate file used to authenticate kube-apiserver as a client to ETCD Server.

$ k describe -n kube-system pod kube-apiserver-controlplane | grep -i etcd-cert

3

Identify the key used to authenticate kubeapi-server to the kubelet server.

$ k describe -n kube-system pod kube-apiserver-controlplane | grep -i kubelet

4

Identify the ETCD Server Certificate used to host ETCD server.

$ k describe pod -n kube-system etcd-controlplane | grep -i cert

5

Identify the ETCD Server CA Root Certificate used to serve ETCD Server.

ETCD can have its own CA. So this may be a different CA certificate than the one used by kube-api server.

$ k describe pod -n kube-system etcd-controlplane | grep -i ca

6

What is the Common Name (CN) configured on the Kube API Server Certificate?

OpenSSL Syntax: openssl x509 -in file-path.crt -text -noout

$ k describe pod -n kube-system kube-apiserver-controlplane | grep -i cert

--tls-cert-file=/etc/kubernetes/pki/apiserver.crt

$ openssl x509 -in /etc/kubernetes/pki/apiserver.crt -text

7

What is the name of the CA who issued the Kube API Server Certificate?

$ k describe pod -n kube-system kube-apiserver-controlplane | grep -i cert

--tls-cert-file=/etc/kubernetes/pki/apiserver.crt

$ openssl x509 -in /etc/kubernetes/pki/apiserver.crt -text

8

What is the Common Name (CN) configured on the ETCD Server certificate?

$ k describe pod -n kube-system etcd-controlplane | grep -i cert

--cert-file=/etc/kubernetes/pki/etcd/server.crt

$ openssl x509 -in /etc/kubernetes/pki/etcd/server.crt -text

9

How long, from the issued date, is the Kube-API Server Certificate valid for?

File: /etc/kubernetes/pki/etcd/server.crt

$ openssl x509 -in /etc/kubernetes/pki/etcd/server.crt -text |grep -iA 5 valid

10

How long, from the issued date, is the Root CA Certificate valid for?

File: /etc/kubernetes/pki/ca.crt

$ penssl x509 -in /etc/kubernetes/pki/ca.crt -text |grep -iA 5 valid

11

Kubectl suddenly stops responding to your commands. Check it out! Someone recently modified the /etc/kubernetes/manifests/etcd.yaml file

You are asked to investigate and fix the issue. Once you fix the issue wait for sometime for kubectl to respond. Check the logs of the ETCD container.

$ vi /etc/kubernetes/manifests/etcd.yaml
--cert-file=/etc/kubernetes/pki/etcd/server.crt

12

The kube-api server stopped again! Check it out. Inspect the kube-api server logs and identify the root cause and fix the issue.

Run crictl ps -a command to identify the kube-api server container. Run crictl logs container-id command to view the logs.

$ crictl ps -a
$ crictl logs 2118a0ecfb1fe

"command failed" err="context deadline exceeded"

$ ps -aux | grep -i kubelet

--config=/var/lib/kubelet/config.yaml

$ cat /var/lib/kubelet/config.yaml | grep -i static

staticPodPath: /etc/kubernetes/manifests

$ cd /etc/kubernetes/manifests
$ vi kube-apiserver.yaml

--etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt

Certificates API

1

Create a CertificateSigningRequest object with the name akshay with the contents of the akshay.csr file


As of kubernetes 1.19, the API to use for CSR is certificates.k8s.io/v1.

Please note that an additional field called signerName should also be added when creating CSR. For client authentication to the API server we will use the built-in signer kubernetes.io/kube-apiserver-client.

$ cat akshay.csr | base64 -w 0
$ vi akshay-csr.yaml

apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: akshay
spec:
  groups:
  - system:authenticated
  request: <Paste the base64 encoded value of the CSR file>
  signerName: kubernetes.io/kube-apiserver-client
  usages:
  - client auth
  
$ k apply -f akshay-csr.yaml

2

What is the Condition of the newly created Certificate Signing Request object?

$ k get csr

3

Approve the CSR Request

$ k certificate approve akshay

4

How many CSR requests are available on the cluster?

$ k get csr

5

Hmmm.. You are not aware of a request coming in. What groups is this CSR requesting access to?

Check the details about the request. Preferebly in YAML.

$ k get csr agent-smith -o yaml

6

That doesn't look very right. Reject that request.

$ k certificate deny agent-smith

7

Let's get rid of it. Delete the new CSR object

$ k delete csr agent-smith

KubeConfig

1

How many clusters are defined in the default kubeconfig file?

$ k config view

2

A new kubeconfig file named my-kube-config is created. It is placed in the /root directory. How many clusters are defined in that kubeconfig file?

$ cat my-kube-config

3

I would like to use the dev-user to access test-cluster-1. Set the current context to the right one so I can do that.

Once the right context is identified, use the kubectl config use-context command.

$ k config --kubeconfig=/root/my-kube-config use-context research

4

We don't want to have to specify the kubeconfig file option on each command.

Set the my-kube-config file as the default kubeconfig by overwriting the content of ~/.kube/config with the content of the my-kube-config file.

$ cp my-kube-config .kube/config

5

With the current-context set to research, we are trying to access the cluster. However something seems to be wrong. Identify and fix the issue.

Try running the kubectl get pods command and look for the error. All users certificates are stored at /etc/kubernetes/pki/users.

$ k get pods

error: unable to read client-cert /etc/kubernetes/pki/users/dev-user/developer-user.crt for dev-user due to open /etc/kubernetes/pki/users/dev-user/developer-user.crt: no such file or directory

$ vi ~/.kube/config

- name: dev-user
  user:
    client-certificate: /etc/kubernetes/pki/users/dev-user/dev-user.crt
    client-key: /etc/kubernetes/pki/users/dev-user/dev-user.key

Role Based Access Controls

1

Inspect the environment and identify the authorization modes configured on the cluster.

Check the kube-apiserver settings.

$ k describe pod -n kube-system kube-apiserver-controlplane | grep -i auth

2

How many roles exist in the default namespace?

$ k get roles

3

How many roles exist in all namespaces together?

$ k get roles -A --no-headers | wc -l

4

What are the resources the kube-proxy role in the kube-system namespace is given access to?

$ k describe role -n kube-system kube-proxy

5

What actions can the kube-proxy role perform on configmaps?

6

Which account is the kube-proxy role assigned to?

$ k describe rolebinding -n kube-system kube-proxy

7

A user dev-user is created. User's details have been added to the kubeconfig file. Inspect the permissions granted to the user. Check if the user can list pods in the default namespace.

Use the --as dev-user option with kubectl to run commands as the dev-user.

$ k get pods --as dev-user

8

Create the necessary roles and role bindings required for the dev-user to create, list and delete pods in the default namespace.

Use the given spec:

Role: developer
Role Resources: pods
Role Actions: list
Role Actions: create
Role Actions: delete
RoleBinding: dev-user-binding
RoleBinding: Bound to dev-user

$ k create role developer --verb=list,create,delete --resource=pods
$ k create rolebinding dev-user-binding --role=developer --user=dev-user

9

A set of new roles and role-bindings are created in the blue namespace for the dev-user. However, the dev-user is unable to get details of the dark-blue-app pod in the blue namespace. Investigate and fix the issue.

We have created the required roles and rolebindings, but something seems to be wrong.

$ k edit role -n blue developer

resourceNames:
- dark-blue-app

10

Add a new rule in the existing role developer to grant the dev-user permissions to create deployments in the blue namespace.

Remember to add api group "apps".

$ k edit role -n blue developer

- apiGroups:
  - apps
  resources:
  - deployments
  verbs:
  - create

Cluster Roles

1

How many ClusterRoles do you see defined in the cluster?

$ k get clusterroles --no-headers | wc -l

2

How many ClusterRoleBindings exist on the cluster?

$ k get clusterrolebindings --no-headers | wc -l

3

What user/groups are the cluster-admin role bound to?

The ClusterRoleBinding for the role is with the same name.

$ k describe clusterrolebinding cluster-admin

4

A new user michelle joined the team. She will be focusing on the nodes in the cluster. Create the required ClusterRoles and ClusterRoleBindings so she gets access to the nodes.

$ k create clusterrole michelle --verb=* --resource=node
$ k create clusterrolebinding michelle-rb --clusterrole=michelle --user=michelle

5

michelle's responsibilities are growing and now she will be responsible for storage as well. Create the required ClusterRoles and ClusterRoleBindings to allow her access to Storage.

Get the API groups and resource names from command kubectl api-resources. Use the given spec:

ClusterRole: storage-admin
Resource: persistentvolumes
Resource: storageclasses
ClusterRoleBinding: michelle-storage-admin
ClusterRoleBinding Subject: michelle
ClusterRoleBinding Role: storage-admin

$ k create clusterrole storage-admin --verb=* --resource=persistentvolumes --verb=* --resource=storageclasses
$ k create clusterrolebinding michelle-storage-admin --clusterrole=storage-admin --user=michelle

Service Accounts

1

How many Service Accounts exist in the default namespace?

$ k get sa

2

What is the secret token used by the default service account?

$ k describe sa default

3

We just deployed the Dashboard application. Inspect the deployment. What is the image used by the deployment?

$ k describe deploy web-dashboard | grep -i image

4

Inspect the Dashboard Application POD and identify the Service Account mounted on it.

$ k describe pod web-dashboard-97c9c59f6-rmf5w 

5

At what location is the ServiceAccount credentials available within the pod?

$ k describe pod web-dashboard-97c9c59f6-lxnmr | grep -i serviceaccount

6

The application needs a ServiceAccount with the Right permissions to be created to authenticate to Kubernetes. The default ServiceAccount has limited access. Create a new ServiceAccount named dashboard-sa.

$ k create serviceaccount dashboard-sa

7

You shouldn't have to copy and paste the token each time. The Dashboard application is programmed to read token from the secret mount location. However currently, the default service account is mounted. Update the deployment to use the newly created ServiceAccount

Edit the deployment to change ServiceAccount from default to dashboard-sa.

$ k edit deploy web-dashboard

serviceAccountName: dashboard-sa

Image Secruity

1

What secret type must we choose for docker registry?

$ k create secret --help

2

We have an application running on our cluster. Let us explore it first. What image is the application using?

$ k describe pod web-694fcfd956-crtqb | grep -i image

3

We decided to use a modified version of the application from an internal private registry. Update the image of the deployment to use a new image from myprivateregistry.com:5000

The registry is located at myprivateregistry.com:5000. Don't worry about the credentials for now. We will configure them in the upcoming steps.

$ k edit deploy web
image: myprivateregistry.com:5000/nginx:alpine

4

Create a secret object with the credentials required to access the registry.

Name: private-reg-cred
Username: dock_user
Password: dock_password
Server: myprivateregistry.com:5000
Email: dock_user@myprivateregistry.com

$ k create secret docker-registry private-reg-cred --docker-username=dock_user --docker-password=dock_password --docker-server=myprivateregistry.com:5000 --docker-email=dock_user@myprivateregistry.com

5

Configure the deployment to use credentials from the new secret to pull images from the private registry

$ k edit deploy web
imagePullSecrets: 
- name: private-reg-cred

Security Contexts

1

What is the user used to execute the sleep process within the ubuntu-sleeper pod?

In the current(default) namespace.

$ k exec ubuntu-sleeper -- whoami

2

Edit the pod ubuntu-sleeper to run the sleep process with user ID 1010.

Note: Only make the necessary changes. Do not modify the name or image of the pod.

$ k edit pod ubuntu-sleeper

securityContext:
  runAsUser: 1010
    
$ k delete pod ubuntu-sleeper --force
$ k apply -f /tmp/kubectl-edit-153834775.yaml

3

A Pod definition file named multi-pod.yaml is given. With what user are the processes in the web container started?

The pod is created with multiple containers and security contexts defined at the Pod and Container level.

$ cat multi-pod.yaml

4

Update pod ubuntu-sleeper to run as Root user and with the SYS_TIME capability.

Note: Only make the necessary changes. Do not modify the name of the pod.

$ k edit pod ubuntu-sleeper

securityContext:
  capabilities:
    add: ["SYS_TIME"]
    
$ k delete pod ubuntu-sleeper --force
$ k apply -f /tmp/kubectl-edit-1779938961.yaml

5

Now update the pod to also make use of the NET_ADMIN capability.

Note: Only make the necessary changes. Do not modify the name of the pod.

$ k edit pod ubuntu-sleeper

securityContext:
  capabilities:
    add: ["SYS_TIME","NET_ADMIN"]
    
$ k delete pod ubuntu-sleeper --force
$ k apply -f /tmp/kubectl-edit-3357527143.yaml

Network Policies

1

How many network policies do you see in the environment?

We have deployed few web applications, services and network policies. Inspect the environment.

$ k get netpol

2

Which pod is the Network Policy applied on?

$ k describe netpol payroll-policy 

PodSelector:     name=payroll

$ k get pod --show-labels | grep -i name=payroll

3

Create a network policy to allow traffic from the Internal application only to the payroll-service and db-service.

Use the spec given below. You might want to enable ingress traffic to the pod to test your rules in the UI.

Also, ensure that you allow egress traffic to DNS ports TCP and UDP (port 53) to enable DNS resolution from the internal pod.

Policy Name: internal-policy
Policy Type: Egress
Egress Allow: payroll
Payroll Port: 8080
Egress Allow: mysql
MySQL Port: 3306

$ vi internal-policy.yaml

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: internal-policy
spec:
  podSelector:
    matchLabels:
      name: internal
  policyTypes:
  - Egress
  - Ingress
  ingress:
    - {}
  egress:
  - to:
    - podSelector:
        matchLabels:
          name: mysql
    ports:
    - protocol: TCP
      port: 3306

  - to:
    - podSelector:
        matchLabels:
          name: payroll
    ports:
    - protocol: TCP
      port: 8080

  - ports:
    - protocol: UDP
      port: 53
    - protocol: UDP
      prot: 53

$ k apply -f internal-policy.yaml