04-certificate-authority

Provisioning a CA and Generating TLS Certificates

---

이번 랩에서는 openssl을 사용하여 PKI 인프라를 설정하고 인증 기관(Certificate Authority)을 부트스트랩해 kube-apiserver, kube-controller-manager, kube-scheduler, kubelet 그리고 kube-proxy와 같은 컴포넌트를 위한 TLS 인증서를 생성할 것입니다.

Certificate Authority

---

이 섹션에서는 인증 기관(Certificate Authority)를 설정해 Kubernetes 컴포넌트를 위한 추가 TLS 인증서를 생성할 수 있도록합니다.

openssl 을 사용해 CA를 설정하고 인증서를 생성하는 과정은 특히 처음 할 때 시간이 많이 걸릴 수 있습니다.

아래 ca.conf 파일은 Kubernetes 컴포넌트에 대한 인증서를 생성하는데 필요한 모든 정보가 정의 되어있습니다. 해당 파일은 각 환경에 맞게 수정하시면 됩니다. (가장 중요한 부분이라 생각되니, 오타 없이 수정하셔야합니다.)

mkdir /k8s-hardway/tls/
cat <<EOF > /k8s-hardway/tls/ca.conf 
[req]
distinguished_name = req_distinguished_name
prompt             = no
x509_extensions    = ca_x509_extensions

[ca_x509_extensions]
basicConstraints = CA:TRUE
keyUsage         = cRLSign, keyCertSign

[req_distinguished_name]
C   = KR
CN  = CA

[admin]
distinguished_name = admin_distinguished_name
prompt             = no
req_extensions     = default_req_extensions

[admin_distinguished_name]
CN = admin
O  = system:masters

# Service Accounts
#
# The Kubernetes Controller Manager leverages a key pair to generate
# and sign service account tokens as described in the
# [managing service accounts](https://kubernetes.io/docs/admin/service-accounts-admin/)
# documentation.

[service-accounts]
distinguished_name = service-accounts_distinguished_name
prompt             = no
req_extensions     = default_req_extensions

[service-accounts_distinguished_name]
CN = service-accounts

# Worker Nodes
#
# Kubernetes uses a [special-purpose authorization mode](https://kubernetes.io/docs/admin/authorization/node/)
# called Node Authorizer, that specifically authorizes API requests made
# by [Kubelets](https://kubernetes.io/docs/concepts/overview/components/#kubelet).
# In order to be authorized by the Node Authorizer, Kubelets must use a credential
# that identifies them as being in the 'system:nodes' group, with a username
# of 'system:node:<nodeName>'.

[node-1]
distinguished_name = node-1_distinguished_name
prompt             = no
req_extensions     = node-1_req_extensions

[node-1_req_extensions]
basicConstraints     = CA:FALSE
extendedKeyUsage     = clientAuth, serverAuth
keyUsage             = critical, digitalSignature, keyEncipherment
nsCertType           = client
nsComment            = "Node-1 Certificate"
subjectAltName       = DNS:node-1, IP:127.0.0.1
subjectKeyIdentifier = hash

[node-1_distinguished_name]
CN = system:node:node-1
O  = system:nodes
C  = KR

[node-2]
distinguished_name = node-2_distinguished_name
prompt             = no
req_extensions     = node-2_req_extensions

[node-2_req_extensions]
basicConstraints     = CA:FALSE
extendedKeyUsage     = clientAuth, serverAuth
keyUsage             = critical, digitalSignature, keyEncipherment
nsCertType           = client
nsComment            = "Node-2 Certificate"
subjectAltName       = DNS:node-2, IP:127.0.0.1
subjectKeyIdentifier = hash

[node-2_distinguished_name]
CN = system:node:node-2
O  = system:nodes
C  = KR

# Kube Proxy Section
[kube-proxy]
distinguished_name = kube-proxy_distinguished_name
prompt             = no
req_extensions     = kube-proxy_req_extensions

[kube-proxy_req_extensions]
basicConstraints     = CA:FALSE
extendedKeyUsage     = clientAuth, serverAuth
keyUsage             = critical, digitalSignature, keyEncipherment
nsCertType           = client
nsComment            = "Kube Proxy Certificate"
subjectAltName       = DNS:kube-proxy, IP:127.0.0.1
subjectKeyIdentifier = hash

[kube-proxy_distinguished_name]
CN = system:kube-proxy
O  = system:node-proxier
C  = KR

# Controller Manager
[kube-controller-manager]
distinguished_name = kube-controller-manager_distinguished_name
prompt             = no
req_extensions     = kube-controller-manager_req_extensions

[kube-controller-manager_req_extensions]
basicConstraints     = CA:FALSE
extendedKeyUsage     = clientAuth, serverAuth
keyUsage             = critical, digitalSignature, keyEncipherment
nsCertType           = client
nsComment            = "Kube Controller Manager Certificate"
subjectAltName       = DNS:kube-proxy, IP:127.0.0.1
subjectKeyIdentifier = hash

[kube-controller-manager_distinguished_name]
CN = system:kube-controller-manager
O  = system:kube-controller-manager
C  = KR

# Scheduler
[kube-scheduler]
distinguished_name = kube-scheduler_distinguished_name
prompt             = no
req_extensions     = kube-scheduler_req_extensions

[kube-scheduler_req_extensions]
basicConstraints     = CA:FALSE
extendedKeyUsage     = clientAuth, serverAuth
keyUsage             = critical, digitalSignature, keyEncipherment
nsCertType           = client
nsComment            = "Kube Scheduler Certificate"
subjectAltName       = DNS:kube-scheduler, IP:127.0.0.1
subjectKeyIdentifier = hash

[kube-scheduler_distinguished_name]
CN = system:kube-scheduler
O  = system:system:kube-scheduler
C  = KR

# API Server
#
# The Kubernetes API server is automatically assigned the 'kubernetes'
# internal dns name, which will be linked to the first IP address ('172.16.0.1')
# from the address range ('172.16.0.0/24') reserved for internal cluster
# services.

[kube-api-server]
distinguished_name = kube-api-server_distinguished_name
prompt             = no
req_extensions     = kube-api-server_req_extensions

[kube-api-server_req_extensions]
basicConstraints     = CA:FALSE
extendedKeyUsage     = clientAuth, serverAuth
keyUsage             = critical, digitalSignature, keyEncipherment
nsCertType           = client
nsComment            = "Kube Scheduler Certificate"
subjectAltName       = @kube-api-server_alt_names
subjectKeyIdentifier = hash

[kube-api-server_alt_names]
IP.0  = 127.0.0.1
IP.1  = 172.16.0.1
DNS.0 = kubernetes
DNS.1 = kubernetes.default
DNS.2 = kubernetes.default.svc
DNS.3 = kubernetes.default.svc.cluster
DNS.4 = kubernetes.svc.cluster.local
DNS.5 = controlplane-1.kubernetes.local
DNS.6 = api-server.kubernetes.local

[kube-api-server_distinguished_name]
CN = kubernetes
C  = KR

[default_req_extensions]
basicConstraints     = CA:FALSE
extendedKeyUsage     = clientAuth
keyUsage             = critical, digitalSignature, keyEncipherment
nsCertType           = client
nsComment            = "Admin Client Certificate"
subjectKeyIdentifier = hash
EOF

Kubernetes the hard way 튜토리얼을 완료하기 위해 ca.conf 파일의 모든 내용을 이해할 필요는 없지만,

openssl 과 인증서 관리를 위한 설정을 배우는 출발점으로 생각하면 좋습니다.

모든 certificate authority는 Private key와 root certificate로 시작됩니다.

이번 섹션에서는 Self-signed certificate authority를 생성 할 것이며, 튜토리얼을 진행하는데 있어 이것만으로 충분하지만, 실제 운영 환경에서는 이렇게 하지 않는것이 좋습니다.

CA 설정 파일, certificate 그리고 private key를 생성합니다.

{
  openssl genrsa -out /k8s-hardway/tls/ca.key 4096
  openssl req -x509 -new -sha512 -noenc \
    -key /k8s-hardway/tls/ca.key -days 3653 \
    -config /k8s-hardway/tls/ca.conf \
    -out /k8s-hardway/tls/ca.crt
}

ls -al /k8s-hardway/tls/
# --- result ---
# total 24
# drwxr-xr-x 2 root root 4096 May 29 01:00 .
# drwxr-xr-x 4 root root 4096 May 29 00:47 ..
# -rw-r--r-- 1 root root 5643 May 29 00:49 ca.conf
# -rw-r--r-- 1 root root 1793 May 29 01:00 ca.crt
# -rw------- 1 root root 3272 May 29 01:00 ca.key

Create Client and Server Certificates

---

이번 섹션에서는 각 Kubernetes 컴포넌트의 Client와 Server 인증서와 Kubernetes admin 사용자에 대한 Client 인증서를 생성합니다.

kubernetes 컴포넌트의 certificate와 private key를 생성합니다.

certs=(
  "admin" "node-1" "node-2"
  "kube-proxy" "kube-scheduler"
  "kube-controller-manager"
  "kube-api-server"
  "service-accounts"
)

for i in ${certs[*]}; do
  openssl genrsa -out "/k8s-hardway/tls/${i}.key" 4096

  openssl req -new -key "/k8s-hardway/tls/${i}.key" -sha256 \
    -config "/k8s-hardway/tls/ca.conf" -section ${i} \
    -out "/k8s-hardway/tls/${i}.csr"
  
  openssl x509 -req -days 3653 -in "/k8s-hardway/tls/${i}.csr" \
    -copy_extensions copyall \
    -sha256 -CA "/k8s-hardway/tls/ca.crt" \
    -CAkey "/k8s-hardway/tls/ca.key" \
    -CAcreateserial \
    -out "/k8s-hardway/tls/${i}.crt"
done

위 명령어를 실행하면, private key, certificate request와 각 Kubernetes 컴포넌트에 대한 Singed SSL Certificate 파일이 생성됩니다.

ls -1 /k8s-hardway/tls/*.crt /k8s-hardway/tls/*.key /k8s-hardway/tls/*.csr

/k8s-hardway/tls/admin.crt
/k8s-hardway/tls/admin.csr
/k8s-hardway/tls/admin.key
/k8s-hardway/tls/ca.crt
/k8s-hardway/tls/ca.key
/k8s-hardway/tls/kube-api-server.crt
/k8s-hardway/tls/kube-api-server.csr
/k8s-hardway/tls/kube-api-server.key
/k8s-hardway/tls/kube-controller-manager.crt
/k8s-hardway/tls/kube-controller-manager.csr
/k8s-hardway/tls/kube-controller-manager.key
/k8s-hardway/tls/kube-proxy.crt
/k8s-hardway/tls/kube-proxy.csr
/k8s-hardway/tls/kube-proxy.key
/k8s-hardway/tls/kube-scheduler.crt
/k8s-hardway/tls/kube-scheduler.csr
/k8s-hardway/tls/kube-scheduler.key
/k8s-hardway/tls/node-1.crt
/k8s-hardway/tls/node-1.csr
/k8s-hardway/tls/node-1.key
/k8s-hardway/tls/node-2.crt
/k8s-hardway/tls/node-2.csr
/k8s-hardway/tls/node-2.key
/k8s-hardway/tls/service-accounts.crt
/k8s-hardway/tls/service-accounts.csr
/k8s-hardway/tls/service-accounts.key

Distribute the Client and Server Certificates

---

이 섹션에서는 jumpbox 서버에서 node-1, node-2 서버로 각 Kubernetes 컴포넌트가 인증서를 쌍으로 찾을 수 있는 디렉터리로 이전에 생성한 인증서를 복사합니다.

node-1, node-2 서버에 맞게 certificate와 private key를 복사합니다.

for host in node-1 node-2; do
  ssh root@$host mkdir /var/lib/kubelet/
  
  scp /k8s-hardway/tls/ca.crt root@$host:/var/lib/kubelet/
    
  scp /k8s-hardway/tls/$host.crt \
    root@$host:/var/lib/kubelet/kubelet.crt
    
  scp /k8s-hardway/tls/$host.key \
    root@$host:/var/lib/kubelet/kubelet.key
done

ca.crt                                        100% 1793     4.2MB/s   00:00
node-1.crt                                    100% 2041     3.8MB/s   00:00
node-1.key                                    100% 3272     6.1MB/s   00:00
ca.crt                                        100% 1793     3.9MB/s   00:00
node-2.crt                                    100% 2041     4.8MB/s   00:00
node-2.key                                    100% 3272     7.7MB/s   00:0

controlplane-1 서버로 동일하게 인증서를 복사합니다.

scp \
 /k8s-hardway/tls/ca.key /k8s-hardway/tls/ca.crt \
  /k8s-hardway/tls/kube-api-server.key /k8s-hardway/tls/kube-api-server.crt \
  /k8s-hardway/tls/service-accounts.key /k8s-hardway/tls/service-accounts.crt \
  root@controlplane-1:~/

ca.key                                        100% 3272     5.7MB/s   00:00
ca.crt                                        100% 1793     5.5MB/s   00:00
kube-api-server.key                           100% 3272     9.5MB/s   00:00
kube-api-server.crt                           100% 2256     5.9MB/s   00:00
service-accounts.key                          100% 3272    10.5MB/s   00:00
service-accounts.crt                          100% 1952     5.9MB/s   00:00

kube-proxy, kube-controller-manager, kube-schedueler 그리고 kubelet 클라이언트 인증서는 다음 실습에 클라이언트 인증 설정 파일을 생성하는데 사용됩니다.

Next: Generating Kubernetes Configuration Files for Authentication