study 6차
✍ K8S 인증/인가
출처 - 김태민 기술 블로그 - 링크 링크2
서비스 어카운트(Service Account)
API 서버 사용 : kubectl(config, 다수 클러스터 관리 가능), 서비스 어카운트, https(x.509 Client Certs)
인증(Authentication)
- X.509 Client Certs : kubeconfig 에 CA crt(발급 기관 인증서) , Client crt(클라이언트 인증서) , Client key**(클라이언트 개인키) 를 통해 인증
- kubectl : 여러 클러스터(kubeconfig)를 관리 가능 - contexts 에 클러스터와 유저 및 인증서/키** 참고
- Service Account** : 기본 서비스 어카운트(default) - 시크릿(CA crt 와 token)
인가(Authorization)
- 인가 방식 : RBAC(Role, RoleBinding)**, ABAC, Webhook, Node Authorization⇒
RBAC 발음을 어떻게 하시나요? - RBAC** : 역할 기반의 권한 관리, 사용자와 역할을 별개로 선언 후 두가지를 조합(binding)해서 사용자에게 권한을 부여하여 kubectl or API로 관리 가능
- Namespace/Cluster - Role/ClusterRole, RoleBinding/ClusterRoleBinding, Service Account
- Role(롤) - (RoleBinding 롤 바인딩) - Service Account(서비스 어카운트) : 롤 바인딩은 롤과 서비스 어카운트를 연결
- Role(네임스페이스내 자원의 권한) vs ClusterRole(클러스터 수준의 자원의 권한)
kube/config 파일 내용
- clusters : kubectl 이 사용할 쿠버네티스 API 서버의 접속 정보 목록. 원격의 쿠버네티스 API 서버의 주소를 추가해 사용 가능
- users : 쿠버네티스의 API 서버에 접속하기 위한 사용자 인증 정보 목록. (서비스 어카운트의 토큰, 혹은 인증서의 데이터 등)
- contexts : cluster 항목과 users 항목에 정의된 값을 조합해 최종적으로 사용할 쿠버네티스 클러스터의 정보(컨텍스트)를 설정.
- 예를 들어 clusters 항목에 클러스터 A,B 가 정의돼 있고, users 항목에 사용자 a,b 가 정의돼 있다면 cluster A + user a 를 조합해,
'cluster A 에 user a 로 인증해 쿠버네티스를 사용한다' 라는 새로운 컨텍스트를 정의할 수 있습니다.
- kubectl 을 사용하려면 여러 개의 컨텍스트 중 하나를 선택.
cat .kube/config
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1Ea3dNVEl5TkRjMU1sb1hEVE14TURnek1ESXlORGMxTWxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTG1qCml1cW11UUxWTXN6UE83VUpxTkdCNHdXQ3RnYTl1cFcwYUVNVmUrZm41YXZZMWxUWUZqZjBCb1VlQXhOWmc5YXoKRU1FZVJMWCt1ZzhqTDNETjhCTzEwdUEwSzF6b3ZpQVVtbDlCU2dNWU9FOHpUMFJsV2tvcnBtVDNGai9td1lJagpEemRxYld6MlpuQ1FoQ3dvYURzdlpoUVNMRTh6dnFwU0F5c0hNSUdzV3J0anI4aC9QaW52dnF5bUo0UlFhWlY3CnNuZ0lzMDBqakdGbFowcUVueWZMSGtBeHpjSktVUnJHamFsZm1RdmZ3WkZ2Z0pjam5rSG9jb3g0T0JKUEh0N2EKdFE1OEpBTTF3cng0b3pFSjh1MExsa21LOWYwWGVzQmRGeUhFamZ1elhTYml0Q09sbTR1Q1o3UkVRVmRjZWk1SAo3Tjg1M1RjbWRIck9tRkQwZVpVQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZLRVYvZFNBUkJteVhyLytxUkVnb1h5QUg3UTZNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFDQ0M4cDRQRmdoVVFDbW5weWk1SDAxYVRNYXp0Si9pdkw0amxiMWJNdXc3ZjJNZmM0UQpDRGw2UWVNd2FpYk9raHNrVGhMTEtRckQwQ0xqWXNCSy9iNVhQSTNtMmoxS0cvc1ExREFPL0hNdmt6RmkzUDdrCmJHOUErdWk1YXJPREs5eWJFQ2NtUG5adnVmWkFSY3d3dkp1ZGRMUy9QZERkOW9ZVGgzV3FQMjloVk9tZnZUS3kKNFhzeVg0cHk5dzVTNkYxaGVpUE9odnprMWRzNWFZZENBR1E5R0ZRb3BIQSs1Wm9YOWJjazFuN0FiMDVua0UrUQprMTVnc1VhQWFEMGVGUlRHY0tRTzM5dW1ZdkxhVnUrL20xcDFFRWU0YWdLdktvUGZlZ1VJTFQ0dGtLdjFwcWYvCmhIZldDUFo3Vy9ldmRZODI5WmtudE1HWHZ5QXZaWHFUZE1KZwotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
server: https://192.168.100.10:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
namespace: default
user: kubernetes-admin
name: admin@k8s
current-context: admin@k8s
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURJVENDQWdtZ0F3SUJBZ0lJUzFnbmhwU0N5Q2d3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TVRBNU1ERXlNalEzTlRKYUZ3MHlNakE1TURFeU1qUTNOVFZhTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQW52eXoxc1R1SXRpKzE3WmQKVVRXTFVxMUxIL2VJN01lMkI0K2ZNZlhKSStlM2xCVnp5RXpIV0ZOR1phM2JYbkYvS0VJaDJRcmpOcXh0bGswSgpIOW83dUtVZmRyVjhNL3IzZmxidUN1VG9lZnN3UFROQmJhbGladzVPRXl0VWV6V3ZxK3VUZzFmeExZVUl6Zk4xCldxMzhiU2pjYlhQa3Q3UWJZVThqUEpMMmlKalBlbVFRN1FnTW9pUmlsNXM2TzRCZnNYbzNCbDNrdUY0VDlCK1MKVzE2VmpQTnRMQ0pxQW1ENEt1ZWdBcWl3RHdDNFVScjhNbDhJaHJmL2FzT2JTZnVqTG5HL1Npd2V6dnJ4bHJnUgo0QVBlNjFSOU1RZFFjaldsT1Z2TXQrSXhlSnlrbWdmeHJsNFJmbytFOWVNK0VTNzFHaVhnQmtycFp0NGxQWURsClllSVZQd0lEQVFBQm8xWXdWREFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RBWURWUjBUQVFIL0JBSXdBREFmQmdOVkhTTUVHREFXZ0JTaEZmM1VnRVFac2w2Ly9xa1JJS0Y4Z0IrMApPakFOQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBa0ZqdDJPNW5ZQUkxRHRrZnh6R1RPbFdGT1F3b3FKelBHQXJSCmRoTnFXL3JjUlhyYkgzZ3FHaXF4cmQ2anczblJiYThCRWxOazE0YUtYWGVYRnU0U0YyYTJCY3RzKzhkNE9VSkwKeU1pUVBpN0g2Q3RrQ0o2QzRCZDU4Vk5XaVM0YVg4b0ExQWloZWp0cURRc2U2MCtna2JoSlJwdnM0WGRVUkNTdgpFL3NqZWgvc1JIVjBJYWNrNzlTVEduSUdlVVUrbUxwVlF1bHZkd1lkVDhXK08zMkpRbFk1Z3pTZllFMkI2YjB4Ci9TK1dORU9QTzhhaTlmQkQ5cWJ1dWdRd2wzSkNYT005amZLV1gzOTBZZzhYcWhndEhuR0JDdlcwbjQxY0ZLUDgKQVFFdXRnbDNhQ0ZibWZFZ2Z3cWlUVFc3R3EzSklZSTZrZ3EwNGxUbVdKa1gvQnZmaXc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBbnZ5ejFzVHVJdGkrMTdaZFVUV0xVcTFMSC9lSTdNZTJCNCtmTWZYSkkrZTNsQlZ6CnlFekhXRk5HWmEzYlhuRi9LRUloMlFyak5xeHRsazBKSDlvN3VLVWZkclY4TS9yM2ZsYnVDdVRvZWZzd1BUTkIKYmFsaVp3NU9FeXRVZXpXdnErdVRnMWZ4TFlVSXpmTjFXcTM4YlNqY2JYUGt0N1FiWVU4alBKTDJpSmpQZW1RUQo3UWdNb2lSaWw1czZPNEJmc1hvM0JsM2t1RjRUOUIrU1cxNlZqUE50TENKcUFtRDRLdWVnQXFpd0R3QzRVUnI4Ck1sOElocmYvYXNPYlNmdWpMbkcvU2l3ZXp2cnhscmdSNEFQZTYxUjlNUWRRY2pXbE9Wdk10K0l4ZUp5a21nZngKcmw0UmZvK0U5ZU0rRVM3MUdpWGdCa3JwWnQ0bFBZRGxZZUlWUHdJREFRQUJBb0lCQUQzOHFPR0R4cFV2akxqdQpFVlFvWERuUDl3cHZxS01vK24vWUwybDdPd0VVeHk2bGJvOFo0RjgvbUtMc05pdU1kTmR0Y1dUK0tiaVhZZUxJCkJsYTA3N1ArTFZaTFRERzRGK2JhWGRWQmlxS0VuVG8vVWJNLzUyM20xZW9EYXR6ZkFhODJHajJMZkMwVFFXdUwKRUtaYVQ2RC8zWEdQVGcyUjIxc0ZUK2UrSlFEOGRnc25oNE9vVlQrTkRacC9kU0JHYXZNQTFZUmo0bFhwY1U5RAo5bW15ckxRZFlRcE56K1U4cGZKdHhIcXlGSWhOakZmK0JkNHdRdEhrN3NOODE4Um9JalZHV3RYeGVhZXFOMXVtCnFlWEhFNHVDRG5tYS9qTElLLzBRaWlMZTZ1WGVTMk1udG1UUjJ1d0paOWh5V3NsYnlTb2oyQmNONVBaaHpGK3kKMUtyZEFZRUNnWUVBenNEeUFtZ1dUUXI5M083ZnlSR1U5azBad01LRFVSK25Lb0xQcUNhSmxQeE4xaG1zTkJmWApKWURsZ3cwVTk5R1lmRGJZUTdjS3BaRE8xWHZpWTI4K1UxY21nM2xVMVFVOTdFR0N3ejVxMnNjUFY0SDBhZmxnCmNUQko5dGo1ZTkzVS9sVDFpd0M1eEFONlpjektTbzhYSytNQ29nUkEyeEFZZjFJZnJTZmhoVzBDZ1lFQXhOc2kKQ2oxS29FQzV0TjlEaW41eFQzMUVBTjlwVmtONkZlcy9nZC9JSFREWXJLSytaMnNpVVNhR1NyaHYwZkc1ZGVwagpIMjdEeVF6cW1aUUlpaE44cFB5TzRSOXMya21la3RISUZqMjRnSUpQZDNzS3BaS1QwQjJmZUErTXVCOFlsclRGCk0ycTJ2V1JHeHFmMERMZmpWNm5JVkZkQ1hJWFZLMjlRcWprdkZkc0NnWUFmUGRxVDhJU0dLY1lJajNQelh4dkMKU0E0L0tXVk1hZHNKdW5DRWVTWkxCQUVDL0NnZ1N3WHduZFNRZy9hS0ovckJza3ZsbDVBZFNvOW1oT3pGbDdhMApRelFIbzlya3dZRUU1VFZNS1c5ZUZieEV2ZGRmK0JYUnBMbFllcHJnVTdudW9Jbmw4anNmMm1LeFpVdWdEcFV5CnhYL05XWlV2UlBSZXNOc21nQ004MVFLQmdRQ0xSOFFJM0o3TlRaNVhNOVJVeSt1ZDR6SlhMN3NXMXIwdGZ2bTcKQ1R0TU5BQkovUWVjb25kd1ZVS1U0WFAwWmdQalF3Z0krRlM4RGxCNmd2dWJ2ZmZsdisvVHBtbGM5Tk9tYTVrVwo2MnA4T2piQmdhUGh6QmliR2lwM1J3RTRVSUFVT1NpQm5aSlg0L2dUbkVlWExCQkZPUkpOWWtQSXRNUkRiQW4xCnRtbnpHd0tCZ0J3NHhLanNEUUozcCtxWW50cTdtVzhLS2hPWTFMRWczOVJ4Snd1aEord0VSZUh5TGhIcEU5SFkKUndxbUVCYjdvY2dDcmV6bWR5WndUSXZkMGEzaStBbWpucTd1QU1DUFpNUjU0a2FkNUpmZmVib0FzbXcwSW5aeApvVGltQXNya3BmRlVxZzZsSVBIMEtuUEVTVWQxQlJLS2I5dTUzTWpwZEZiVkhWZVZhVEtlCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==네임스페이스와 서비스 어카운트 생성 후 확인
- 파드 기동 시 서비스 어카운트 한 개가 할당되며, 서비스 어카운트 기반 인증/인가를 함, 미지정 시 기본 서비스 어카운트가 할당
- 서비스 어카운트에 자동 생성된 시크릿에 저장된 토큰으로 쿠버네티스 API에 대한 인증 정보로 사용 할 수 있다 ← 1.23 이전 버전의 경우에만 해당
# 네임스페이스(Namespace, NS) 생성 및 확인
kubectl create namespace dev-team
kubectl create ns infra-team
# 네임스페이스 확인
kubectl get ns
# 네임스페이스에 각각 서비스 어카운트 생성 : serviceaccounts 약자(=sa)
kubectl create sa dev-k8s -n dev-team
kubectl create sa infra-k8s -n infra-team
# 서비스 어카운트 정보 확인
kubectl get sa -n dev-team
kubectl get sa dev-k8s -n dev-team -o yaml | yh
kubectl get sa -n infra-team
kubectl get sa infra-k8s -n infra-team -o yaml | yh(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl get ns
NAME STATUS AGE
default Active 63m
dev-team Active 43s
infra-team Active 16s
kube-node-lease Active 63m
kube-public Active 63m
kube-system Active 63m
monitoring Active 4m30s
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl get sa dev-k8s -n dev-team -o yaml | yh
apiVersion: v1
kind: ServiceAccount
metadata:
creationTimestamp: "2023-06-03T12:44:41Z"
name: dev-k8s
namespace: dev-team
resourceVersion: "13468"
uid: 1234567890~1234-1234-1234-1234567890~
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl get sa -n infra-team
NAME SECRETS AGE
default 0 2m35s
infra-k8s 0 98s
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl get sa infra-k8s -n infra-team -o yaml | yh
apiVersion: v1
kind: ServiceAccount
metadata:
creationTimestamp: "2023-06-03T12:44:42Z"
name: infra-k8s
namespace: infra-team
resourceVersion: "13472"
uid: 1234567890~1234-1234-1234-1234567890~
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
# dev-k8s 서비스어카운트의 토큰 정보 확인
DevTokenName=$(kubectl get sa dev-k8s -n dev-team -o jsonpath="{.secrets[0].name}")
DevToken=$(kubectl get secret -n dev-team $DevTokenName -o jsonpath="{.data.token}" | base64 -d)
echo $DevToken(primedo_eks@myeks:default) [root@myeks-bastion ~]# DevTokenName=$(kubectl get sa dev-k8s -n dev-team -o jsonpath="{.secrets[0].name}")
(primedo_eks@myeks:default) [root@myeks-bastion ~]# DevToken=$(kubectl get secret -n dev-team $DevTokenName -o jsonpath="{.data.token}" | base64 -d)
(primedo_eks@myeks:default) [root@myeks-bastion ~]# echo $DevToken
서비스 어카운트를 지정하여 파드 생성 후 권한 테스트
# 각각 네임스피이스에 kubectl 파드 생성 - 컨테이너이미지
# docker run --rm --name kubectl -v /path/to/your/kube/config:/.kube/config bitnami/kubectl:latest
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
name: dev-kubectl
namespace: dev-team
spec:
serviceAccountName: dev-k8s
containers:
- name: kubectl-pod
image: bitnami/kubectl:1.24.10
command: ["tail"]
args: ["-f", "/dev/null"]
terminationGracePeriodSeconds: 0
EOF
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
name: infra-kubectl
namespace: infra-team
spec:
serviceAccountName: infra-k8s
containers:
- name: kubectl-pod
image: bitnami/kubectl:1.24.10
command: ["tail"]
args: ["-f", "/dev/null"]
terminationGracePeriodSeconds: 0
EOF
# 확인
kubectl get pod -A
kubectl get pod -o dev-kubectl -n dev-team -o yaml
serviceAccount: dev-k8s
...
kubectl get pod -o infra-kubectl -n infra-team -o yaml
serviceAccount: infra-k8s
...
# 파드에 기본 적용되는 서비스 어카운트(토큰) 정보 확인
kubectl exec -it dev-kubectl -n dev-team -- ls /run/secrets/kubernetes.io/serviceaccount
kubectl exec -it dev-kubectl -n dev-team -- cat /run/secrets/kubernetes.io/serviceaccount/token
kubectl exec -it dev-kubectl -n dev-team -- cat /run/secrets/kubernetes.io/serviceaccount/namespace
kubectl exec -it dev-kubectl -n dev-team -- cat /run/secrets/kubernetes.io/serviceaccount/ca.crt
# 각각 파드로 Shell 접속하여 정보 확인 : 단축 명령어(alias) 사용
alias k1='kubectl exec -it dev-kubectl -n dev-team -- kubectl'
alias k2='kubectl exec -it infra-kubectl -n infra-team -- kubectl'
# 권한 테스트
k1 get pods # kubectl exec -it dev-kubectl -n dev-team -- kubectl get pods 와 동일한 실행 명령이다!
k1 run nginx --image nginx:1.20-alpine
k1 get pods -n kube-system
k2 get pods # kubectl exec -it infra-kubectl -n infra-team -- kubectl get pods 와 동일한 실행 명령이다!
k2 run nginx --image nginx:1.20-alpine
k2 get pods -n kube-system
# (옵션) kubectl auth can-i 로 kubectl 실행 사용자가 특정 권한을 가졌는지 확인
k1 auth can-i get pods
no(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl get pod -o dev-kubectl -n dev-team -o yaml
apiVersion: v1
items:
- apiVersion: v1
kind: Pod
metadata:
annotations:
kubernetes.io/psp: eks.privileged
creationTimestamp: "2023-06-03T13:00:00Z"
name: dev-kubectl
namespace: dev-team
resourceVersion: "17362"
uid: 1234567890~1234-1234-1234-1234567890~
spec:
containers:
- args:
- -f
- /dev/null
command:
- tail
image: bitnami/kubectl:1.24.10
imagePullPolicy: IfNotPresent
name: kubectl-pod
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-njtbz
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: ip-192-168-3-207.ap-northeast-2.compute.internal
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: dev-k8s
serviceAccountName: dev-k8s
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl get pod -o infra-kubectl -n infra-team -o yaml
apiVersion: v1
items:
- apiVersion: v1
kind: Pod
metadata:
annotations:
kubernetes.io/psp: eks.privileged
creationTimestamp: "2023-06-03T13:00:12Z"
name: infra-kubectl
namespace: infra-team
resourceVersion: "17413"
uid: 1234567890~1234-1234-1234-1234567890~
spec:
containers:
- args:
- -f
- /dev/null
command:
- tail
image: bitnami/kubectl:1.24.10
imagePullPolicy: IfNotPresent
name: kubectl-pod
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-wmc92
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: ip-192-168-2-123.ap-northeast-2.compute.internal
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: infra-k8s
serviceAccountName: infra-k8s
각각 네임스페이스에 롤(Role)를 생성 후 서비스 어카운트 바인딩
- 롤(Role) : apiGroups 와 resources 로 지정된 리소스에 대해 verbs** 권한을 인가
- 실행 가능한 조작(verbs)* : (모두 처리), create(생성), delete(삭제), get(조회), list(목록조회), patch(일부업데이트), update(업데이트), watch(변경감시)
# 각각 네임스페이스내의 모든 권한에 대한 롤 생성
cat <<EOF | kubectl create -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: role-dev-team
namespace: dev-team
rules:
- apiGroups: ["*"]
resources: ["*"]
verbs: ["*"]
EOF
cat <<EOF | kubectl create -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: role-infra-team
namespace: infra-team
rules:
- apiGroups: ["*"]
resources: ["*"]
verbs: ["*"]
EOF
# 롤 확인
kubectl get roles -n dev-team
kubectl get roles -n infra-team
kubectl get roles -n dev-team -o yaml
kubectl describe roles role-dev-team -n dev-team
...
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
*.* [] [] [*]
# 롤바인딩 생성 : '서비스어카운트 <-> 롤' 간 서로 연동
cat <<EOF | kubectl create -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: roleB-dev-team
namespace: dev-team
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: role-dev-team
subjects:
- kind: ServiceAccount
name: dev-k8s
namespace: dev-team
EOF
cat <<EOF | kubectl create -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: roleB-infra-team
namespace: infra-team
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: role-infra-team
subjects:
- kind: ServiceAccount
name: infra-k8s
namespace: infra-team
EOF
# 롤바인딩 확인
kubectl get rolebindings -n dev-team
kubectl get rolebindings -n infra-team
kubectl get rolebindings -n dev-team -o yaml
kubectl describe rolebindings roleB-dev-team -n dev-team
...
Role:
Kind: Role
Name: role-dev-team
Subjects:
Kind Name Namespace
---- ---- ---------
ServiceAccount dev-k8s dev-team(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl describe rolebindings roleB-dev-team -n dev-team
Name: roleB-dev-team
Labels: <none>
Annotations: <none>
Role:
Kind: Role
Name: role-dev-team
Subjects:
Kind Name Namespace
---- ---- ---------
ServiceAccount dev-k8s dev-team
(primedo_eks@myeks:default) [root@myeks-bastion ~]#서비스 어카운트를 지정하여 생성한 파드에서 다시 권한 테스트
# 각각 파드로 Shell 접속하여 정보 확인 : 단축 명령어(alias) 사용
alias k1='kubectl exec -it dev-kubectl -n dev-team -- kubectl'
alias k2='kubectl exec -it infra-kubectl -n infra-team -- kubectl'
# 권한 테스트
k1 get pods
k1 run nginx --image nginx:1.20-alpine
k1 get pods
k1 delete pods nginx
k1 get pods -n kube-system
k1 get nodes
k2 get pods
k2 run nginx --image nginx:1.20-alpine
k2 get pods
k2 delete pods nginx
k2 get pods -n kube-system
k2 get nodes
# (옵션) kubectl auth can-i 로 kubectl 실행 사용자가 특정 권한을 가졌는지 확인
k1 auth can-i get pods
yes(primedo_eks@myeks:default) [root@myeks-bastion ~]# k1 auth can-i get pods
yestag✍ EKS 인증/인가
동작 : 사용자/애플리케이션 → k8s 사용 시 ⇒ 인증은 AWS IAM, 인가는 K8S RBAC
RBAC 관련 krew 플러그인
# 설치
kubectl krew install access-matrix rbac-tool rbac-view rolesum
# Show an RBAC access matrix for server resources
kubectl access-matrix # Review access to cluster-scoped resources
kubectl access-matrix --namespace default # Review access to namespaced resources in 'default'
# RBAC Lookup by subject (user/group/serviceaccount) name
kubectl rbac-tool lookup
kubectl rbac-tool lookup system:masters
SUBJECT | SUBJECT TYPE | SCOPE | NAMESPACE | ROLE
+----------------+--------------+-------------+-----------+---------------+
system:masters | Group | ClusterRole | | cluster-admin
kubectl rbac-tool lookup system:nodes # eks:node-bootstrapper
kubectl rbac-tool lookup system:bootstrappers # eks:node-bootstrapper
kubectl describe ClusterRole eks:node-bootstrapper
# RBAC List Policy Rules For subject (user/group/serviceaccount) name
kubectl rbac-tool policy-rules
kubectl rbac-tool policy-rules -e '^system:.*'
# Generate ClusterRole with all available permissions from the target cluster
kubectl rbac-tool show
# Shows the subject for the current context with which one authenticates with the cluster
kubectl rbac-tool whoami
{Username: "kubernetes-admin",
UID: "aws-iam-authenticator:911283.....:AIDA5ILF2FJ......",
Groups: ["system:masters",
"system:authenticated"],
Extra: {accessKeyId: ["AKIA5ILF2FJ....."],
arn: ["arn:aws:iam::911283....:user/admin"],
canonicalArn: ["arn:aws:iam::911283....:user/admin"],
principalId: ["AIDA5ILF2FJ....."],
sessionName: [""]}}
# Summarize RBAC roles for subjects : ServiceAccount(default), User, Group
kubectl rolesum -h
kubectl rolesum aws-node -n kube-system
kubectl rolesum -k User system:kube-proxy
kubectl rolesum -k Group system:masters
# [터미널1] A tool to visualize your RBAC permissions
kubectl rbac-view
INFO[0000] Getting K8s client
INFO[0000] serving RBAC View and http://localhost:8800
## 이후 해당 작업용PC 공인 IP:8800 웹 접속
echo -e "RBAC View Web http://$(curl -s ipinfo.io/ip):8800"(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl rbac-tool lookup system:masters
W0603 22:30:28.585936 12238 warnings.go:67] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
SUBJECT | SUBJECT TYPE | SCOPE | NAMESPACE | ROLE
+----------------+--------------+-------------+-----------+---------------+
system:masters | Group | ClusterRole | | cluster-admin
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl rbac-tool whoami
{Username: "kubernetes-admin",
UID: "aws-iam-authenticator:account-id:123456789~",
Groups: ["system:masters",
"system:authenticated"],
Extra: {accessKeyId: ["1234567890~~~~~~~~~~~"],
arn: ["arn:aws:iam::account-id:user/primedo_eks"],
canonicalArn: ["arn:aws:iam::account-id:user/primedo_eks"],
principalId: ["1234567890~~~"],
sessionName: [""]}}
(primedo_eks@myeks:default) [root@myeks-bastion ~]#(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl rolesum aws-node -n kube-system
ServiceAccount: kube-system/aws-node
Secrets:
Policies:
• [CRB] */aws-node ⟶ [CR] */aws-node
Resource Name Exclude Verbs G L W C U P D DC
*.extensions [*] [-] [-] ✖ ✔ ✔ ✖ ✖ ✖ ✖ ✖
eniconfigs.crd.k8s.amazonaws.com [*] [-] [-] ✔ ✔ ✔ ✖ ✖ ✖ ✖ ✖
events.[,events.k8s.io] [*] [-] [-] ✖ ✔ ✖ ✔ ✖ ✔ ✖ ✖
namespaces [*] [-] [-] ✔ ✔ ✔ ✖ ✖ ✖ ✖ ✖
nodes [*] [-] [-] ✔ ✔ ✔ ✖ ✔ ✖ ✖ ✖
pods [*] [-] [-] ✔ ✔ ✔ ✖ ✖ ✖ ✖ ✖
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl rolesum -k User system:kube-proxy
User: system:kube-proxy
Policies:
• [CRB] */system:node-proxier ⟶ [CR] */system:node-proxier
Resource Name Exclude Verbs G L W C U P D DC
endpoints [*] [-] [-] ✖ ✔ ✔ ✖ ✖ ✖ ✖ ✖
endpointslices.discovery.k8s.io [*] [-] [-] ✖ ✔ ✔ ✖ ✖ ✖ ✖ ✖
events.[,events.k8s.io] [*] [-] [-] ✖ ✖ ✖ ✔ ✔ ✔ ✖ ✖
nodes [*] [-] [-] ✔ ✔ ✔ ✖ ✖ ✖ ✖ ✖
services [*] [-] [-] ✖ ✔ ✔ ✖ ✖ ✖ ✖ ✖
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl rolesum -k Group system:masters
Group: system:masters
Policies:
• [CRB] */cluster-admin ⟶ [CR] */cluster-admin
Resource Name Exclude Verbs G L W C U P D DC
*.* [*] [-] [-] ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# echo -e "RBAC View Web http://$(curl -s ipinfo.io/ip):8800"
RBAC View Web http://13.124.208.30:8800
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
RBAC View Web http://13.124.208.30:8800
rag인증/인가 완벽 분석 해보기
kubectl 명령 → aws eks get-token → EKS Service endpoint(STS)에 토큰 요청 ⇒ 응답값 디코드(Pre-Signed URL 이며 GetCallerIdentity..) - 링크
- STS** Security Token Service : AWS 리소스에 대한 액세스를 제어할 수 있는 임시 보안 자격 증명(STS)을 생성하여 신뢰받는 사용자에게 제공할 수 있음
https://ap-northeast-2.console.aws.amazon.com/cloudtrail/home?region=ap-northeast-2#/events?EventSource=sts.amazonaws.com
AWS CLI버전 1.16.156 이상에서는 별도 aws-iam-authenticator 설치 없이 aws eks get-token으로 사용 가능 - Docs
# sts caller id의 ARN 확인
aws sts get-caller-identity --query Arn
"arn:aws:iam::<자신의 Account ID>:user/admin"
# kubeconfig 정보 확인
cat ~/.kube/config | yh
...
- name: admin@myeks.ap-northeast-2.eksctl.io
user:
exec:
apiVersion: client.authentication.k8s.io/v1beta1
args:
- eks
- get-token
- --output
- json
- --cluster-name
- myeks
- --region
- ap-northeast-2
command: aws
env:
- name: AWS_STS_REGIONAL_ENDPOINTS
value: regional
interactiveMode: IfAvailable
provideClusterInfo: false
# Get a token for authentication with an Amazon EKS cluster.
# This can be used as an alternative to the aws-iam-authenticator.
aws eks get-token help
# 임시 보안 자격 증명(토큰)을 요청 : expirationTimestamp 시간경과 시 토큰 재발급됨
aws eks get-token --cluster-name $CLUSTER_NAME | jq
aws eks get-token --cluster-name $CLUSTER_NAME | jq -r '.status.token'kubectl의 Client-Go 라이브러리는 Pre-Signed URL을 Bearer Token으로 EKS API Cluster Endpoint로 요청을 보냄
tag
PAYLOAD의 값을 URL Decode Online 에서 DECODE로 확인 - 링크
https://sts.ap-northeast-2.amazonaws.com/?
Action=GetCallerIdentity&
Version=2011-06-15&
X-Amz-Algorithm=AWS4-HMAC-SHA256&
X-Amz-Credential=AKIA5ILF.../20230525/ap-northeast-2/sts/aws4_request&
X-Amz-Date=20230525T120720Z&
X-Amz-Expires=60&
X-Amz-SignedHeaders=host;x-k8s-aws-id&
X-Amz-Signature=6e09b846da702767f38c78831986cb558.....EKS API는 Token Review 를 Webhook token authenticator에 요청 ⇒ (STS GetCallerIdentity 호출) AWS IAM 해당 호출 인증 완료 후 User/Role에 대한 ARN 반환
- 참고로 Webhook token authenticator 는 aws-iam-authenticator 를 사용 - 링크 Github
https://ap-northeast-2.console.aws.amazon.com/cloudtrail/home?region=ap-northeast-2#/events?EventName=GetCallerIdentity# tokenreviews api 리소스 확인
kubectl api-resources | grep authentication
tokenreviews authentication.k8s.io/v1 false TokenReview
# List the fields for supported resources.
kubectl explain tokenreviews
...
DESCRIPTION:
TokenReview attempts to authenticate a token to a known user. Note:
TokenReview requests may be cached by the webhook token authenticator
plugin in the kube-apiserver.(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl api-resources | grep authentication
tokenreviews authentication.k8s.io/v1 false TokenReview
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl explain tokenreviews
KIND: TokenReview
VERSION: authentication.k8s.io/v1
DESCRIPTION:
TokenReview attempts to authenticate a token to a known user. Note:
TokenReview requests may be cached by the webhook token authenticator
plugin in the kube-apiserver.이제 쿠버네티스 RBAC 인가를 처리합니다. 개인적인 생각이지만 플랫폼간 인증 이외에 인가까지 처리 통합은 쉽지 않은 것 같습니다 okta도
- 해당 IAM User/Role 확인이 되면 k8s aws-auth configmap에서 mapping 정보를 확인하게 됩니다.
- aws-auth 컨피그맵에 'IAM 사용자, 역할 arm, K8S 오브젝트' 로 권한 확인 후 k8s 인가 허가가 되면 최종적으로 동작 실행을 합니다.
- 참고로 EKS를 생성한 IAM principal은 aws-auth 와 상관없이 kubernetes-admin Username으로 system:masters 그룹에 권한을 가짐 - 링크
# Webhook api 리소스 확인
kubectl api-resources | grep Webhook
mutatingwebhookconfigurations admissionregistration.k8s.io/v1 false MutatingWebhookConfiguration
validatingwebhookconfigurations admissionregistration.k8s.io/v1 false ValidatingWebhookConfiguration
# validatingwebhookconfigurations 리소스 확인
kubectl get validatingwebhookconfigurations
NAME WEBHOOKS AGE
eks-aws-auth-configmap-validation-webhook 1 50m
vpc-resource-validating-webhook 2 50m
aws-load-balancer-webhook 3 8m27s
kubectl get validatingwebhookconfigurations eks-aws-auth-configmap-validation-webhook -o yaml | kubectl neat | yh
# aws-auth 컨피그맵 확인
kubectl get cm -n kube-system aws-auth -o yaml | kubectl neat | yh
apiVersion: v1
kind: ConfigMap
metadata:
name: aws-auth
namespace: kube-system
data:
mapRoles: |
- groups:
- system:bootstrappers
- system:nodes
rolearn: arn:aws:iam::91128.....:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-1OS1WSTV0YB9X
username: system:node:{{EC2PrivateDNSName}}
#---<아래 생략(추정), ARN은 EKS를 설치한 IAM User , 여기 있었을경우 만약 실수로 삭제 시 복구가 가능했을까?---
mapUsers: |
- groups:
- system:masters
userarn: arn:aws:iam::111122223333:user/admin
username: kubernetes-admin
# EKS 설치한 IAM User 정보 >> system:authenticated는 어떤 방식으로 추가가 되었는지 궁금???
kubectl rbac-tool whoami
{Username: "kubernetes-admin",
UID: "aws-iam-authenticator:9112834...:AIDA5ILF2FJIR2.....",
Groups: ["system:masters",
"system:authenticated"],
...
# system:masters , system:authenticated 그룹의 정보 확인
kubectl rbac-tool lookup system:masters
kubectl rbac-tool lookup system:authenticated
kubectl rolesum -k Group system:masters
kubectl rolesum -k Group system:authenticated
# system:masters 그룹이 사용 가능한 클러스터 롤 확인 : cluster-admin
kubectl describe clusterrolebindings.rbac.authorization.k8s.io cluster-admin
Name: cluster-admin
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
Role:
Kind: ClusterRole
Name: cluster-admin
Subjects:
Kind Name Namespace
---- ---- ---------
Group system:masters
# cluster-admin 의 PolicyRule 확인 : 모든 리소스 사용 가능!
kubectl describe clusterrole cluster-admin
Name: cluster-admin
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
*.* [] [] [*]
[*] [] [*]
# system:authenticated 그룹이 사용 가능한 클러스터 롤 확인
kubectl describe ClusterRole system:discovery
kubectl describe ClusterRole system:public-info-viewer
kubectl describe ClusterRole system:basic-user
kubectl describe ClusterRole eks:podsecuritypolicy:privileged(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl api-resources | grep Webhook
mutatingwebhookconfigurations admissionregistration.k8s.io/v1 false MutatingWebhookConfiguration
validatingwebhookconfigurations admissionregistration.k8s.io/v1 false ValidatingWebhookConfiguration
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl get validatingwebhookconfigurations
NAME WEBHOOKS AGE
aws-load-balancer-webhook 3 164m
eks-aws-auth-configmap-validation-webhook 1 3h40m
kube-prometheus-stack-admission 1 161m
vpc-resource-validating-webhook 2 3h40m
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl get validatingwebhookconfigurations eks-aws-auth-configmap-validation-webhook -o yaml | kubectl neat | yh
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
name: eks-aws-auth-configmap-validation-webhook
webhooks:
- admissionReviewVersions:
- v1
clientConfig:
caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMvakNDQWVhZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJek1EWXdNekV4TkRB@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
HMEdzeUtQME5NRnlQOEtxZEN3cTNMbXJ3R2RJWnF4enYzbFc0OTNDcXliek9VVVkKc2h3PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
url: https://127.0.0.1:21375/validate
failurePolicy: Ignore
matchPolicy: Equivalent
name: eks-aws-auth-configmap-validation-webhook.amazonaws.com
namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
rules:
- apiGroups:
- ""
apiVersions:
- v1
operations:
- UPDATE
resources:
- configmaps
scope: '*'
sideEffects: None
timeoutSeconds: 5
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl get cm -n kube-system aws-auth -o yaml | kubectl neat | yh
apiVersion: v1
data:
mapRoles: |
- groups:
- system:bootstrappers
- system:nodes
rolearn: arn:aws:iam::account-id:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-1ANECPD8VGOD2
username: system:node:{{EC2PrivateDNSName}}
kind: ConfigMap
metadata:
name: aws-auth
namespace: kube-system
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl rbac-tool whoami
{Username: "kubernetes-admin",
UID: "aws-iam-authenticator:account-id:1234567890~~~~",
Groups: ["system:masters",
"system:authenticated"],
Extra: {accessKeyId: ["1234567890~~~~"],
arn: ["arn:aws:iam::account-id:user/primedo_eks"],
canonicalArn: ["arn:aws:iam::account-id:user/primedo_eks"],
principalId: ["1234567890~~~~"],
sessionName: [""]}}
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl rbac-tool whoami
{Username: "kubernetes-admin",
UID: "aws-iam-authenticator:account-id:1234567890~~~~",
Groups: ["system:masters",
"system:authenticated"],
Extra: {accessKeyId: ["1234567890~~~~"],
arn: ["arn:aws:iam::account-id:user/primedo_eks"],
canonicalArn: ["arn:aws:iam::account-id:user/primedo_eks"],
principalId: ["1234567890~~~~"],
sessionName: [""]}}
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl rbac-tool lookup system:masters
W0604 00:23:30.399579 17805 warnings.go:67] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
SUBJECT | SUBJECT TYPE | SCOPE | NAMESPACE | ROLE
+----------------+--------------+-------------+-----------+---------------+
system:masters | Group | ClusterRole | | cluster-admin
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl rbac-tool lookup system:authenticated
W0604 00:23:37.975684 17858 warnings.go:67] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
SUBJECT | SUBJECT TYPE | SCOPE | NAMESPACE | ROLE
+----------------------+--------------+-------------+-----------+----------------------------------+
system:authenticated | Group | ClusterRole | | system:public-info-viewer
system:authenticated | Group | ClusterRole | | system:basic-user
system:authenticated | Group | ClusterRole | | system:discovery
system:authenticated | Group | ClusterRole | | eks:podsecuritypolicy:privileged
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl rolesum -k Group system:masters
Group: system:masters
Policies:
• [CRB] */cluster-admin ⟶ [CR] */cluster-admin
Resource Name Exclude Verbs G L W C U P D DC
*.* [*] [-] [-] ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl rolesum -k Group system:authenticated
W0604 00:23:52.773174 17964 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
Group: system:authenticated
Policies:
• [CRB] */eks:podsecuritypolicy:authenticated ⟶ [CR] */eks:podsecuritypolicy:privileged
Name PRIV RO-RootFS Volumes Caps SELinux RunAsUser FSgroup SUPgroup
eks.privileged True False [*] [*] RunAsAny RunAsAny RunAsAny RunAsAny
• [CRB] */system:basic-user ⟶ [CR] */system:basic-user
Resource Name Exclude Verbs G L W C U P D DC
selfsubjectaccessreviews.authorization.k8s.io [*] [-] [-] ✖ ✖ ✖ ✔ ✖ ✖ ✖ ✖
selfsubjectrulesreviews.authorization.k8s.io [*] [-] [-] ✖ ✖ ✖ ✔ ✖ ✖ ✖ ✖
• [CRB] */system:discovery ⟶ [CR] */system:discovery
• [CRB] */system:public-info-viewer ⟶ [CR] */system:public-info-viewer
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl describe clusterrolebindings.rbac.authorization.k8s.io cluster-admin
Name: cluster-admin
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
Role:
Kind: ClusterRole
Name: cluster-admin
Subjects:
Kind Name Namespace
---- ---- ---------
Group system:masters
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl describe clusterrole cluster-admin
Name: cluster-admin
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
*.* [] [] [*]
[*] [] [*]
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl describe ClusterRole system:discovery
Name: system:discovery
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
[/api/*] [] [get]
[/api] [] [get]
[/apis/*] [] [get]
[/apis] [] [get]
[/healthz] [] [get]
[/livez] [] [get]
[/openapi/*] [] [get]
[/openapi] [] [get]
[/readyz] [] [get]
[/version/] [] [get]
[/version] [] [get]
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl describe ClusterRole system:public-info-viewer
Name: system:public-info-viewer
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
[/healthz] [] [get]
[/livez] [] [get]
[/readyz] [] [get]
[/version/] [] [get]
[/version] [] [get]
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl describe ClusterRole system:basic-user
Name: system:basic-user
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
selfsubjectaccessreviews.authorization.k8s.io [] [] [create]
selfsubjectrulesreviews.authorization.k8s.io [] [] [create]
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl describe ClusterRole eks:podsecuritypolicy:privileged
Name: eks:podsecuritypolicy:privileged
Labels: eks.amazonaws.com/component=pod-security-policy
kubernetes.io/cluster-service=true
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
podsecuritypolicies.policy [] [eks.privileged] [use]
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
데브옵스 신입 사원을 위한 myeks-bastion-2에 설정 해보기
1.[myeks-bastion] testuser 사용자 생성
# testuser 사용자 생성
aws iam create-user --user-name testuser
# 사용자에게 프로그래밍 방식 액세스 권한 부여
aws iam create-access-key --user-name testuser
{
"AccessKey": {
"UserName": "testuser",
"AccessKeyId": "AKIA5ILF2##",
"Status": "Active",
"SecretAccessKey": "TxhhwsU8##",
"CreateDate": "2023-05-23T07:40:09+00:00"
}
}
# testuser 사용자에 정책을 추가
aws iam attach-user-policy --policy-arn arn:aws:iam::aws:policy/AdministratorAccess --user-name testuser
# get-caller-identity 확인
aws sts get-caller-identity --query Arn
"arn:aws:iam::911283464785:user/admin"
# EC2 IP 확인 : myeks-bastion-EC2-2 PublicIPAdd 확인
aws ec2 describe-instances --query "Reservations[*].Instances[*].{PublicIPAdd:PublicIpAddress,PrivateIPAdd:PrivateIpAddress,InstanceName:Tags[?Key=='Name']|[0].Value,Status:State.Name}" --filters Name=instance-state-name,Values=running --output table(primedo_eks@myeks:default) [root@myeks-bastion ~]# aws ec2 describe-instances --query "Reservations[*].Instances[*].{PublicIPAdd:PublicIpAddress,PrivateIPAdd:PrivateIpAddress,InstanceName:Tags[?Key=='Name']|[0].Value,Status:State.Name}" --filters Name=instance-state-name,Values=running --output table
----------------------------------------------------------------------
| DescribeInstances |
+---------------------+-----------------+-----------------+----------+
| InstanceName | PrivateIPAdd | PublicIPAdd | Status |
+---------------------+-----------------+-----------------+----------+
| myeks-ng1-Node | 192.168.3.207 | 13.209.89.166 | running |
| myeks-ng1-Node | 192.168.2.123 | 3.35.5.177 | running |
| myeks-ng1-Node | 192.168.1.73 | 43.202.41.248 | running |
| myeks-bastion-EC2 | 192.168.1.100 | 13.124.208.30 | running |
| myeks-bastion-EC2-2| 192.168.1.200 | 43.202.44.138 | running |
+---------------------+-----------------+-----------------+----------+
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#- [myeks-bastion-2] testuser 자격증명 설정 및 확인
# get-caller-identity 확인 >> 왜 안될까요?
aws sts get-caller-identity --query Arn
# testuser 자격증명 설정
aws configure
AWS Access Key ID [None]: AKIA5ILF2F...
AWS Secret Access Key [None]: ePpXdhA3cP....
Default region name [None]: ap-northeast-2
# get-caller-identity 확인
aws sts get-caller-identity --query Arn
"arn:aws:iam::911283464785:user/testuser"
# kubectl 시도 >> testuser도 AdministratorAccess 권한을 가지고 있는데, 실패 이유는?
kubectl get node -v6
ls ~/.kube[root@myeks-bastion-2 ~]# kubectl get node -v6
I0604 00:31:22.513427 2372 round_trippers.go:553] GET http://localhost:8080/api?timeout=32s in 0 milliseconds
E0604 00:31:22.513530 2372 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
I0604 00:31:22.513549 2372 cached_discovery.go:120] skipped caching discovery info due to Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused3.[myeks-bastion] testuser에 system:masters 그룹 부여로 EKS 관리자 수준 권한 설정
# 방안1 : eksctl 사용 >> iamidentitymapping 실행 시 aws-auth 컨피그맵 작성해줌
# Creates a mapping from IAM role or user to Kubernetes user and groups
eksctl create iamidentitymapping --cluster $CLUSTER_NAME --username testuser --group system:masters --arn arn:aws:iam::$ACCOUNT_ID:user/testuser
# 확인
kubectl get cm -n kube-system aws-auth -o yaml | kubectl neat | yh
...
# 방안2 : 아래 edit로 mapUsers 내용 직접 추가!
kubectl edit cm -n kube-system aws-auth
---
apiVersion: v1
data:
mapRoles: |
- groups:
- system:bootstrappers
- system:nodes
rolearn: arn:aws:iam::911283464785:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-LHQ7DWHQQRZJ
username: system:node:{{EC2PrivateDNSName}}
mapUsers: |
- groups:
- system:masters
userarn: arn:aws:iam::911283464785:user/testuser
username: testuser
...
# 확인 : 기존에 있는 role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-YYYYY 는 어떤 역할/동작을 하는 걸까요?
eksctl get iamidentitymapping --cluster $CLUSTER_NAME
ARN USERNAME GROUPS ACCOUNT
arn:aws:iam::911283464785:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-LHQ7DWHQQRZJ system:node:{{EC2PrivateDNSName}} system:bootstrappers,system:nodes
arn:aws:iam::911283464785:user/testuser testuser system:masters4.[myeks-bastion-2] testuser kubeconfig 생성 및 kubectl 사용 확인
# testuser kubeconfig 생성 >> aws eks update-kubeconfig 실행이 가능한 이유는?, 3번 설정 후 약간의 적용 시간 필요
aws eks update-kubeconfig --name $CLUSTER_NAME --user-alias testuser
# 첫번째 bastic ec2의 config와 비교해보자
cat ~/.kube/config | yh
# kubectl 사용 확인
kubectl ns default
kubectl get node -v6
# rbac-tool 후 확인 >> 기존 계정과 비교해보자 >> system:authenticated 는 system:masters 설정 시 따라오는 것 같은데, 추가 동작 원리는 모르겠네요???
kubectl krew install rbac-tool && kubectl rbac-tool whoami
{Username: "testuser",
UID: "aws-iam-authenticator:911283464785:AIDA5ILF2FJIV65KG6RBM",
Groups: ["system:masters",
"system:authenticated"],
Extra: {accessKeyId: ["1234567890~~~~"],
arn: ["arn:aws:iam::911283464785:user/testuser"],
canonicalArn: ["arn:aws:iam::911283464785:user/testuser"],
...(**testuser**:default) [root@myeks-bastion-2 ~]# kubectl krew install rbac-tool && kubectl rbac-tool whoami
Updated the local copy of plugin index.
Installing plugin: rbac-tool
W0604 00:37:48.040514 2993 install.go:160] Skipping plugin "rbac-tool", it is already installed
{Username: "testuser",
UID: "aws-iam-authenticator:account-id:1234567890~~~~",
Groups: ["system:masters",
"system:authenticated"],
Extra: {accessKeyId: ["1234567890~~~~"],
arn: ["arn:aws:iam::account-id:user/testuser"],
canonicalArn: ["arn:aws:iam::account-id:user/testuser"],
principalId: ["1234567890~~~~"],
sessionName: [""]}}
(**testuser**:default) [root@myeks-bastion-2 ~]#- [myeks-bastion] testuser 의 Group 변경(system:masters → system:authenticated)으로 RBAC 동작 확인
# 방안2 : 아래 edit로 mapUsers 내용 직접 수정 system:authenticated
kubectl edit cm -n kube-system aws-auth
...
# 확인
eksctl get iamidentitymapping --cluster $CLUSTER_NAME(primedo_eks@myeks:default) [root@myeks-bastion ~]# eksctl get iamidentitymapping --cluster $CLUSTER_NAME
ARN USERNAME GROUPS ACCOUNT
arn:aws:iam::account-id:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-1ANECPD8VGOD2 system:node:{{EC2PrivateDNSName}} system:bootstrappers,system:nodes
arn:aws:iam::account-id:user/testuser testuser system:masters
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# eksctl get iamidentitymapping --cluster $CLUSTER_NAME
ARN USERNAME GROUPS ACCOUNT
arn:aws:iam::account-id:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-1ANECPD8VGOD2 system:node:{{EC2PrivateDNSName}} system:bootstrappers,system:nodes
arn:aws:iam::account-id:user/testuser testuser system:authenticated
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#- [myeks-bastion-2] testuser kubectl 사용 확인
# 시도
kubectl get node -v6
kubectl api-resources -v5(**testuser**:default) [root@myeks-bastion-2 ~]#
(**testuser**:default) [root@myeks-bastion-2 ~]# kubectl get node -v6
I0604 00:48:20.933555 3869 loader.go:373] Config loaded from file: /root/.kube/config
I0604 00:48:22.118793 3869 round_trippers.go:553] GET https://3C04D23FACA0F3ABCBDEE85966E90032.gr7.ap-northeast-2.eks.amazonaws.com/api/v1/nodes?limit=500 403 Forbidden in 1163 milliseconds
I0604 00:48:22.119084 3869 helpers.go:246] server response object: [{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "nodes is forbidden: User \"testuser\" cannot list resource \"nodes\" in API group \"\" at the cluster scope",
"reason": "Forbidden",
"details": {
"kind": "nodes"
},
"code": 403
}]
Error from server (Forbidden): nodes is forbidden: User "testuser" cannot list resource "nodes" in API group "" at the cluster scope
(**testuser**:default) [root@myeks-bastion-2 ~]# kubectl api-resources -v5
NAME SHORTNAMES APIVERSION NAMESPACED KIND
bindings v1 true Binding
componentstatuses cs v1 false ComponentStatus
configmaps cm v1 true ConfigMap
endpoints ep v1 true Endpoints
events ev v1 true Event
limitranges limits v1 true LimitRange
namespaces ns v1 false Namespace
nodes no v1 false Node
persistentvolumeclaims pvc v1 true PersistentVolumeClaim
persistentvolumes pv v1 false PersistentVolume
pods po v1 true Pod
podtemplates v1 true PodTemplate
replicationcontrollers rc v1 true ReplicationController
resourcequotas quota v1 true ResourceQuota
secrets v1 true Secret
serviceaccounts sa v1 true ServiceAccount
services svc v1 true Service
mutatingwebhookconfigurations admissionregistration.k8s.io/v1 false MutatingWebhookConfiguration
validatingwebhookconfigurations admissionregistration.k8s.io/v1 false ValidatingWebhookConfiguration
customresourcedefinitions crd,crds apiextensions.k8s.io/v1 false CustomResourceDefinition
apiservices apiregistration.k8s.io/v1 false APIService
controllerrevisions apps/v1 true ControllerRevision
daemonsets ds apps/v1 true DaemonSet
deployments deploy apps/v1 true Deployment
replicasets rs apps/v1 true ReplicaSet
statefulsets sts apps/v1 true StatefulSet
tokenreviews authentication.k8s.io/v1 false TokenReview
localsubjectaccessreviews authorization.k8s.io/v1 true LocalSubjectAccessReview
selfsubjectaccessreviews authorization.k8s.io/v1 false SelfSubjectAccessReview
selfsubjectrulesreviews authorization.k8s.io/v1 false SelfSubjectRulesReview
subjectaccessreviews authorization.k8s.io/v1 false SubjectAccessReview
horizontalpodautoscalers hpa autoscaling/v2 true HorizontalPodAutoscaler
cronjobs cj batch/v1 true CronJob
jobs batch/v1 true Job
certificatesigningrequests csr certificates.k8s.io/v1 false CertificateSigningRequest
leases coordination.k8s.io/v1 true Lease
eniconfigs crd.k8s.amazonaws.com/v1alpha1 false ENIConfig
endpointslices discovery.k8s.io/v1 true EndpointSlice
ingressclassparams elbv2.k8s.aws/v1beta1 false IngressClassParams
targetgroupbindings elbv2.k8s.aws/v1beta1 true TargetGroupBinding
events ev events.k8s.io/v1 true Event
flowschemas flowcontrol.apiserver.k8s.io/v1beta2 false FlowSchema
prioritylevelconfigurations flowcontrol.apiserver.k8s.io/v1beta2 false PriorityLevelConfiguration
nodes metrics.k8s.io/v1beta1 false NodeMetrics
pods metrics.k8s.io/v1beta1 true PodMetrics
alertmanagerconfigs amcfg monitoring.coreos.com/v1alpha1 true AlertmanagerConfig
alertmanagers am monitoring.coreos.com/v1 true Alertmanager
podmonitors pmon monitoring.coreos.com/v1 true PodMonitor
probes prb monitoring.coreos.com/v1 true Probe
prometheuses prom monitoring.coreos.com/v1 true Prometheus
prometheusrules promrule monitoring.coreos.com/v1 true PrometheusRule
servicemonitors smon monitoring.coreos.com/v1 true ServiceMonitor
thanosrulers ruler monitoring.coreos.com/v1 true ThanosRuler
ingressclasses networking.k8s.io/v1 false IngressClass
ingresses ing networking.k8s.io/v1 true Ingress
networkpolicies netpol networking.k8s.io/v1 true NetworkPolicy
runtimeclasses node.k8s.io/v1 false RuntimeClass
poddisruptionbudgets pdb policy/v1 true PodDisruptionBudget
podsecuritypolicies psp policy/v1beta1 false PodSecurityPolicy
clusterrolebindings rbac.authorization.k8s.io/v1 false ClusterRoleBinding
clusterroles rbac.authorization.k8s.io/v1 false ClusterRole
rolebindings rbac.authorization.k8s.io/v1 true RoleBinding
roles rbac.authorization.k8s.io/v1 true Role
priorityclasses pc scheduling.k8s.io/v1 false PriorityClass
csidrivers storage.k8s.io/v1 false CSIDriver
csinodes storage.k8s.io/v1 false CSINode
csistoragecapacities storage.k8s.io/v1 true CSIStorageCapacity
storageclasses sc storage.k8s.io/v1 false StorageClass
volumeattachments storage.k8s.io/v1 false VolumeAttachment
securitygrouppolicies sgp vpcresources.k8s.aws/v1beta1 true SecurityGroupPolicy
(**testuser**:default) [root@myeks-bastion-2 ~]#7.[myeks-bastion]에서 testuser IAM 맵핑 삭제
# testuser IAM 맵핑 삭제
eksctl delete iamidentitymapping --cluster $CLUSTER_NAME --arn arn:aws:iam::$ACCOUNT_ID:user/testuser
# Get IAM identity mapping(s)
eksctl get iamidentitymapping --cluster $CLUSTER_NAME
kubectl get cm -n kube-system aws-auth -o yaml | yh(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# eksctl delete iamidentitymapping --cluster $CLUSTER_NAME --arn arn:aws:iam::$ACCOUNT_ID:user/testuser
2023-06-04 00:49:59 [ℹ] removing identity "arn:aws:iam::account-id:user/testuser" from auth ConfigMap (username = "testuser", groups = ["system:authenticated"])
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# eksctl get iamidentitymapping --cluster $CLUSTER_NAME
ARN USERNAME GROUPS ACCOUNT
arn:aws:iam::account-id:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-1ANECPD8VGOD2 system:node:{{EC2PrivateDNSName}}system:bootstrappers,system:nodes
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl get cm -n kube-system aws-auth -o yaml | yh
apiVersion: v1
data:
mapRoles: |
- groups:
- system:bootstrappers
- system:nodes
rolearn: arn:aws:iam::account-id:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-1ANECPD8VGOD2
username: system:node:{{EC2PrivateDNSName}}
mapUsers: |
[]
kind: ConfigMap
metadata:
creationTimestamp: "2023-06-03T11:49:01Z"
name: aws-auth
namespace: kube-system
resourceVersion: "59899"
uid: 123456-ff16-43d1-b635-1234567890
(primedo_eks@myeks:default) [root@myeks-bastion ~]#8.[myeks-bastion-2] testuser kubectl 사용 확인
# 시도
kubectl get node -v6
kubectl api-resources -v5(**testuser**:default) [root@myeks-bastion-2 ~]#
(**testuser**:default) [root@myeks-bastion-2 ~]# kubectl get node -v6
I0604 00:50:53.691026 4063 loader.go:373] Config loaded from file: /root/.kube/config
I0604 00:50:55.177637 4063 round_trippers.go:553] GET https://3C04D23FACA0F3ABCBDEE85966E90032.gr7.ap-northeast-2.eks.amazonaws.com/api/v1/nodes?limit=500 401 Unauthorized in 1468 milliseconds
I0604 00:50:55.177912 4063 helpers.go:246] server response object: [{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "Unauthorized",
"reason": "Unauthorized",
"code": 401
}]
error: You must be logged in to the server (Unauthorized)
(**testuser**:default) [root@myeks-bastion-2 ~]# kubectl api-resources -v5
E0604 00:51:05.115987 4117 memcache.go:265] couldn't get current server API group list: the server has asked for the client to provide credentials
I0604 00:51:05.116113 4117 cached_discovery.go:120] skipped caching discovery info due to the server has asked for the client to provide credentials
NAME SHORTNAMES APIVERSION NAMESPACED KIND
I0604 00:51:05.116320 4117 helpers.go:246] server response object: [{
"metadata": {},
"status": "Failure",
"message": "the server has asked for the client to provide credentials",
"reason": "Unauthorized",
"details": {
"causes": [
{
"reason": "UnexpectedServerResponse",
"message": "unknown"
}
]
},
"code": 401
}]
error: You must be logged in to the server (the server has asked for the client to provide credentials)
(**testuser**:default) [root@myeks-bastion-2 ~]#9.(참고) config 샘플 - 링크
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
mapRoles: |
- groups:
- system:bootstrappers
- system:nodes
rolearn: arn:aws:iam::111122223333:role/my-role
username: system:node:{{EC2PrivateDNSName}}
- groups:
- eks-console-dashboard-full-access-group
rolearn: arn:aws:iam::111122223333:role/my-console-viewer-role
username: my-console-viewer-role
mapUsers: |
- groups:
- system:masters
userarn: arn:aws:iam::111122223333:user/admin
username: admin
- groups:
- eks-console-dashboard-restricted-access-group
userarn: arn:aws:iam::444455556666:user/my-user
username: my-user✍ EKS IRSA
EC2 Instance Profile : 사용하기 편하지만, 최소 권한 부여 원칙에 위배하며 보안상 권고하지 않음 - 링크 → IRSA를 쓰시라
# 설정 예시 1 : eksctl 사용 시
eksctl create cluster --name $CLUSTER_NAME ... --external-dns-access --full-ecr-access --asg-access
# 설정 예시 2 : eksctl로 yaml 파일로 노드 생성 시
cat myeks.yaml | yh
...
managedNodeGroups:
- amiFamily: AmazonLinux2
iam:
withAddonPolicies:
albIngress: false
appMesh: false
appMeshPreview: false
autoScaler: true
awsLoadBalancerController: false
certManager: true
cloudWatch: true
ebs: false
efs: false
externalDNS: true
fsx: false
imageBuilder: true
xRay: false
...
# 설정 예시 3 : 테라폼
...
동작 : k8s파드 → AWS 서비스 사용 시 ⇒ AWS STS/IAM ↔ IAM OIDC Identity Provider(EKS IdP) 인증/인가 (EKS인증하고 반대다!!!)
k8s api 접근 단계
- AuthN → AuthZ → Admisstion Control 권한이 있는 사용자에 한해서 관리자(Admin)가 특정 행동을 제한(validate) 혹은 변경(mutate) - 링크 Slack
- AuthN & AuthZ - MutatingWebhook - Object schema validation - ValidatingWebhook → etcd
- Admission Control도 Webhook으로 사용자에게 API가 열려있고, 사용자는 자신만의 Admission Controller를 구현할 수 있으며,
이를 Dynamic Admission Controller라고 부르고, 크게 MutatingWebhook 과 ValidatingWebhook 로 나뉩니다. MutatingWebhook은 사용자가 요청한 request에 대해서 관리자가 임의로 값을 변경하는 작업입니다.ValidatingWebhook은 사용자가 요청한 request에 대해서 관리자기 허용을 막는 작업입니다.
kubectl get validatingwebhookconfigurations
kubectl get mutatingwebhookconfigurationsJWT : Bearer type - JWT(JSON Web Token) X.509 Certificate의 lightweight JSON 버전
- Bearer type 경우, 서버에서 지정한 어떠한 문자열도 입력할 수 있습니다. 하지만 굉장히 허술한 느낌을 받습니다.
- 이를 보완하고자 쿠버네티스에서 Bearer 토큰을 전송할 때 주로 JWT (JSON Web Token) 토큰을 사용합니다.
- JWT는 X.509 Certificate와 마찬가지로 private key를 이용하여 토큰을 서명하고 public key를 이용하여 서명된 메세지를 검증합니다.
- 이러한 메커니즘을 통해 해당 토큰이 쿠버네티스를 통해 생성된 valid한 토큰임을 인증할 수 있습니다.
- X.509 Certificate의 lightweight JSON 버전이라고 생각하면 편리합니다.
- jwt는 JSON 형태로 토큰 형식을 정의한 스펙입니다. jwt는 쿠버네티스에서 뿐만 아니라 다양한 웹 사이트에서 인증, 권한 허가, 세션관리 등의 목적으로 사용합니다.
- Header: 토큰 형식와 암호화 알고리즘을 선언합니다.
- Payload: 전송하려는 데이터를 JSON 형식으로 기입합니다.
- Signature: Header와 Payload의 변조 가능성을 검증합니다.
- 각 파트는 base64 URL 인코딩이 되어서
.으로 합쳐지게 됩니다.
OIDC : 사용자를 인증해 사용자에게 액세스 권한을 부여할 수 있게 해주는 프로토콜 ⇒ [커피고래]님 블로그 OpenID Connect - 링크
- OAuth 2.0 : 권한허가 처리 프로토콜, 다른 서비스에 접근할 수 있는 권한을 획득하거나 반대로 다른 서비스에게 권한을 부여할 수 있음 - 생활코딩
- 위임 권한 부여 Delegated Authorization, 사용자 인증 보다는 제한된 사람에게(혹은 시스템) 제한된 권한을 부여하는가, 예) 페이스북 posting 권한
- Access Token : 발급처(OAuth 2.0), 서버의 리소스 접근 권한
- OpenID : 비영리기관인 OpenID Foundation에서 추진하는 개방형 표준 및 분산 인증 Authentication 프로토콜, 사용자 인증 및 사용자 정보 제공(id token) - 링크
- ID Token : 발급처(OpenID Connect), 유저 프로필 정보 획득
- OIDC OpenID Connect = OpenID 인증 + OAuth2.0 인가, JSON 포맷을 이용한 RESful API 형식으로 인증 - 링크
iss: 토큰 발행자sub: 사용자를 구분하기 위한 유니크한 구분자email: 사용자의 이메일iat: 토큰이 발행되는 시간을 Unix time으로 표기한 것exp: 토큰이 만료되는 시간을 Unix time으로 표기한 것aud: ID Token이 어떤 Client를 위해 발급된 것인지.
- IdP Open Identify Provider : 구글, 카카오와 같이 OpenID 서비스를 제공하는 신원 제공자.
- OpenID Connect에서 IdP의 역할을 OAuth가 수행 - 링크
- RP Relying Party : 사용자를 인증하기 위해 IdP에 의존하는 주체
- AWS SDK는 AWS_ROLE_ARN 및 AWS_WEB_IDENTITY_TOKEN_FILE 이름의 환경변수를 읽어들여 Web Identity 토큰으로 AssumeRoleWithWebIdentify를 호출함으로써 Assume Role을 시도하여 임시 자격 증명을 획득하고, 특정 IAM Role 역할을 사용할 수 있게 됩니다.
- 이때 Assume Role 동작을 위한 인증은 AWS가 아닌 외부 Web IdP(EKS IdP)에 위임하여 처리합니다.
실습1 - 링크
# 파드1 생성
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: eks-iam-test1
spec:
containers:
- name: my-aws-cli
image: amazon/aws-cli:latest
args: ['s3', 'ls']
restartPolicy: Never
automountServiceAccountToken: false
EOF
# 확인
kubectl get pod
kubectl describe pod
# 로그 확인
kubectl logs eks-iam-test1
# 파드1 삭제
kubectl delete pod eks-iam-test1- CloudTrail 이벤트 ListBuckets 확인 → 기록 표시까지 약간의 시간 필요
{
...
"userIdentity": {
"type": "AssumedRole",
"principalId": "xxxx",
"arn": "arn:aws:sts::111122223333:assumed-role/eksctl-eks-oidc-demo-nodegroup-ng-NodeInstanceRole-xxxx/xxxx",
"accountId": "111122223333",
"accessKeyId": "AKIAIOSFODNN7EXAMPLE",
"sessionContext": {
"sessionIssuer": {
"type": "Role",
"principalId": "xxxx",
"arn": "arn:aws:iam::xxxx:role/eksctl-eks-oidc-demo-nodegroup-ng-NodeInstanceRole-xxxx",
"accountId": "111122223333",
"userName": "eksctl-eks-oidc-demo-nodegroup-ng-NodeInstanceRole-xxxx"
},
"webIdFederationData": {},
"attributes": {
"creationDate": "2021-12-04T14:54:49Z",
"mfaAuthenticated": "false"
},
"ec2RoleDelivery": "2.0"
}
},
"eventTime": "2021-12-04T15:09:20Z",
"eventSource": "s3.amazonaws.com",
"eventName": "ListBuckets",
"awsRegion": "us-east-2",
"sourceIPAddress": "192.0.2.1",
"userAgent": "[aws-cli/2.4.5 Python/3.8.8 Linux/5.4.156-83.273.amzn2.x86_64 docker/x86_64.amzn.2 prompt/off command/s3.ls]",
"errorCode": "AccessDenied",
"errorMessage": "Access Denied",
"requestParameters": {
"Host": "s3.us-east-2.amazonaws.com"
},
...
}tag(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl describe pod
Name: eks-iam-test1
Namespace: default
Priority: 0
Service Account: default
Node: ip-192-168-3-207.ap-northeast-2.compute.internal/192.168.3.207
Start Time: Sun, 04 Jun 2023 01:31:43 +0900
Labels: <none>
Annotations: kubernetes.io/psp: eks.privileged
Status: Failed
IP: 192.168.3.230
IPs:
IP: 192.168.3.230
Containers:
my-aws-cli:
Container ID: containerd://085c43860e953a060e93ae2be2a4de8cfed7a6fc645ee53a2180db21cc6baa7e
Image: amazon/aws-cli:latest
Image ID: docker.io/amazon/aws-cli@sha256:221ff6588005667221ff7e5e3b6cde2e1d2dd04fbe124652292e5fac652a8234
Port: <none>
Host Port: <none>
Args:
s3
ls
State: Terminated
Reason: Error
Exit Code: 254
Started: Sun, 04 Jun 2023 01:31:54 +0900
Finished: Sun, 04 Jun 2023 01:31:55 +0900
Ready: False
Restart Count: 0
Environment: <none>
Mounts: <none>
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes: <none>
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 15s default-scheduler Successfully assigned default/eks-iam-test1 to ip-192-168-3-207.ap-northeast-2.compute.internal
Normal Pulling 15s kubelet Pulling image "amazon/aws-cli:latest"
Normal Pulled 4s kubelet Successfully pulled image "amazon/aws-cli:latest" in 10.692493716s
Normal Created 4s kubelet Created container my-aws-cli
Normal Started 4s kubelet Started container my-aws-cli
실습2 - Kubernetes Service Accounts : https://jwt.io/
- Kubernetes Pods are given an identity through a Kubernetes concept called a Kubernetes Service Account.
- When a Service Account is created, a JWT token is automatically created as a Kubernetes Secret.
- This Secret can then be mounted into Pods and used by that Service Account to authenticate to the Kubernetes API Server.
# 파드2 생성
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: eks-iam-test2
spec:
containers:
- name: my-aws-cli
image: amazon/aws-cli:latest
command: ['sleep', '36000']
restartPolicy: Never
EOF
# 확인
kubectl get pod
kubectl describe pod
# aws 서비스 사용 시도
kubectl exec -it eks-iam-test2 -- aws s3 ls
# 서비스 어카운트 토큰 확인
SA_TOKEN=$(kubectl exec -it eks-iam-test2 -- cat /var/run/secrets/kubernetes.io/serviceaccount/token)
echo $SA_TOKEN
# jwt 혹은 아래 JWT 웹 사이트 이용
jwt decode $SA_TOKEN --json --iso8601
...
#헤더
{
"alg": "RS256",
"kid": "1a8fcaee12b3a8f191327b5e9b997487ae93baab"
}
# 페이로드 : OAuth2에서 쓰이는 aud, exp 속성 확인! > projectedServiceAccountToken 기능으로 토큰에 audience,exp 항목을 덧붙힘
## iss 속성 : EKS OpenID Connect Provider(EKS IdP) 주소 > 이 EKS IdP를 통해 쿠버네티스가 발급한 토큰이 유요한지 검증
{
"aud": [
"https://kubernetes.default.svc" # 해당 주소는 k8s api의 ClusterIP 서비스 주소 도메인명, kubectl get svc kubernetes
],
"exp": 1716619848,
"iat": 1685083848,
"iss": "https://oidc.eks.ap-northeast-2.amazonaws.com/id/F6A7523462E8E6CDADEE5D41DF2E71F6",
"kubernetes.io": {
"namespace": "default",
"pod": {
"name": "eks-iam-test2",
"uid": "10dcccc8-a16c-4fc7-9663-13c9448e107a"
},
"serviceaccount": {
"name": "default",
"uid": "acb6c60d-0c5f-4583-b83b-1b629b0bdd87"
},
"warnafter": 1685087455
},
"nbf": 1685083848,
"sub": "system:serviceaccount:default:default"
}
# 파드2 삭제
kubectl delete pod eks-iam-test2Volumes:
kube-api-access-gmlb6:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
(primedo_eks@myeks:default) [root@myeks-bastion ~]# SA_TOKEN=$(kubectl exec -it eks-iam-test2 -- cat /var/run/secrets/kubernetes.io/serviceaccount/token)
(primedo_eks@myeks:default) [root@myeks-bastion ~]# echo $SA_TOKEN.
eyJhbGciOiJSUzI1NiIsImtpZCI6ImY1MTJmNjk3NzEwMTFiN2EyZTc0MzAwZGZlMTQ1OWY4YzQ5MTA1YTgifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjIl0sImV4cCI6MTcxNzM0NjMyMCwiaWF0IjoxNjg1ODEwMzIwLCJpc3MiOiJodHRwczovL29pZGMuZWtzLmFwLW5vcnRoZWFzdC0yLmFtYXpvbmF3cy5jb20vaWQvM0MwNEQyM0ZBQ0EwRjNBQkNCREVFODU5NjZFOTAwMzIiLCJrdWJlcm5ldGVzLmlvIjp7Im5hbWVzcGFjZSI6ImRlZmF1bHQiLCJwb2QiOnsibmFtZSI6ImVrcy1pYW0tdGVzdDIiLCJ1aW
실습3 - ‣ : This webhook is for mutating pods that will require AWS IAM access
- For the webhook to inject a new Token into our Pod, we are going to create a new Kubernetes Service Account, annotate our Service Account with an AWS IAM role ARN, and then reference this new Kubernetes Service Account in a Kubernetes Pod. The eksctl tool can be used to automate a few steps for us, but all of these steps can also be done manually.
- The
eksctl create iamserviceaccountcommand creates:- A Kubernetes Service Account
- An IAM role with the specified IAM policy
- A trust policy on that IAM role
- Finally, it will also annotate the Kubernetes Service Account with the IAM Role Arn created.
# Create an iamserviceaccount - AWS IAM role bound to a Kubernetes service account
eksctl create iamserviceaccount \
--name my-sa \
--namespace default \
--cluster $CLUSTER_NAME \
--approve \
--attach-policy-arn $(aws iam list-policies --query 'Policies[?PolicyName==`AmazonS3ReadOnlyAccess`].Arn' --output text)
# 확인 >> 웹 관리 콘솔에서 CloudFormation Stack >> IAM Role 확인
# aws-load-balancer-controller IRSA는 어떤 동작을 수행할 것 인지 생각해보자!
eksctl get iamserviceaccount --cluster $CLUSTER_NAME
# Inspecting the newly created Kubernetes Service Account, we can see the role we want it to assume in our pod.
kubectl get sa
kubectl describe sa my-sa
Name: my-sa
Namespace: default
Labels: app.kubernetes.io/managed-by=eksctl
Annotations: eks.amazonaws.com/role-arn: arn:aws:iam::911283464785:role/eksctl-myeks-addon-iamserviceaccount-default-Role1-1MJUYW59O6QGH
Image pull secrets: <none>
Mountable secrets: <none>
Tokens: <none>
Events: <none>
"Federated": "----:oidc-provider/oidc.eks.ap-northeast-2.amazonaws.com/id/---"
EKS
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# eksctl get iamserviceaccount --cluster $CLUSTER_NAME
NAMESPACE NAME ROLE ARN
default my-sa arn:aws:iam::account-id:role/eksctl-myeks-addon-iamserviceaccount-default-Role1-10IEGPLEO2IYT
kube-system aws-load-balancer-controller arn:aws:iam::account-id:role/eksctl-myeks-addon-iamserviceaccount-kube-sy-Role1-58X36QVU977V
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# eksctl get iamserviceaccount --cluster $CLUSTER_NAME
NAMESPACE NAME ROLE ARN
default my-sa arn:aws:iam::account-id:role/eksctl-myeks-addon-iamserviceaccount-default-Role1-10IEGPLEO2IYT
kube-system aws-load-balancer-controller arn:aws:iam::account-id:role/eksctl-myeks-addon-iamserviceaccount-kube-sy-Role1-58X36QVU977V
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl get sa
NAME SECRETS AGE
default 0 5h15m
my-sa 0 9m11s
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl describe sa my-sa
Name: my-sa
Namespace: default
Labels: app.kubernetes.io/managed-by=eksctl
Annotations: eks.amazonaws.com/role-arn: arn:aws:iam::account-id:role/eksctl-myeks-addon-iamserviceaccount-default-Role1-10IEGPLEO2IYT
Image pull secrets: <none>
Mountable secrets: <none>
Tokens: <none>
Events: <none>
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
Annotations: eks.amazonaws.com/role-arn: arn:aws:iam::account-id:role/eksctl-myeks-addon-iamserviceaccount-default-Role1-10IEGPLEO2IYT
- Let’s see how this IAM role looks within the AWS Management Console. Navigate to IAM and then IAM Roles and search for the role. You will see the Annotations field when you describe your service account. ⇒ IAM Role 확인
- Select the Trust relationships tab and select Edit trust relationship to view the policy document.
- You can see that this policy is allowing an identity
system:serviceaccount:default:my-sato assume the role usingsts:AssumeRoleWithWebIdentityaction. The principal for this policy is an OIDC provider.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::911283464785:oidc-provider/oidc.eks.ap-northeast-2.amazonaws.com/id/F6A7523462E8E6CDADEE5D41DF2E71F6"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.ap-northeast-2.amazonaws.com/id/F6A7523462E8E6CDADEE5D41DF2E71F6:sub": "system:serviceaccount:default:my-sa",
"oidc.eks.ap-northeast-2.amazonaws.com/id/F6A7523462E8E6CDADEE5D41DF2E71F6:aud": "sts.amazonaws.com"
}
}
}
]
}Now let’s see what happens when we use this new Service Account within a Kubernetes Pod. 신규 파드 만들자!
# 파드3번 생성
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: eks-iam-test3
spec:
serviceAccountName: my-sa
containers:
- name: my-aws-cli
image: amazon/aws-cli:latest
command: ['sleep', '36000']
restartPolicy: Never
EOF
# 해당 SA를 파드가 사용 시 mutatingwebhook으로 Env,Volume 추가함
kubectl get mutatingwebhookconfigurations pod-identity-webhook -o yaml | kubectl neat | yh
# 파드 생성 yaml에 없던 내용이 추가됨!!!!!
# Pod Identity Webhook은 mutating webhook을 통해 아래 Env 내용과 1개의 볼륨을 추가함
kubectl get pod eks-iam-test3
kubectl describe pod eks-iam-test3
...
Environment:
AWS_STS_REGIONAL_ENDPOINTS: regional
AWS_DEFAULT_REGION: ap-northeast-2
AWS_REGION: ap-northeast-2
AWS_ROLE_ARN: arn:aws:iam::911283464785:role/eksctl-myeks-addon-iamserviceaccount-default-Role1-GE2DZKJYWCEN
AWS_WEB_IDENTITY_TOKEN_FILE: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
Mounts:
/var/run/secrets/eks.amazonaws.com/serviceaccount from aws-iam-token (ro)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-69rh8 (ro)
...
Volumes:
aws-iam-token:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 86400
kube-api-access-sn467:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
...
# 파드에서 aws cli 사용 확인
eksctl get iamserviceaccount --cluster $CLUSTER_NAME
kubectl exec -it eks-iam-test3 -- aws sts get-caller-identity --query Arn
"arn:aws:sts::911283464785:assumed-role/eksctl-myeks-addon-iamserviceaccount-default-Role1-GE2DZKJYWCEN/botocore-session-1685179271"
# 되는 것고 안되는 것은 왜그런가?
kubectl exec -it eks-iam-test3 -- aws s3 ls
kubectl exec -it eks-iam-test3 -- aws ec2 describe-instances --region ap-northeast-2
kubectl exec -it eks-iam-test3 -- aws ec2 describe-vpcs --region ap-northeast-2 Environment:
AWS_STS_REGIONAL_ENDPOINTS: regional
AWS_DEFAULT_REGION: ap-northeast-2
AWS_REGION: ap-northeast-2
AWS_ROLE_ARN: arn:aws:iam::account-id:role/eksctl-myeks-addon-iamserviceaccount-default-Role1-10IEGPLEO2IYT
AWS_WEB_IDENTITY_TOKEN_FILE: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
Volumes:
aws-iam-token:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 86400
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl exec -it eks-iam-test3 -- aws sts get-caller-identity --query Arn
"arn:aws:sts::account-id:assumed-role/eksctl-myeks-addon-iamserviceaccount-default-Role1-10IEGPLEO2IYT/botocore-session-1685811726"
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl exec -it eks-iam-test3 -- aws s3 ls
2023-05-20 13:13:45 cf-templates-3e92fx3x5elt-ap-northeast-2
(primedo_eks@myeks:default) [root@myeks-bastion ~]#권한은 S3만 되어 있어서 ec2는 권한 없음
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl exec -it eks-iam-test3 -- aws ec2 describe-instances --region ap-northeast-2
An error occurred (UnauthorizedOperation) when calling the DescribeInstances operation: You are not authorized to perform this operation.
command terminated with exit code 254
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl exec -it eks-iam-test3 -- aws ec2 describe-vpcs --region ap-northeast-2
An error occurred (UnauthorizedOperation) when calling the DescribeVpcs operation: You are not authorized to perform this operation.
command terminated with exit code 254
(primedo_eks@myeks:default) [root@myeks-bastion ~]## 파드에 볼륨 마운트 2개 확인
kubectl get pod eks-iam-test3 -o json | jq -r '.spec.containers | .[].volumeMounts'
[
{
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
"name": "kube-api-access-sn467",
"readOnly": true
},
{
"mountPath": "/var/run/secrets/eks.amazonaws.com/serviceaccount",
"name": "aws-iam-token",
"readOnly": true
}
]
# aws-iam-token 볼륨 정보 확인 : JWT 토큰이 담겨져있고, exp, aud 속성이 추가되어 있음
kubectl get pod eks-iam-test3 -o json | jq -r '.spec.volumes[] | select(.name=="aws-iam-token")'
{
"name": "aws-iam-token",
"projected": {
"defaultMode": 420,
"sources": [
{
"serviceAccountToken": {
"audience": "sts.amazonaws.com",
"expirationSeconds": 86400,
"path": "token"
}
}
]
}
}
# api 리소스 확인
kubectl api-resources |grep hook
mutatingwebhookconfigurations admissionregistration.k8s.io/v1 false MutatingWebhookConfiguration
validatingwebhookconfigurations admissionregistration.k8s.io/v1 false ValidatingWebhookConfiguration
#
kubectl explain mutatingwebhookconfigurations
#
kubectl get MutatingWebhookConfiguration
NAME WEBHOOKS AGE
pod-identity-webhook 1 147m
vpc-resource-mutating-webhook 1 147m
# pod-identity-webhook 확인
kubectl describe MutatingWebhookConfiguration pod-identity-webhook
kubectl get MutatingWebhookConfiguration pod-identity-webhook -o yaml | yh(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl get pod eks-iam-test3 -o json | jq -r '.spec.containers | .[].volumeMounts'
[
{
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
"name": "kube-api-access-2x92x",
"readOnly": true
},
{
"mountPath": "/var/run/secrets/eks.amazonaws.com/serviceaccount",
"name": "aws-iam-token",
"readOnly": true
}
]
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl get pod eks-iam-test3 -o json | jq -r '.spec.volumes[] | select(.name=="aws-iam-token")'
{
"name": "aws-iam-token",
"projected": {
"defaultMode": 420,
"sources": [
{
"serviceAccountToken": {
"audience": "sts.amazonaws.com",
"expirationSeconds": 86400,
"path": "token"
}
}
]
}
}
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# kubectl api-resources |grep hook
mutatingwebhookconfigurations admissionregistration.k8s.io/v1 false MutatingWebhookConfiguration
validatingwebhookconfigurations admissionregistration.k8s.io/v1 false ValidatingWebhookConfiguration
(primedo_eks@myeks:default) [root@myeks-bastion ~]#- If we exec into the running Pod and inspect this token, we can see that it looks slightly different from the previous SA Token.
- You can see that the intended audience for this token is now
sts.amazonaws.com, the issuer who has created and signed this token is still our OIDC provider, and finally, the expiration of the token is much shorter at 24 hours. We can modify the expiration duration for the service account usingeks.amazonaws.com/token-expirationannotation in our Pod definition or Service Account definition. - The mutating webhook does more than just mount an additional token into the Pod. The mutating webhook also injects environment variables.
https://jwt.io/
# AWS_WEB_IDENTITY_TOKEN_FILE 확인
IAM_TOKEN=$(kubectl exec -it eks-iam-test3 -- cat /var/run/secrets/eks.amazonaws.com/serviceaccount/token)
echo $IAM_TOKEN
# JWT 웹 확인
{
"aud": [
"sts.amazonaws.com"
],
"exp": 1685175662,
"iat": 1685089262,
"iss": "https://oidc.eks.ap-northeast-2.amazonaws.com/id/F6A7523462E8E6CDADEE5D41DF2E71F6",
"kubernetes.io": {
"namespace": "default",
"pod": {
"name": "eks-iam-test3",
"uid": "123456789-1234-123456-1234-1234567890"
},
"serviceaccount": {
"name": "my-sa",
"uid": "123456789-1234-123456-1234-1234567890"
}
},
"nbf": 1685089262,
"sub": "system:serviceaccount:default:my-sa"
}
# env 변수 확인
kubectl get pod eks-iam-test3 -o json | jq -r '.spec.containers | .[].env'
[
{
"name": "AWS_STS_REGIONAL_ENDPOINTS",
"value": "regional"
},
{
"name": "AWS_DEFAULT_REGION",
"value": "ap-northeast-2"
},
{
"name": "AWS_REGION",
"value": "ap-northeast-2"
},
{
"name": "AWS_ROLE_ARN",
"value": "arn:aws:iam::911283464785:role/eksctl-myeks-addon-iamserviceaccount-default-Role1-1MJUYW59O6QGH"
},
{
"name": "AWS_WEB_IDENTITY_TOKEN_FILE",
"value": "/var/run/secrets/eks.amazonaws.com/serviceaccount/token"
}
]- Now that our workload has a token it can use to attempt to authenticate with IAM, the next part is getting AWS IAM to trust these tokens. AWS IAM supports federated identities using OIDC identity providers. This feature allows IAM to authenticate AWS API calls with supported identity providers after receiving a valid OIDC JWT. This token can then be passed to AWS STS
AssumeRoleWithWebIdentityAPI operation to get temporary IAM credentials. - The OIDC JWT token we have in our Kubernetes workload is cryptographically signed, and IAM should trust and validate these tokens before the AWS STS
AssumeRoleWithWebIdentityAPI operation can send the temporary credentials. As part of the Service Account Issuer Discovery feature of Kubernetes, EKS is hosting a public OpenID provider configuration document (Discovery endpoint) and the public keys to validate the token signature (JSON Web Key Sets – JWKS) athttps://OIDC_PROVIDER_URL/.well-known/openid-configuration.
# Let’s take a look at this endpoint. We can use the aws eks describe-cluster command to get the OIDC Provider URL.
IDP=$(aws eks describe-cluster --name myeks --query cluster.identity.oidc.issuer --output text)
# Reach the Discovery Endpoint
curl -s $IDP/.well-known/openid-configuration | jq -r '.'
# In the above output, you can see the jwks (JSON Web Key set) field, which contains the set of keys containing the public keys used to verify JWT (JSON Web Token).
# Refer to the documentation to get details about the JWKS properties.
curl -s $IDP/keys | jq -r '.'(primedo_eks@myeks:default) [root@myeks-bastion ~]#
(primedo_eks@myeks:default) [root@myeks-bastion ~]# IDP=$(aws eks describe-cluster --name myeks --query cluster.identity.oidc.issuer --output text)
(primedo_eks@myeks:default) [root@myeks-bastion ~]# curl -s $IDP/.well-known/openid-configuration | jq -r '.'
{
"issuer": "https://oidc.eks.ap-northeast-2.amazonaws.com/id/3C04D23FACA0F3ABCBDEE85966E90032",
"jwks_uri": "https://oidc.eks.ap-northeast-2.amazonaws.com/id/3C04D23FACA0F3ABCBDEE85966E90032/keys",
"authorization_endpoint": "urn:kubernetes:programmatic_authorization",
"response_types_supported": [
"id_token"
],
"subject_types_supported": [
"public"
],
"claims_supported": [
"sub",
"iss"
],
"id_token_signing_alg_values_supported": [
"RS256"
]
}
(primedo_eks@myeks:default) [root@myeks-bastion ~]#
primedo