Encrypting Confidential Data at Rest

Yu Sang Min·2025년 6월 9일

CKA

목록 보기

35/110

💬 https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/

$ etcdctl
$ apt-get install etcd-client

ETCDCTL_API=3 etcdctl \
   --cacert=/etc/kubernetes/pki/etcd/ca.crt   \
   --cert=/etc/kubernetes/pki/etcd/server.crt \
   --key=/etc/kubernetes/pki/etcd/server.key  \
   get /registry/secrets/default/<secret-name>
   
/registry/secrets/default/my-secret
k8s


v1Secret

        my-secretdefault"*$c913a0db-d0ac-43fb-ac08-dc719304b7412aB
kubectl-createUpdatevFieldsV1:-
+{"f:data":{".":{},"f:Key1":{}},"f:type":{}}B
Key1
    supersecretOpaque"

hexdump를 추가하지 않았을때 (간혹 올바른 형식으로 보이지 않아서)

ETCDCTL_API=3 etcdctl \
   --cacert=/etc/kubernetes/pki/etcd/ca.crt   \
   --cert=/etc/kubernetes/pki/etcd/server.crt \
   --key=/etc/kubernetes/pki/etcd/server.key  \
   get /registry/secrets/default/<secret-name> | hexdump -C
   
00000000  2f 72 65 67 69 73 74 72  79 2f 73 65 63 72 65 74  |/registry/secret|
00000010  73 2f 64 65 66 61 75 6c  74 2f 6d 79 2d 73 65 63  |s/default/my-sec|
00000020  72 65 74 0a 6b 38 73 00  0a 0c 0a 02 76 31 12 06  |ret.k8s.....v1..|
00000030  53 65 63 72 65 74 12 d0  01 0a b0 01 0a 09 6d 79  |Secret........my|
00000040  2d 73 65 63 72 65 74 12  00 1a 07 64 65 66 61 75  |-secret....defau|
00000050  6c 74 22 00 2a 24 63 39  31 33 61 30 64 62 2d 64  |lt".*$c913a0db-d|
00000060  30 61 63 2d 34 33 66 62  2d 61 63 30 38 2d 64 63  |0ac-43fb-ac08-dc|
00000070  37 31 39 33 30 34 62 37  34 31 32 00 38 00 42 08  |719304b7412.8.B.|
00000080  08 8d 93 9b c2 06 10 00  8a 01 61 0a 0e 6b 75 62  |..........a..kub|
00000090  65 63 74 6c 2d 63 72 65  61 74 65 12 06 55 70 64  |ectl-create..Upd|
000000a0  61 74 65 1a 02 76 31 22  08 08 8d 93 9b c2 06 10  |ate..v1"........|
000000b0  00 32 08 46 69 65 6c 64  73 56 31 3a 2d 0a 2b 7b  |.2.FieldsV1:-.+{|
000000c0  22 66 3a 64 61 74 61 22  3a 7b 22 2e 22 3a 7b 7d  |"f:data":{".":{}|
000000d0  2c 22 66 3a 4b 65 79 31  22 3a 7b 7d 7d 2c 22 66  |,"f:Key1":{}},"f|
000000e0  3a 74 79 70 65 22 3a 7b  7d 7d 42 00 12 13 0a 04  |:type":{}}B.....|
000000f0  4b 65 79 31 12 0b 73 75  70 65 72 73 65 63 72 65  |Key1..supersecre|
00000100  74 1a 06 4f 70 61 71 75  65 1a 00 22 00 0a        |t..Opaque.."..|
0000010e

위에 보이는 이것이 etcd에 저장되는 데이터
자세히 보면 secret이나 비밀번호든 보인다.
저장한 것이 실제로 이렇게 보이는 것을 알 수 있다.
따라서, 데이터는 암호화 되지 않은 형식으로 etcd에 저장된다.
etcd에 액세스 할 수 있는 사람은 누구나 secret으로 저장된 모든 정보를 얻을 수 있다. 이것은 보안취약점이 된다.

🔐 이미 암호화가 활성화 되어있는지 확인 하기

$ ps -aux | grep kube-api | grep "encryption-provider-config"

$ vi /etc/kubernetes/manifests/kube-apiserver.yaml

<중략>
spec:
  containers:
  - command:
    - kube-apiserver
    - --advertise-address=172.30.1.2
    - --allow-privileged=true
    - --authorization-mode=Node,RBAC
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --enable-admission-plugins=NodeRestriction
    - --enable-bootstrap-token-auth=true
    - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
    - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
    - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
    - --etcd-servers=https://127.0.0.1:2379
    - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kube
    - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-clie
    - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostnam
    - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client
    - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.
    - --requestheader-allowed-names=front-proxy-client
    - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-
    - --requestheader-extra-headers-prefix=X-Remote-Extra-
    - --requestheader-group-headers=X-Remote-Group
    - --requestheader-username-headers=X-Remote-User
    - --secure-port=6443
    - --service-account-issuer=https://kubernetes.default.svc.cluster
    - --service-account-key-file=/etc/kubernetes/pki/sa.pub
    - --service-account-signing-key-file=/etc/kubernetes/pki/sa.key

ps -aux ... 명령어의 결과를 반환하지 않고 따라서 이 옵션은 구성되지 않음.

👉🏻 구성파일을 만들고 옵션으로 전달하기

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:    // 이 부분에서 암호화할 리소스를 골라서 선택할 수 있다.
      - secrets
      - configmaps
      - pandas.awesome.bears.example # a custom resource API
    providers:    // providers는 배열이다
      # This configuration does not provide data confidentiality. The first
      # configured provider is specifying the "identity" mechanism, which
      # stores resources as plain text.
      #
      - identity: {} # plain text, in other words NO encryption // ID에 어떤 값도 입력하지 않으면 암호화 되지 않는다
      - aesgcm:     // 다양한 암호화 알고리즘이다.
          keys:
            - name: key1
              secret: c2VjcmV0IGlzIHNlY3VyZQ==
            - name: key2
              secret: dGhpcyBpcyBwYXNzd29yZA==
      - aescbc:
          keys:
            - name: key1
              secret: c2VjcmV0IGlzIHNlY3VyZQ==
            - name: key2
              secret: dGhpcyBpcyBwYXNzd29yZA==
      - secretbox:
          keys:
            - name: key1
              secret: YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXoxMjM0NTY=
  - resources:
      - events
    providers:
      - identity: {} # do not encrypt Events even though *.* is specified below 
  - resources:
      - '*.apps' # wildcard match requires Kubernetes 1.27 or later
    providers:
      - aescbc:
          keys:
          - name: key2
            secret: c2VjcmV0IGlzIHNlY3VyZSwgb3IgaXMgaXQ/Cg==
  - resources:
      - '*.*' # wildcard match requires Kubernetes 1.27 or later
    providers:
      - aescbc:
          keys:
          - name: key3
            secret: c2VjcmV0IGlzIHNlY3VyZSwgSSB0aGluaw==

identify 필드가 맨위가 아니라 암호화 알고리즘 중 하나가 맨 위에 있어야 etcd의 데이터를 암호화 한다.

# 간단한 예시

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
      - configmaps
      - pandas.awesome.bears.example
    providers:
      - aescbc:
          keys:
            - name: key1
              # See the following text for more details about the secret value
              secret: <BASE 64 ENCODED SECRET>
      - identity: {} # this fallback allows reading unencrypted secrets;
                     # for example, during initial migration

모든 secret을 암호화하도록 지정하고 aescbc 알고리즘 provider를 사용

$ head -c 32 /dev/urandom | base64

위 명령어를 사용하여 32바이트 무작위 키를 생성할 수 있다.
생성된 키를 다음과 같이 secret 필드에 붙여넣기 한다.

# vi enc.yaml

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
      - configmaps
      - pandas.awesome.bears.example
    providers:
      - aescbc:
          keys:
            - name: key1
              # See the following text for more details about the sec
              secret: wxzGkxm0kXEvvsrZbgb7dE61/wT9XBAu4M4KN1wcRKU=
      - identity: {} # this fallback allows reading unencrypted secre
                     # for example, during initial migration

암호화 구성파일을 생성하였다.
이 파일을 kube-apiserver에 추가하면 된다
내부에 마운트를 해야하므로 볼륨과 볼륨 마운트를 사용

apiVersion: v1
kind: Pod
metadata:
  annotations:
    kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 10.20.30.40:443
  creationTimestamp: null
  labels:
    app.kubernetes.io/component: kube-apiserver
    tier: control-plane
  name: kube-apiserver
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-apiserver
    ...
    - --encryption-provider-config=/etc/kubernetes/enc/enc.yaml  # add this line
    volumeMounts:
    ...
    - name: enc                           # add this line
      mountPath: /etc/kubernetes/enc      # add this line, pod 내 매핑될 경로
      readOnly: true                      # add this line
    ...
  volumes:
  ...
  - name: enc                             # add this line
    hostPath:                             # add this line
      path: /etc/kubernetes/enc           # add this line, 로컬 디렉토리
      type: DirectoryOrCreate             # add this line

띄어쓰기나 주석 주의할것, 인식되면 kube-apiserver 생성 잘 안됨

📝 새로운 secret 만들기

$ kubectl create secret generic 
$ kubectl create secret generic my-secret2 --from-literal=key2=topsecret
secret/my-secret2 created
$ k get secret
NAME         TYPE     DATA   AGE
my-secret    Opaque   1      40m
my-secret2   Opaque   1      23s

ETCDCTL_API=3 etcdctl \
   --cacert=/etc/kubernetes/pki/etcd/ca.crt   \
   --cert=/etc/kubernetes/pki/etcd/server.crt \
   --key=/etc/kubernetes/pki/etcd/server.key  \
   get /registry/secrets/default/my-secret2 | hexdump -C

00000000  2f 72 65 67 69 73 74 72  79 2f 73 65 63 72 65 74  |/registry/secret|
00000010  73 2f 64 65 66 61 75 6c  74 2f 6d 79 2d 73 65 63  |s/default/my-sec|
00000020  72 65 74 32 0a 6b 38 73  3a 65 6e 63 3a 61 65 73  |ret2.k8s:enc:aes|
00000030  63 62 63 3a 76 31 3a 6b  65 79 31 3a 91 b9 98 fc  |cbc:v1:key1:....|
00000040  a0 89 dc 05 9c c9 3c 74  7b 86 a2 b6 17 6f 08 25  |......<t{....o.%|
00000050  cc d2 67 b6 92 bc e5 a4  d1 bc 8a 1c aa 67 91 b4  |..g..........g..|
00000060  d3 1f a2 13 a2 69 68 da  6f 29 8d 15 a4 2d 0c f7  |.....ih.o)...-..|
00000070  81 aa 8c e4 67 4f ec 59  86 01 a9 d6 26 00 51 6f  |....gO.Y....&.Qo|
00000080  ab e7 9c 52 50 a3 69 ec  1d 0f b6 ca ae 15 7e 69  |...RP.i.......~i|
00000090  07 3b 93 4b bb 87 fd 8f  d2 c7 bd 56 1c 15 73 83  |.;.K.......V..s.|
000000a0  c1 71 35 a2 79 73 a5 0c  c4 0f b1 fd d5 e2 1c c4  |.q5.ys..........|
000000b0  6f e5 b3 c8 c9 ed 68 38  f8 61 ea cf 5b 70 35 d6  |o.....h8.a..[p5.|
000000c0  61 4a 98 a8 fe b6 bb f8  c6 54 07 db 72 c0 89 be  |aJ.......T..r...|
000000d0  ff 14 df 15 7f 76 10 e7  d3 ec 82 f7 03 53 e7 24  |.....v.......S.$|
000000e0  e5 12 85 81 9f dd 97 ed  92 a5 47 25 c3 9a 43 0b  |..........G%..C.|
000000f0  f3 be f6 d3 ed 3a 93 26  eb c6 f5 b9 73 9e 21 15  |.....:.&....s.!.|
00000100  19 f9 4e f5 bb 7c fd af  b7 b9 64 74 e0 19 c2 38  |..N..|....dt...8|
00000110  67 9d ec d7 45 61 bd 7b  60 8e 74 78 69 e7 4d dd  |g...Ea.{`.txi.M.|
00000120  42 99 95 05 b6 34 ae b7  4e 09 9f 11 91 70 2f bb  |B....4..N....p/.|
00000130  6e 19 e6 ed 47 e2 6e 74  d1 2d 78 84 0a           |n...G.nt.-x..|
0000013d

my-sceret2는 암호화 되어있다
my-secret은 평문으로 되어있다
encryption이 활성화 된 이후 새롭게 만드는 항목만 암호화되기 때문

♻️ 기존에 생성된 평문 secret 전체 암호화 방법

$ kubectl get secrets --all-namespaces -o json | kubectl replace -f -
secret/my-secret replaced
secret/my-secret2 replaced

$ ETCDCTL_API=3 etcdctl \
   --cacert=/etc/kubernetes/pki/etcd/ca.crt   \
   --cert=/etc/kubernetes/pki/etcd/server.crt \
   --key=/etc/kubernetes/pki/etcd/server.key  \
   get /registry/secrets/default/my-secret | hexdump -C
   
   00000000  2f 72 65 67 69 73 74 72  79 2f 73 65 63 72 65 74  |/registry/secret|
00000010  73 2f 64 65 66 61 75 6c  74 2f 6d 79 2d 73 65 63  |s/default/my-sec|
00000020  72 65 74 32 0a 6b 38 73  3a 65 6e 63 3a 61 65 73  |ret2.k8s:enc:aes|
00000030  63 62 63 3a 76 31 3a 6b  65 79 31 3a 91 b9 98 fc  |cbc:v1:key1:....|
00000040  a0 89 dc 05 9c c9 3c 74  7b 86 a2 b6 17 6f 08 25  |......<t{....o.%|
00000050  cc d2 67 b6 92 bc e5 a4  d1 bc 8a 1c aa 67 91 b4  |..g..........g..|
00000060  d3 1f a2 13 a2 69 68 da  6f 29 8d 15 a4 2d 0c f7  |.....ih.o)...-..|
00000070  81 aa 8c e4 67 4f ec 59  86 01 a9 d6 26 00 51 6f  |....gO.Y....&.Qo|
00000080  ab e7 9c 52 50 a3 69 ec  1d 0f b6 ca ae 15 7e 69  |...RP.i.......~i|
00000090  07 3b 93 4b bb 87 fd 8f  d2 c7 bd 56 1c 15 73 83  |.;.K.......V..s.|
000000a0  c1 71 35 a2 79 73 a5 0c  c4 0f b1 fd d5 e2 1c c4  |.q5.ys..........|
000000b0  6f e5 b3 c8 c9 ed 68 38  f8 61 ea cf 5b 70 35 d6  |o.....h8.a..[p5.|
000000c0  61 4a 98 a8 fe b6 bb f8  c6 54 07 db 72 c0 89 be  |aJ.......T..r...|
000000d0  ff 14 df 15 7f 76 10 e7  d3 ec 82 f7 03 53 e7 24  |.....v.......S.$|
000000e0  e5 12 85 81 9f dd 97 ed  92 a5 47 25 c3 9a 43 0b  |..........G%..C.|
000000f0  f3 be f6 d3 ed 3a 93 26  eb c6 f5 b9 73 9e 21 15  |.....:.&....s.!.|
00000100  19 f9 4e f5 bb 7c fd af  b7 b9 64 74 e0 19 c2 38  |..N..|....dt...8|
00000110  67 9d ec d7 45 61 bd 7b  60 8e 74 78 69 e7 4d dd  |g...Ea.{`.txi.M.|
00000120  42 99 95 05 b6 34 ae b7  4e 09 9f 11 91 70 2f bb  |B....4..N....p/.|
00000130  6e 19 e6 ed 47 e2 6e 74  d1 2d 78 84 0a           |n...G.nt.-x..|
0000013d
controlplane:/etc/kubernetes/manifests$ kubectl get secrets --all-namespaces -o json | kubectl replace -f -
secret/my-secret replaced
secret/my-secret2 replaced

$ ETCDCTL_API=3 etcdctl \
   --cacert=/etc/kubernetes/pki/etcd/ca.crt   \
   --cert=/etc/kubernetes/pki/etcd/server.crt \
   --key=/etc/kubernetes/pki/etcd/server.key  \
   get /registry/secrets/default/my-secret | hexdump -C
00000000  2f 72 65 67 69 73 74 72  79 2f 73 65 63 72 65 74  |/registry/secret|
00000010  73 2f 64 65 66 61 75 6c  74 2f 6d 79 2d 73 65 63  |s/default/my-sec|
00000020  72 65 74 0a 6b 38 73 3a  65 6e 63 3a 61 65 73 63  |ret.k8s:enc:aesc|
00000030  62 63 3a 76 31 3a 6b 65  79 31 3a 9b 89 f8 11 13  |bc:v1:key1:.....|
00000040  65 bd 8b 72 ac f4 bc 28  ad b3 26 99 09 af 11 e2  |e..r...(..&.....|
00000050  2a 84 c3 c3 0e e7 de f6  d4 46 80 02 ae ee 3d c3  |*........F....=.|
00000060  22 38 5d cc 64 c5 96 75  d9 5c d3 85 5c b1 7d 03  |"8].d..u.\..\.}.|
00000070  62 d3 67 f1 b2 6e f5 dd  1e d2 5d 24 29 2a 17 78  |b.g..n....]$)*.x|
00000080  5e 60 2c 59 82 da 8a d2  9a fd b5 7e 58 7d 07 fc  |^`,Y.......~X}..|
00000090  30 2a 01 39 fa ec f5 db  94 77 03 da 9f 81 03 d1  |0*.9.....w......|
000000a0  8e af 54 3d 59 55 f6 48  ff af b2 f5 d5 ff f9 4a  |..T=YU.H.......J|
000000b0  65 a2 8b ed a2 df d7 54  79 cc b9 a1 db 30 cf ee  |e......Ty....0..|
000000c0  7e a0 c4 2d 86 d8 6d 32  5b 50 cd 03 10 56 fc 9f  |~..-..m2[P...V..|
000000d0  47 8b 31 db 3d a5 e4 e8  d8 29 3e 07 a4 7e bc d5  |G.1.=....)>..~..|
000000e0  99 a1 39 14 f1 ea d0 81  7f da 09 f6 aa 4f 8c 81  |..9..........O..|
000000f0  09 07 f3 f2 1e f5 ef 13  e5 37 aa de 1d a1 71 90  |.........7....q.|
00000100  28 0e 56 10 32 8a 03 13  9d af 6c e6 a6 33 2a fe  |(.V.2.....l..3*.|
00000110  ca 31 df 80 37 6d 64 ee  1f f1 8b ac 5c 48 68 ba  |.1..7md.....\Hh.|
00000120  27 d8 42 96 36 fb 79 fc  82 6a a5 4a c6 34 0c 6e  |'.B.6.y..j.J.4.n|
00000130  c3 3b 4d 88 ed c5 2a b5  27 4b d8 0a              |.;M...*.'K..|
0000013c

my-secret을 조회해봐도 평문을 확인할 수 없다.

Yu Sang Min

React, Node.js, AWS, Git, Github, Github Action, Docker, K8S

이전 포스트

Configure Secrets in Applications

다음 포스트

Encrypting Confidential Data at Rest

CKA

🔐 이미 암호화가 활성화 되어있는지 확인 하기

👉🏻 구성파일을 만들고 옵션으로 전달하기

📝 새로운 secret 만들기

♻️ 기존에 생성된 평문 secret 전체 암호화 방법

Configure Secrets in Applications

Init Container

0개의 댓글