ImagePullSecrets

ECR에서 Image를 Pull해서 실행하기 위해선 AWS CLI 인증이 필요하다.

K8S 환경에서는 이를 secret으로 관리해 deployment 및 service를 생성하는 yml에서 사용해 과정을 간략화할 수 있다.

SECRET 작업

1. AWS Login

$ aws configure 
AWS Access Key ID [None]: # ACCESS KEY
AWS Secret Access Key [None]: # SECRET KEY
Default region name [None]: 
Default output format [None]:

2. ECR Login

aws ecr get-login-password --region ap-northeast-2 \
--profile default | docker login --username AWS \
--password-stdin <ACCOUNT ID>.dkr.ecr.<Your-Region>.amazonaws.com

3. ECR-SECRET

kubectl create secret generic ecr-secret \
    --from-file=.dockerconfigjson=/root/.docker/config.json \
    --type=kubernetes.io/dockerconfigjson \
    --namespace=<name-space>

kubectl create secret docker-registry ecr-secret \
--docker-server=846035848117.dkr.ecr.<Your-Region>.amazonaws.com \
--docker-username=AWS \
--docker-password=$(aws ecr get-login-password --region <Your-Region>) \
--namespace=<name-space>

4. Cron Job

ECR-SECRET 으로 생성한 시크릿은 aws에서 발급해주는 토큰으로 만료기간이 12시간이다. 따라서 6시간마다 갱신해주는 작업이 필요하므로 이를 Cron Job으로 구현해 사용하자.

aws-credential.yml

apiVersion: v1
kind: Secret
metadata:
  name: aws-credentials
  namespace: <배포할 NS 이름>
type: Opaque
data:
  aws_access_key_id: <Base64-encoded-access-key-id>
  aws_secret_access_key: <Base64-encoded-secret-access-key>

secret-rbac.yml

secret 리소스는 권한 없이 default service account로는 삭제할 수 없으므로 권한을 추가해주어야 한다.

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: production
  name: secret-manager
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["create", "delete", "get", "list", "patch", "update"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: secret-manager-binding
  namespace: production
subjects:
- kind: ServiceAccount
  name: default
  namespace: production
roleRef:
  kind: Role
  name: secret-manager
  apiGroup: rbac.authorization.k8s.io

ecr-credentials-refresh.yml

AWS CLI 이미지에는 쿠버네티스가 없으므로 설치해주어야한다.

apiVersion: batch/v1
kind: CronJob
metadata:
  name: ecr-credentials-refresh
  namespace: production
spec:
  schedule: "0 */6 * * *"  # Every 6 hours
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: refresh-ecr-credentials
            image: amazon/aws-cli
            command:
            - /bin/sh
            - -c
            - |
              curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl &&
              chmod +x ./kubectl &&
              mv ./kubectl /usr/local/bin/kubectl &&
              echo "Deleting existing secret..." &&
              kubectl delete secret ecr-secret -n production &&
              echo "Creating new secret..." &&
              kubectl create secret docker-registry ecr-secret \
                --docker-server=<AWS_ACCOUNT_NUM>.dkr.ecr.ap-northeast-2.amazonaws.com \
                --docker-username=AWS \
                --docker-password=$(aws ecr get-login-password --region <REAGION>) \
                --namespace=<NAMESPACE>
            env:
            - name: AWS_ACCESS_KEY_ID
              valueFrom:
                secretKeyRef:
                  name: aws-credentials
                  key: aws_access_key_id
            - name: AWS_SECRET_ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  name: aws-credentials
                  key: aws_secret_access_key
          restartPolicy: OnFailure

Image Pull

Deployment & Service

apiVersion: apps/v1
kind: Deployment
metadata:
  name: contract-service-deployment
  labels:
    app: contract-service-app
spec:
  replicas: 3 # 원하는 파드 수
  selector:
    matchLabels:
      app: contract-service-app
  template:
    metadata:
      labels:
        app: contract-service-app
    spec:
      containers:
        - name: contract-service-app
          image: <ECR-SRC-URL>
          ports:
            - containerPort: 9000
      imagePullSecrets:
        - name: ecr-secret
---
apiVersion: v1
kind: Service
metadata:
  name: contract-service
  labels:
    app: contract-service-app
spec:
  type: NodePort
  selector:
    app: contract-service-app
  ports:
    - protocol: TCP
      port: 9000
      targetPort: 9000