[Cloude] AWS EKS

✔️ AWS EKS

• Amazon Elastic Kubernetes Service(Amazon EKS)
• EKS는 Kubernetes를 실행하는 데 사용할 수 있는 관리형 서비스이다.
• 노드를 설치, 작동 및 유지 관리할 필요가 없다는 장점

https://docs.aws.amazon.com/ko_kr/eks/latest/userguide/what-is-eks.html

✔️ AWS EKS 사용해보기

$ choco install awscli aws-iam-authenticator eksctl kubernetes-helm

aws 계정 등록

$ aws configure

$ eksctl create cluster --name myeks --nodes=3 --region=ap-northeast-2

⭐ Load Balancer Service ❌ = class lb로생성되기 때문에 nlb로 변경해주어야함
⭐ Ingress: ❌

✔️ YAML 파일을 이용해서 EKS 배포하기

$ mkdir aws-eks
$ cd aws-eks

myeks.yaml

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: myeks-custom  # 이름은 같을 수 없다
  region: ap-northeast-2
  version: "1.22"

가용영역 지정
- 필수는 아니나 일반적으로 지정하는 편

# AZ
availabilityZones: ["ap-northeast-2a", "ap-northeast-2b",  "ap-northeast-2c"]

eks와 AWS iam계정을 연결 해 주는 부분
wellKnownPolicies : 계정에 해당되는 기본적인 권한 설정

# IAM OIDC & Service Account
iam:
  withOIDC: true
  serviceAccounts:
    - metadata:
        name: aws-load-balancer-controller # Addon 추가할 때 필요
        namespace: kube-system
      wellKnownPolicies:
        awsLoadBalancerController: true
    - metadata:
        name: ebs-csi-controller-sa
        namespace: kube-system
      wellKnownPolicies:
        ebsCSIController: true
    - metadata:
        name: cluster-autoscaler
        namespace: kube-system
      wellKnownPolicies:
        autoScaler: true

Worker노드의 그룹

# Managed Node Groups
managedNodeGroups:
  # On-Demand Instance
  - name: myeks-ng1
    instanceType: t3.medium
    minSize: 2
    desiredCapacity: 3
    maxSize: 4
    privateNetworking: true
    ssh:
      allow: true
      publicKeyPath: ./keypair/myeks.pub
    availabilityZones: ["ap-northeast-2a", "ap-northeast-2b", "ap-northeast-2c"]
    iam:
      withAddonPolicies:
        autoScaler: true
        albIngress: true
        cloudWatch: true
        ebs: true

# Fargate Profiles
fargateProfiles:
  - name: fg-1
    selectors:
    - namespace: dev
      labels:
        env: fargate
        

# CloudWatch Logging
cloudWatch:
  clusterLogging:
    enableTypes: ["*"]
    

키 생성

$ mkdir keypair
$ ssh-keygen -f keypair/myssh

$ eksctl create cluster -f myeks.yaml

✔️ NLB for LoadBalancer Service

✔️ AWS Load Balancer Controller 설치

$ helm repo add eks https://aws.github.io/eks-charts
$ helm repo update

Account 확인

$ aws sts get-caller-identity
{
    "UserId": "AIxxx",
    "Account": "4xxx",
    "Arn": "arn:aws:iam::4xxx"
}

$ helm install aws-load-balancer-controller eks/aws-load-balancer-controller -n kube-system --set clusterName=myeks-lunaris --set serviceAccount.create=false --set serviceAccount.name=aws-load-balancer-controller --set image.repository=[account]412059376128.dkr.ecr.ap-northeast-2.amazonaws.com/amazon/aws-load-balancer-controller

NAME: aws-load-balancer-controller
LAST DEPLOYED: Mon May 30 20:56:35 2022
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
AWS Load Balancer controller installed!

샘플 코드
myapp.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myweb-deploy
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: myweb
          image: ghcr.io/c1t1d0s7/go-myweb
          ports:
            - containerPort: 8080

`mysvc.yaml

apiVersion: v1
kind: Service
metadata:
  name: myweb-svc-lb
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "external"
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "instance"
    service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
spec:
  type: LoadBalancer
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080

✔️ Ingress for ALB

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myweb-ing
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/target-type: instance
    alb.ingress.kubernetes.io/scheme: internet-facing
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: myweb-svc-lb
                port:
                  number: 80

• alb.ingress.kubernetes.io/target-type
  - instance: EC2 타겟
  - ip: Pod 타겟(Fargate)
• alb.ingress.kubernetes.io/scheme
  - internal: 내부
  - internet-facing: 외부

✔️ EBS for CSI

EBS 스냅샷
EBS 크기 변경

$ eksctl get iamserviceaccount --cluster myeks-custom

NAMESPACE       NAME                            ROLE ARN
kube-system     aws-load-balancer-controller    arn:aws:iam::xxxxx:role/eksctl-myeks-custom-addon-iamserviceaccount-Role1-11N0OKMVG2DYY
kube-system     aws-node                        arn:aws:iam::xxxx:role/eksctl-myeks-custom-addon-iamserviceaccount-Role1-CLMK7A6K5NL3
kube-system     cluster-autoscaler              arn:aws:iam::xxxx:role/eksctl-myeks-custom-addon-iamserviceaccount-Role1-1S02W28MZOSL4
kube-system     ebs-csi-controller-sa           arn:aws:iam::xxxx:role/eksctl-myeks-custom-addon-iamserviceaccount-Role1-15HLE8HBOD9CN

$ eksctl create addon --name aws-ebs-csi-driver --cluster myeks-lunaris--service-account-role-arn  arn:aws:iam::xxxx:role/eksctl-myeks-lunaris-addon-iamserviceaccount-Role1-15HLE8HBOD9CN --force

✔️ Metrics Server

$ kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

✔️ Cluster Autoscaler

✔️ 수동 스케일링

$ eksctl scale nodegroup --name myeks-ng1 --cluster myeks-lunaris --nodes 2

✔️자동 스케일링

$ curl -o cluster-autoscaler-autodiscover.yaml https://raw.githubusercontent.com/kubernetes/autoscaler/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yaml

cluster-autoscaler-autodiscover.yaml

...
163: - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/myeks-lunaris
...

$ kubectl apply -f cluster-autoscaler-autodiscover.yaml

$ kubectl patch deployment cluster-autoscaler -n kube-system -p '{"spec":{"template":{"metadata":{"annotations":{"cluster-autoscaler.kubernetes.io/safe-to-evict": "false"}}}}}'

$ kubectl -n kube-system edit deployment.apps/cluster-autoscaler

      - command:
        - ./cluster-autoscaler
        - --v=4
        - --stderrthreshold=info
        - --cloud-provider=aws
        - --skip-nodes-with-local-storage=false
        - --expander=least-waste
        - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/myeks-custom
        - --balance-similar-node-groups
        - --skip-nodes-with-system-pods=false
        image: k8s.gcr.io/autoscaling/cluster-autoscaler:v1.22.6

수정

• --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/myeks-custom
• --balance-similar-node-groups
• --skip-nodes-with-system-pods=false
• image: k8s.gcr.io/autoscaling/cluster-autoscaler:v1.22.2

$ kubectl set image deployment cluster-autoscaler -n kube-system cluster-autoscaler=k8s.gcr.io/autoscaling/cluster-autoscaler:v1.22.2

샘플 코드

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myweb-deploy
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: myweb
          image: ghcr.io/c1t1d0s7/go-myweb:alpine
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: 200m
              memory: 200M
            limits:
              cpu: 200m
              memory: 200M

✔️CloudWatch Container Insight

ClusterName=myeks-custom
RegionName=ap-northeast-2
FluentBitHttpPort='2020'
FluentBitReadFromHead='Off'
[[ ${FluentBitReadFromHead} = 'On' ]] && FluentBitReadFromTail='Off'|| FluentBitReadFromTail='On'
[[ -z ${FluentBitHttpPort} ]] && FluentBitHttpServer='Off' || FluentBitHttpServer='On'
curl https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/latest/k8s-deployment-manifest-templates/deployment-mode/daemonset/container-insights-monitoring/quickstart/cwagent-fluent-bit-quickstart.yaml | sed 's/{{cluster_name}}/'${ClusterName}'/;s/{{region_name}}/'${RegionName}'/;s/{{http_server_toggle}}/"'${FluentBitHttpServer}'"/;s/{{http_server_port}}/"'${FluentBitHttpPort}'"/;s/{{read_from_head}}/"'${FluentBitReadFromHead}'"/;s/{{read_from_tail}}/"'${FluentBitReadFromTail}'"/' | kubectl apply -f - 

✔️ 클러스터 삭제

eksctl delete cluster -f .\myeks.yaml --force --disable-nodegroup-eviction