🙂0804 [EKS CI/CD - Ansible,Jenkins]

망지·2022년 8월 4일

📌 EKS CI/CD

📌 EKS 준비

📙 인스턴스 생성

이름 : eks-server
이미지 : amazon linux 2
유형 : t2.micro
키페어 선택
네트워크 : MY-VPC, PUB2C
보안그룹 : DEV-SG
사용자데이터 :

#!/bin/bash
timedatectl set-timezone Asia/Seoul
hostnamectl set-hostname eks-server
cd /tmp
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
./aws/install
curl -o kubectl https://amazon-eks.s3-us-west-2.amazonaws.com/1.21.2/2021-07-05/bin/linux/amd64/kubectl
chmod +x ./kubectl
mv ./kubectl /usr/local/bin
echo "source <(kubectl completion bash)" >> /home/ec2-user/.bashrc
curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
mv /tmp/eksctl /usr/local/bin

만들어두고 중지시킨 jenkins,ansible 인스턴스 실행.

📙 레코드 생성

📙 mobaxterm으로 진입하여 버전확인

[ec2-user@eks-server ~]$ aws --version
aws-cli/2.7.21 Python/3.9.11 Linux/5.10.130-118.517.amzn2.x86_64 exe/x86_64.amzn.2 prompt/off
[ec2-user@eks-server ~]$ kubectl version
Client Version: version.Info{Major:"1", Minor:"21+", GitVersion:"v1.21.2-13+d2965f0db10712", GitCommit:"d2965f0db1071203c6f5bc662c2827c71fc8b20d", GitTreeState:"clean", BuildDate:"2021-06-26T01:02:11Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
[ec2-user@eks-server ~]$ eksctl version
0.107.0

📙 키형식에서 패스워드로 바꿔주기

[ec2-user@eks-server ~]$ sudo passwd ec2-user
Changing password for user ec2-user.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
[ec2-user@eks-server ~]$ sudo vi /etc/ssh/sshd_config
PasswordAuthentication yes
#PermitEmptyPasswords no
#PasswordAuthentication no
[ec2-user@eks-server ~]$ sudo systemctl restart sshd

📙 IAM 역할

IAM - 역할 - 역할 만들기 - 신뢰할 수 있는 엔터티 ; AWS 서비스 - 일반 사용 사례 : EC2 - 다음

AmazonEC2FullAccess 검색하여 체크 - 필터 지우기 -IAMFullAccess 검색하여 체크 - AdministratorAccess 검색하여 체크 -
AWSCloudFormationFullAccess 검색하여체크 - 다음

역할 이름 : eksctl_role - 역할 생성 클릭

EC2 대시보드 - 인스턴스 - eks-server 선택 - 마우스 우클릭 - 보안 - IAM 역할 수정

IAM 역할에서 eksctl_role 선택하고 업데이트

📙 클러스터 생성 및 노드그룹 생성

[ec2-user@eks-server ~]$
eksctl create cluster --name EKS-CLUSTER --region ap-northeast-2 --version 1.21 --vpc-public-subnets subnet-00d4b7ccf0a5221f0,subnet-0833b70ce2cfd51cb --without-nodegroup

[ec2-user@eks-server ~]$
eksctl create nodegroup \
  --cluster EKS-CLUSTER \
  --region ap-northeast-2 \
  --name NODEGROUP \
  --node-type t2.micro \
  --nodes 4 \
  --nodes-min 4 \
  --nodes-max 8 \
  --ssh-access \
  --ssh-public-key docker-key3

📙 ansible과 eks서버 연결

anisible 서버에서 진행


    
    
[ec2-user@ansible-server docker]$ sudo vi /etc/ansible/hosts ## 이 경로에 eks 내부 IP 추가
[eks-server]
10.14.42.197

[ec2-user@ansible-server docker]$ ssh-copy-id 10.14.42.197
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/ec2-user/.ssh/id_rsa.pub"
The authenticity of host '10.14.42.197 (10.14.42.197)' can't be established.
ECDSA key fingerprint is SHA256:7mAVCmKs8gPFNHMdMCEEQi4kl8lafBsu/OObvZqJ5Kk.
ECDSA key fingerprint is MD5:b4:c3:ba:25:89:0c:13:cd:96:d4:e2:60:cf:c7:f5:5e.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys

[ec2-user@ansible-server docker]$ ansible all -m ping
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details
[WARNING]: Platform linux on host 10.14.9.46 is using the discovered Python interpreter at /usr/bin/python, but
future installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.
10.14.9.46 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": false,
    "ping": "pong"
}
[WARNING]: Platform linux on host 10.14.42.197 is using the discovered Python interpreter at /usr/bin/python,
but future installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.
10.14.42.197 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": false,
    "ping": "pong"
}

📙 ansible서버에서 이미지 관리에 필요한 build.yml 실행해보기

[ec2-user@ansible-server ~]$ cd /opt/docker
[ec2-user@ansible-server docker]$ ll
total 944
-rw-rw-r-- 1 ec2-user ec2-user    228 Aug  3 11:48 build.yml
-rw-rw-r-- 1 ec2-user ec2-user    286 Aug  3 12:24 deploy.yml
-rw-rw-r-- 1 ec2-user ec2-user    122 Aug  3 11:23 Dockerfile
-rw-rw-r-- 1 ec2-user ec2-user 951830 Aug  3 14:13 webapp.war

[ec2-user@ansible-server docker]$ vi build.yml
- hosts: ansible-server

  tasks:
  - name: remove docker image
    command: docker rmi -f mj030kk/mytomcat:latest

  - name: create docker image
    command: docker build -t mj030kk/mytomcat:latest .
    args:
      chdir: /opt/docker

  - name: push docker image
    command: docker push mj030kk/mytomcat:latest
    
[ec2-user@ansible-server docker]$ ansible-playbook build.yml

-> 정상 작동 확인

📙 ansible 서버 yml, eks 서버 yaml파일 생성.

[ec2-user@ansible-server docker]$  cd /opt/docker

[ec2-user@ansible-server docker]$  vi kube-deploy-svc.yml
- hosts: eks-server

  tasks:
  - name: remove deploy
    command: kubectl delete -f /home/ec2-user/test/deployment.yaml
    ignore_errors: yes
  - name: kube deploy
    command: kubectl apply -f /home/ec2-user/test/deployment.yaml
  - name: kube svc
    command: kubectl apply -f /home/ec2-user/test/svc.yaml
  - name: update
    command: kubectl rollout restart deployment/web-site-deployment

[ec2-user@eks-server ~]$ mkdir test && cd $_

[ec2-user@eks-server test]$ vi deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-site-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web-site-deployment
  template:
    metadata:
      name: web-site-deployment
      labels:
        app: web-site-deployment
    spec:
      containers:
      - name: web-site-deployment-container
        image: mj030kk/mytomcat:latest
        imagePullPolicy: Always
        
[ec2-user@eks-server test]$ vi svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: loadbalancer-service-deployment
spec:
  type: LoadBalancer
  selector:
    app: web-site-deployment
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 8080

서비스와 deployment 분리 이유 : svc까지 지웠다가 다시만들면 접속 주소가 계속 바뀜. deploy만 바꿔주는게 더 적절.

[ec2-user@eks-server test]$ kubectl apply -f deployment.yaml
deployment.apps/web-site-deployment created
[ec2-user@eks-server test]$ kubectl apply -f svc.yaml
service/loadbalancer-service-deployment created

[ec2-user@eks-server test]$ kubectl get all
NAME                                      READY   STATUS    RESTARTS   AGE
pod/web-site-deployment-b455c4dd4-g2dpt   1/1     Running   0          22s
pod/web-site-deployment-b455c4dd4-qpzw2   1/1     Running   0          22s

NAME                                      TYPE           CLUSTER-IP      EXTERNAL-IP                                                                    PORT(S)          AGE
service/kubernetes                        ClusterIP      172.20.0.1      <none>                                                                         443/TCP          50m
service/loadbalancer-service-deployment   LoadBalancer   172.20.160.25   a23e76eb992bf4fc79651a0b4685506d-1520966570.ap-northeast-2.elb.amazonaws.com   8080:30967/TCP   11s

NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/web-site-deployment   2/2     2            2           23s

NAME                                            DESIRED   CURRENT   READY   AGE
replicaset.apps/web-site-deployment-b455c4dd4   2         2         2       23s

로드밸런서 주소 :8080

로드밸런서 주소 :8080/webapp

[ec2-user@eks-server test]$ kubectl delete -f .

📙 ansible - eks 연결 확인 (ansible playbook)

[ec2-user@ansible-server docker]$ ansible-playbook kube-deploy-svc.yml

[ec2-user@eks-server test]$ kubectl get svc
NAME                              TYPE           CLUSTER-IP      EXTERNAL-IP                                                                    PORT(S)          AGE
kubernetes                        ClusterIP      172.20.0.1      <none>                                                                         443/TCP          71m
loadbalancer-service-deployment   LoadBalancer   172.20.216.19   a459dbcb64f494db0a7fd15ccf3e4803-2089817562.ap-northeast-2.elb.amazonaws.com   8080:32673/TCP   2m1s

-> 정상작동 확인

📌 젠킨스에서 빌드하기

젠킨스 대시보드 - 새로운 아이템 - 이름 : Deploy_On_Kube_Ansible - Copy from : Copy_Artifacts_onto_Ansible - OK

설명 : 메이븐으로 코드를 빌드하고 앤서블을 활용해서 쿠버네티스(EKS)로 배포하자 - 나머지 그대로, 빌드 후 조치 exec command :

cd /opt/docker;
ansible-playbook build.yml;
sleep 10;
ansible-playbook kube-deploy-svc.yml

apply, 저장 - 지금 빌드

아래와 같이 확인 완료.

[ec2-user@eks-server test]$ kubectl get all
NAME                                       READY   STATUS    RESTARTS   AGE
pod/web-site-deployment-7c66bb5dc8-lsj2p   1/1     Running   0          29s
pod/web-site-deployment-7c66bb5dc8-n6pf9   1/1     Running   0          55s

NAME                                      TYPE           CLUSTER-IP      EXTERNAL-IP                                                                    PORT(S)          AGE
service/kubernetes                        ClusterIP      172.20.0.1      <none>                                                                         443/TCP          87m
service/loadbalancer-service-deployment   LoadBalancer   172.20.216.19   a459dbcb64f494db0a7fd15ccf3e4803-2089817562.ap-northeast-2.elb.amazonaws.com   8080:32673/TCP   17m

NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/web-site-deployment   2/2     2            2           59s

NAME                                             DESIRED   CURRENT   READY   AGE
replicaset.apps/web-site-deployment-7c66bb5dc8   2         2         2       55s
replicaset.apps/web-site-deployment-b455c4dd4    0         0         0       59s

엔드포인트 접속 주소는 바뀌지 않은 모습.

📙 서비스주소에 도메인 할당

📙 웹페이지 수정

git bash

r2com@DESKTOP-DD3FU43 MINGW64 ~
$ rm -rf hello-world/

r2com@DESKTOP-DD3FU43 MINGW64 ~
$ git clone https://github.com/mangjini/hello-world.git
Cloning into 'hello-world'...
remote: Enumerating objects: 572, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 572 (delta 0), reused 5 (delta 0), pack-reused 565
Receiving objects: 100% (572/572), 830.88 KiB | 2.25 MiB/s, done.
Resolving deltas: 100% (154/154), done.

r2com@DESKTOP-DD3FU43 MINGW64 ~
$ cd hello-world/

r2com@DESKTOP-DD3FU43 MINGW64 ~/hello-world (master)
$ ls
Dockerfile  pom.xml            regapp-service.yml  test.tml
README.md   regapp-deploy.yml  server/             webapp/

r2com@DESKTOP-DD3FU43 MINGW64 ~/hello-world (master)
$ cd webapp/src/main/webapp/

r2com@DESKTOP-DD3FU43 MINGW64 ~/hello-world/webapp/src/main/webapp (master)
$ ls
WEB-INF/  css/     images/    index.jsp.bak  js/
assets/   gcp.tar  index.jsp  index.jsp.old  test.txt

r2com@DESKTOP-DD3FU43 MINGW64 ~/hello-world/webapp/src/main/webapp (master)
$ vi index.jsp

r2com@DESKTOP-DD3FU43 MINGW64 ~/hello-world/webapp/src/main/webapp (master)
$ git add .

r2com@DESKTOP-DD3FU43 MINGW64 ~/hello-world/webapp/src/main/webapp (master)
$ git commit -m "hi mj index.jsp"
[master 0be2874] hi mj index.jsp
 1 file changed, 1 insertion(+), 1 deletion(-)
 
r2com@DESKTOP-DD3FU43 MINGW64 ~/hello-world/webapp/src/main/webapp (master)
$ git push origin master
Enumerating objects: 13, done.
Counting objects: 100% (13/13), done.
Delta compression using up to 4 threads
Compressing objects: 100% (5/5), done.
Writing objects: 100% (7/7), 507 bytes | 507.00 KiB/s, done.
Total 7 (delta 3), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
To https://github.com/mangjini/hello-world.git
   8a0b4f1..0be2874  master -> master

자동 빌드 확인

변경된 웹페이지 확인

📙 eks 삭제

eks-server에서 진행

kubectl delete all -all
eksctl delete cluster EKS-CLUSTER --region ap-northeast-2

📌 SAA - 엔드포인트

s3 - 버킷생성 - 이름 : s3.lovemj.shop - 버킷만들기
버킷- s3.lovemj.shop - 속성 - 정적 웹사이트 호스팅 활성화 - 인덱스 문서 - index.html - 변경사항 저장

레코드 생성

프라이빗 서브넷에 ec2 생성
이름 : private-ec2
이미지 : amazon linux 2
유형 : t2.micro
키페어 선택
네트워크 : MY-VPC, PRI2A
보안그룹 : DEV-SG

ansible서버에서 private로 진입
private에 eksctl role 역할 부여

[ec2-user@ansible-server ~]$ chmod 400 docker-key3.pem
[ec2-user@ansible-server ~]$ ssh -i docker-key3.pem ec2-user@10.14.79.12

       __|  __|_  )
       _|  (     /   Amazon Linux 2 AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-2/
[ec2-user@ip-10-14-79-12 ~]$ ping s3.lovemj.shop
PING s3.lovemj.shop (52.219.58.64) 56(84) bytes of data.
^C
--- s3.lovemj.shop ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1021ms

[ec2-user@ip-10-14-79-12 ~]$ aws s3 ls s3://s3.lovemj.shop
Unable to locate credentials. You can configure credentials by running "aws configure".
[ec2-user@ip-10-14-79-12 ~]$ aws s3 ls s3://s3.lovemj.shop

엔드포인트 이용하면 아웃바운드 데이터트랜스퍼 비용 발생 X.
내부망 이용하기때문에 보안적으로도 좋음.

📙 ✔️✏️📢⭐️📌

망지

꾸준히, 차근차근

이전 포스트

0803[CI/CD-Ansible,Jenkins]

다음 포스트