42gg Spring projcet CI/CD 적용기

HyunKyu Lee·2023년 11월 28일
42GGCI/CD
0

42gg

목록 보기
4/4

CI/CD 적용하게 된 이유

  • 기존 배포 프로세스 -> ec2에서 tomcat server를 실행시키는데 local에서 build한 파일을 직접 ec2에 옮겨넣고 tomcat 서버를 실행시키는 방식
  • 비효율적이라는 생각이 들었고, 매우 불편해서 CI/CD를 적용해보기로함.

배포 서비스 종류

1. Tomcat -> 스프링 부트 내장 톰켓, Rest API서버
2. Redis -> Jwt, user rank 관리용
3. Mysql -> 데이터 관리용
4. Prometheus -> metric 데이터 관리용
5. Grafana -> 모니터링 툴
6. Nginx -> https 적용

배포 프로세스

  • Prometheus를 위한 Spring actuator 포트 설정 *중요
    • 기본 8080 포트로 열리는데 yml설정에서 8081로 바꿔주자
    • 보안상 민감한 데이터들이 8080포트에서 열리면 위험, 8081 포트에서 열고 인바운드 규칙으로 보안설정
  • gitAction을 활용하여 배포 자동화
    • git action 스크립트에서 gradle build, docker image build -> docker hub push 후 aws ec2원격 접속하여 docker hub 이미지를 실행시키는 방식
  • docker hub image tag를 활용하여 버전 관리 시작

각 서비스 Dockerfile

  • Tomact
FROM openjdk:11-jdk

WORKDIR /app

ARG JAR_FILE=build/libs/server-42gg.jar

COPY ${JAR_FILE} .

EXPOSE 8080
ENTRYPOINT ["java","-jar","server-42gg.jar", \
"--spring.profiles.active=${PROFILE}", \
"--spring.security.oauth2.client.registration.42.client-id=${SPRING_42_CLIENT_ID}", \
"--spring.security.oauth2.client.registration.42.client-secret=${SPRING_42_CLIENT_SECRET}"]
  • env로 설정한 PROFILE로 test, main 서버에 맞춘 yml설정에 맞추어 build되도록함
  • 42Oauth SPRING_42_CLIENT_ID, SPRING_42_CLIENT_SECRET은 한달 주기로 갱신해주어야 하기 때문에 갱신하기 쉽도록 env 파일에서 읽어오도록 설정

  • Nginx
FROM nginx:stable-alpine3.17

RUN mkdir -p /etc/letsencrypt/live/api.42gg.kr

WORKDIR /etc/letsencrypt/live/api.42gg.kr

COPY ./secrets/fullchain.pem .
COPY ./secrets/privkey.pem .

WORKDIR /etc/letsencrypt

COPY ./secrets/options-ssl-nginx.conf .
COPY ./secrets/ssl-dhparams.pem .

WORKDIR /etc/nginx/conf.d

COPY ./config/default.conf .

EXPOSE 80 443

  • docker-compose
version: '3.8'

networks:
  42gg:
    ipam:
      driver: default
      config:
        - subnet: 172.26.0.0/24

services:
  tomcat:
    container_name: tomcat
    image: wken5577/42gg-server
    restart:
      unless-stopped
    depends_on:
      - mysql
      - redis
      - nginx
    env_file:
      - .env
    volumes:
      - /home/ec2-user/logs:/app/logs
    networks:
      42gg:
        ipv4_address: 172.26.0.13

  nginx:
    container_name: nginx
    build:
      context: ./nginx
      dockerfile: Dockerfile
    ports:
      - "80:80"
      - "443:443"
    restart:
      unless-stopped
    networks:
      42gg:
        ipv4_address: 172.26.0.10

  redis:
    container_name: redis
    image: redis:6.2.7
    ports:
      - "6379:6379"
    restart:
      unless-stopped
    networks:
      42gg:
        ipv4_address: 172.26.0.12

  mysql:
    container_name: mysql
    image: mysql:8.0.33
    ports:
      - "3306:3306"
    volumes:
      - /var/lib/mysql:/var/lib/mysql
    env_file:
      - .env
    restart:
      unless-stopped
    networks:
      42gg:
        ipv4_address: 172.26.0.11

  prometheus:
    container_name: prometheus
    build:
      context: ./prometheus
      dockerfile: Dockerfile
    restart:
      unless-stopped
    networks:
      42gg:

  grafana:
    container_name: grafana
    build:
      context: ./grafana
      dockerfile: Dockerfile
    volumes:
      - /var/lib/grafana:/var/lib/grafana:rw
    restart:
      unless-stopped
    networks:
      42gg:

work flow 설명

# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.
# This workflow will build a Java project with Gradle and cache/restore any dependencies to improve the workflow execution time
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-java-with-gradle

name: main-deploy

on:
  workflow_dispatch:

env:
  DOCKER_USER: ${{ secrets.DOCKER_USER }}
  DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
  IMAGE_NAME: wken5577/42gg-server
  LAST_VERSION_TAG : ${{ secrets.LAST_VERSION_TAG }}

permissions:
  contents: read

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Set up JDK 11
        uses: actions/setup-java@v3
        with:
          java-version: '11'
          distribution: 'temurin'

      - name: Make application ymls
        run: |
          cd ./src/main/resources
          touch ./application.yml
          touch ./application-main.yml
          echo "${{ secrets.YML_ACTIVE_MAIN }}" | base64 -d > application.yml
          echo "${{ secrets.APPLICATION_MAIN_YML }}" | base64 -d > application-main.yml
        shell: bash

      - name: Login to Docker Hub
        uses: docker/login-action@v1
        with:
          username: ${{ env.DOCKER_USER }}
          password: ${{ env.DOCKER_PASSWORD }}

      - name: Build with Gradle
        run: ./gradlew bootWar

      - name: build new docker image as latest tag
        run: |
          docker build -t ${{ env.IMAGE_NAME }}:${{ env.VERSION_TAG }} .
          docker push ${{ env.IMAGE_NAME }}:${{ env.VERSION_TAG }}
          docker tag ${{ env.IMAGE_NAME }}:${{ env.VERSION_TAG }} ${{ env.IMAGE_NAME }}:latest
          docker push ${{ env.IMAGE_NAME }}:latest

      - name: Get Github Actions IP
        id: ip
        uses: haythem/public-ip@v1.3

      - name: AWS Credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          aws-access-key-id: ${{ secrets.AWS_SECURITY_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECURITY_SECRET_KEY }}
          aws-region: ap-northeast-2

      - name: Add Github Actions IP to Security group
        run: |
          aws ec2 authorize-security-group-ingress --group-id ${{ secrets.AWS_MAIN_SG_ID }} --protocol tcp --port 22 --cidr ${{ steps.ip.outputs.ipv4 }}/32

      - name: executing docker-compose up on test server
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.MAIN_SERVER_HOST }}
          username: ${{ secrets.MAIN_SERVER_USERNAME }}
          key: ${{ secrets.MAIN_SERVER_PEM }}
          script: |
            cd ./docker
            docker-compose down
            docker rmi ${{ env.IMAGE_NAME }}:latest
            docker-compose up -d

      - name: Remove Github Actions IP From Security Group
        run: |
          aws ec2 revoke-security-group-ingress --group-id ${{ secrets.AWS_MAIN_SG_ID }} --protocol tcp --port 22 --cidr ${{ steps.ip.outputs.ipv4 }}/32

  • Make application ymls
   - name: Make application ymls
        run: |
          cd ./src/main/resources
          touch ./application.yml
          touch ./application-main.yml
          echo "${{ secrets.YML_ACTIVE_MAIN }}" | base64 -d > application.yml
          echo "${{ secrets.APPLICATION_MAIN_YML }}" | base64 -d > application-main.yml
        shell: bash
  • github secret에 base64로 인코딩된 값을 echo로 출력하면서 디코딩하여 각 파일에 저장
  • base64로 인코딩한 이유 → yml파일에 “”, ‘’(quote, dquote) 이 불규칙적으로 작성되어있어 echo로 출력시 인식을 못함

 - name: build new docker image as latest tag
        run: |
          docker build -t ${{ env.IMAGE_NAME }}:${{ env.VERSION_TAG }} .
          docker push ${{ env.IMAGE_NAME }}:${{ env.VERSION_TAG }}
          docker tag ${{ env.IMAGE_NAME }}:${{ env.VERSION_TAG }} ${{ env.IMAGE_NAME }}:latest
          docker push ${{ env.IMAGE_NAME }}:latest
  • 지정한 version tag 로 이미지 build 후 push
  • tag 만 latest 태그로 바꾸어서 push
    • 서버에서는 무조건 latest 이미지를 실행시킴

- name: Get Github Actions IP
        id: ip
        uses: haythem/public-ip@v1.3
  • git action서버에서 aws에 ssh로 접속하기 위해 보안그룹을 추가해주어야하는데 git action서버의 ip를 가져오는 이벤트

- name: AWS Credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          aws-access-key-id: ${{ secrets.AWS_SECURITY_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECURITY_SECRET_KEY }}
          aws-region: ap-northeast-2
  • aws ec2에 접근하기위한 권한이 있는 aws credential 추가

 - name: executing docker-compose up on main server
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.MAIN_SERVER_HOST }}
          username: ${{ secrets.MAIN_SERVER_USERNAME }}
          key: ${{ secrets.MAIN_SERVER_PEM }}
          script: |
            cd ./docker
            docker-compose down
            docker rmi ${{ env.IMAGE_NAME }}:latest
            docker-compose up -d
  • server에 ssh접속 후 docker container 재실행

	- name: Remove Github Actions IP From Security Group
	       run: |
	          aws ec2 revoke-security-group-ingress --group-id ${{ secrets.AWS_MAIN_SG_ID }} --protocol tcp --port 22 --cidr ${{ steps.ip.outputs.ipv4 }}/32
  • git action ip를 보안그룹에서 삭제

  • 참고

GitHub Actions workflow를 수동으로 trigger하기(feat. inquirer.js) | 카카오엔터테인먼트 FE 기술블로그

[CICD] Docker + Github Action + Spring Boot 자동배포환경 만들기

[Github Actions/AWS] Github Actions에서 SSH로 EC2 인스턴스에 접근하기

profile
HyunKyu Lee
backend developer
이전 포스트

Cookie란 무엇인가? SameSite 이슈 해결

0개의 댓글