이번 시간에는 워크로드 생성에 필요한 사용자 정의 이미지를 생성해 봅시다.
GCP 를 기준으로 합니다.
이미지 빌더를 활용하여 GCP 워크로드의 노드의 기본이미지가 될 사용자 정의 이미지를 생성합니다.
이미지 빌더는 Mac, Linux 등에 gcloud cli 로그인 가능한 대부분의 환경에 세팅 가능합니다.
(아직 윈도우는 지원되지 않는 것 같습니다...)
이 글은 Ubuntu Linux 22.04 버전을 기준으로 합니다.
GCP 이미지 빌더: https://image-builder.sigs.k8s.io/capi/providers/gcp
GCP Cluster-Api: https://github.com/kubernetes-sigs/cluster-api-provider-gcp
https://cluster-api.sigs.k8s.io/user/quick-start.html
Prerequisites
https://cloud.google.com/sdk/docs/install-sdk?hl=ko#deb
https://cloud.google.com/iam/docs/service-accounts-create?hl=ko#creating
https://github.com/kubernetes-sigs/cluster-api-provider-gcp/blob/main/docs/book/src/topics/prerequisites.md
Install
mkdir -pv ~/_setup && cd ~/_setup
cat <<EOT > gcloud_install.sh
# install gcloud
sudo apt-get -y update && sudo apt-get -y install unzip jq make python3 apt-transport-https ca-certificates gnupg curl sudo unzip jq
sudo apt-get -y update && sudo apt-get -y install google-cloud-cli
gcloud init
# login gcloud
gcloud auth application-default login
# get application_default_credentials.json
# 아래 경로에 비밀 정보가 담긴 json 파일이 생성됩니다.
cat $HOME/.config/gcloud/application_default_credentials.json
echo '$HOME/.config/gcloud/application_default_credentials.json'
EOT
chmod +x ~/_setup/gcloud_install.sh && ~/_setup/gcloud_install.sh
kubectl --namespace=default get secret managed-test-user-kubeconfig \
-o jsonpath={.data.value} | base64 --decode \
> managed-test.kubeconfig
Set Env
vi ~/.zshrc
export GCP_B64ENCODED_CREDENTIALS=$( cat $HOME/.config/gcloud/application_default_credentials.json | base64 | tr -d '\n' )
export GCP_REGION="us-central1" # TODO 수정 (zone이 아니고 region이라 xx-xxx# 형식)
export GCP_PROJECT="vigilant-router-398710" # TODO 수정
export KUBERNETES_VERSION=1.27.6
export GCP_CONTROL_PLANE_MACHINE_TYPE=n1-standard-2
export GCP_NODE_MACHINE_TYPE=n1-standard-2
export GCP_NETWORK_NAME=default
export CLUSTER_NAME="capi-gcp"
export GCP_PROJECT_ID="vigilant-router-398710" # TODO 수정
export GOOGLE_APPLICATION_CREDENTIALS=$HOME/.config/gcloud/application_default_credentials.json
# export IMAGE_ID="projects/${GCP_PROJECT_ID}/global/images/<image-name>" # TODO 수정 < 이미지 빌드후
source ~/.zshrcCreate Network
# Create routers
gcloud compute routers create "${CLUSTER_NAME}-router" --project="${GCP_PROJECT}" --region="${GCP_REGION}" --network="default"
# Create NAT
gcloud compute routers nats create "${CLUSTER_NAME}-nat" --project="${GCP_PROJECT}" --router-region="${GCP_REGION}" --router="${CLUSTER_NAME}-router" --nat-all-subnet-ip-ranges --auto-allocate-nat-external-ipsSet Image Builder
# Clone the image builder repository if you haven't already.
git clone https://github.com/kubernetes-sigs/image-builder.git $HOME/image-builder
# Change directory to images/capi within the image builder repository
cd $HOME/image-builder/images/capi
# Add bin path
echo 'export PATH="$HOME/image-builder/images/capi/.local/bin:$HOME/.local/bin:$PATH"' >> $HOME/.zshrc
source $HOME/.zshrc
# Check Usable Images
gcloud compute images list --project ${GCP_PROJECT_ID}
# ubuntu-2204-jammy-v20230919
# ubuntu-os-cloud
# ubuntu-2204-lts
# 이미지 스팩 설정
vi /home/ubuntu/image-builder/images/capi/packer/gce/ubuntu-2204.json
# 기본값으로 두셔도 되고 직접 수정해서도 됩니다.
# 저의 경우는 저렴한 zone으로 수정했습니다.
{
"build_name": "ubuntu-2204",
"distribution": "ubuntu",
"distribution_release": "jammy",
"distribution_version": "2204",
"source_image_family": "ubuntu-2204-lts",
"ssh_username": "ubuntu",
"zone": "us-central1-c"
}Build Image
Build
# Run the Make target to generate GCE images.
make build-gce-ubuntu-2204 # 이렇게 하면 쿠버네티스 1.26.7 버전이 설치됩니다.
# 커스텀 Variables를 작성하여 파커 변수로 제공하면 다른 버전을 설치할 수 있습니다.
# https://image-builder.sigs.k8s.io/capi/capi#customization
# v1.27.6으로 바꿔보겠습니다.
cat << EOT > k8s-1-27-6.json
{
"kubernetes_deb_version": "1.27.6-1.1",
"kubernetes_rpm_version": "1.27.6",
"kubernetes_semver": "v1.27.6",
"kubernetes_series": "v1.27"
}
EOT
# 이미지 빌드 (약 15분 소요)
PACKER_VAR_FILES=$HOME/image-builder/images/capi/k8s-v1-27-6.json \
make build-gce-ubuntu-2204
...
==> Wait completed after 15 minutes 25 seconds
==> Builds finished. The artifacts of successful builds are:
--> googlecompute.ubuntu-2204: A disk image was created: cluster-api-ubuntu-2204-v1-27-6-1696578548
# Check that you can see the published images.
gcloud compute images list --project ${GCP_PROJECT_ID} --no-standard-images
Check Image
➜ gcloud compute images list --project ${GCP_PROJECT_ID} --no-standard-images --filter="family:capi-ubuntu-2004-k8s"
NAME PROJECT FAMILY DEPRECATED STATUS
cluster-api-ubuntu-2004-v1-26-7-1693890685 endless-gizmo-393809 capi-ubuntu-2004-k8s-v1-26 READY
export IMAGE_ID="projects/${GCP_PROJECT_ID}/global/images/cluster-api-ubuntu-2204-k8s-v1-26-7-1696536193"
googlecompute.ubuntu-2204-k8s: A disk image was created: cluster-api-ubuntu-2204-k8s-v1-26-7-1696536193export Image ID
# vi ~/.zshrc
# 수정 전
export IMAGE_ID="projects/${GCP_PROJECT_ID}/global/images/<image-name>"
# 수정 후
export IMAGE_ID="projects/${GCP_PROJECT_ID}/global/images/cluster-api-ubuntu-2204-v1-27-6-1696578548"워크로드 클러스터 생성
# https://cluster-api.sigs.k8s.io/user/quick-start.html
clusterctl generate cluster capz \
--infrastructure gcp \
--kubernetes-version v1.27.6 \
--control-plane-machine-count=1 \
--worker-machine-count=2 \
> capz.yaml
k apply -f gcp.yaml
# 생성 상태 확인
➜ k get cluster
NAME PHASE AGE VERSION
gcp Provisioned 7m6s
➜ clusterctl describe --show-conditions all cluster gcpClean-up
Delete Resources
유료회원으로 전환하지 않는 이상 과금이 되지는 않지만 주어진 크래딧을 최대한 활용하기 위해서 사용하지 않는 자원은 삭제해야 합니다.
Delete image
gcloud compute images list --project ${GCP_PROJECT_ID} --no-standard-images
cluster-api-ubuntu-2204-k8s-1-27-6-v1-26-7-1696576404
gcloud compute images delete cluster-api-ubuntu-2204-k8s-1-27-6-v1-26-7-1696576404 --project=${GCP_PROJECT_ID}Delete the NAT gateway
gcloud compute routers nats delete "${CLUSTER_NAME}-nat" --project="${GCP_PROJECT}" \
--router-region="${GCP_REGION}" --router="${CLUSTER_NAME}-router" --quiet || trueDelete the router
gcloud compute routers delete "${CLUSTER_NAME}-router" --project="${GCP_PROJECT}" \
--region="${GCP_REGION}" --quiet || true
Think about a better architecture