kubernetes 1.29 version install kubeflow, airflow 호환되는 버전 체크

목표
1. Kubernetes 1.29 version으로 설치 후 Kubeflow, airflow 호환되는 버전 체크
환경
1. Ubuntu 20.04.1
2. K8s 1.29
3. Airflow 2.10.0
4. Kubeflow 1.9.0

K8s 1.27 , Airflow 2.10.0, Kubeflow 1.8.0

1. K8S 1.29 version 설치 (kubeadm)
2. Docker 설치
   $ sudo apt-get install -y \
   ca-certificates \
   curl \
   gnupg \
   lsb-release

$curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

$ echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

$ sudo apt-get update -y

$ sudo apt-get install -y docker-ce docker-ce-cli containerd.ioInstall

2. Docker daemon 설정 변경
   $ cat << EOF | sudo tee /etc/docker/daemon.json
   {
   "exec-opts": ["native.cgroupdriver=systemd"],
   "log-driver": "json-file",
   "log-opts": {
   "max-size": "100m"
   },
   "storage-driver": "overlay2"
   }
   EOF

$ sudo systemctl enable docker
$ sudo systemctl daemon-reload
$ sudo systemctl restart docker

$ sudo docker info | grep -i cgroup

3. ubuntu swap off 설정
   $ sudo swapoff -a; sudo sed -i '/swap/d' /etc/fstab

4. ubuntu ufw 방화벽 off
   $ sudo systemctl disable --now ufw

5. containerd 설정 변경
   $ cat << EOF | sudo tee -a /etc/modules-load.d/containerd.conf
   overlay
   br_netfilter
   EOF

$ sudo modprobe overlay
$ sudo modprobe br_netfilter

$ sudo apt update -y
$ sudo apt install -y containerd apt-transport-https
$ sudo mkdir /etc/containerd
$ sudo containerd config default > /etc/containerd/config.toml
$ sudo systemctl restart containerd
$ sudo systemctl enable containerd

6. kubernetes 네트워크 포워딩 설정 변경
   $ cat << EOF | sudo tee -a /etc/sysctl.d/kubernetes.conf
   net.bridge.bridge-nf-call-ip6tables = 1
   net.bridge.bridge-nf-call-iptables = 1
   net.ipv4.ip_forward = 1
   EOF

$ sudo sysctl –system

7. Kubernetes 설치
   $ curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
   echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

$ apt-get update
$ apt-get install -y kubelet kubeadm kubectl kubernetes-cni
$ apt-mark hold kubelet kubeadm kubectl kubernetes-cni
$ systemctl restart kubelet
$ systemctl status kubelet
$ sudo kubeadm init --pod-network-cidr=192.168.0.0/16

$ mkdir -p $HOME/.kube$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config$ sudo chown $(id -u):$(id -g) $HOME/.kube/config$ export KUBECONFIG=/etc/kubernetes/admin.conf

8. Calico CNI 설치
   $ kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.28.1/manifests/tigera-operator.yaml
   $ kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.28.1/manifests/custom-resources.yaml
   $ watch kubectl get pods -n calico-system
   $ kubectl taint nodes --all node-role.kubernetes.io/control-plane-

9. Local-path-provisioner 설치
   $ kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.28/deploy/local-path-storage.yaml
   $ kubectl patch storageclass local-path -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
   $ kubectl get sc

10. K9S 설치
    $ curl -sL https://github.com/derailed/k9s/releases/download/v0.26.3/k9s_Linux_x86_64.tar.gz | tar xfz - -C /usr/local/bin k9s
    

2. Airflow 설치 (helm) (2.10.0 version)

3. Helm install
   $ curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
   $ chmod 700 get_helm.sh
   $ ./get_helm.sh

4. Helm airflow repo를 추가
   $ helm repo add apache-airflow https://airflow.apache.org

5. Value file 저장
   $ helm show values apache-airflow/airflow > value.yaml

6. Value file 수정
   $ vi value.yaml
   

5. Airflow install
   $ helm upgrade --install airflow apache-airflow/airflow --namespace airflow --create-namespace -f value.yaml
   

후속단계
1. Svc 를 node port로 변환
$ k edit svc airflow-webserver -n airflow

2. Ui 접속 및 확인
   

3. Git sync 활성화
   레포지토리 생성
   

Ssh key 생성
$ ssh-keygen -t rsa -b 4096 -C your_email@example.com
$ pbcopy < airflow_ssh_key.pub

Deploy keys 설정
repositry setting 페이지로 넘어가면, 왼쪽 사이드 바에 Deploy keys 항목이 있다.
복사한 key를 붙여넣고 Allow write access를 허용해준 후 키를 등록한다.

K8s Secret 생성
$ kubectl create secret generic airflow-git-ssh-secret \
--from-file=gitSshKey=/path/to/.ssh/airflow_ssh_key \
--from-file=id_ed25519.pub=/path/to/.ssh/airflow_ssh_key.pub \
-n airflow
$ kubectl get secret -n airflow

Value file 수정

• enabled: 해당 기능을 사용할 것인지에 대한 여부
• repo: sync에 사용할 github repository 주소
• branch: repository의 어떤 branch를 sync 할 것인지
• depth: root 디렉토리의 depth. 디렉토리 자체를 dags 디렉토리로 사용할 것이므로 1로 놔둔다.
• subPath: 추가로 사용할 경로 path. 참조한 자료에서는 테스트 코드를 subPath에 추가해서 사용한다.
• sshKeySecret: 앞서 작성한 k8s secret의 이름

airflow upgrade
$ helm upgrade --install airflow apache-airflow/airflow -n airflow -f value.yaml

3. Kubeflow 설치 (1.9.0 version)

4. kustomize install
   $ curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash
   $ sudo mv kustomize /usr/local/bin

5. Linux kernel subsystem changes to support many pods
   $ sudo sysctl fs.inotify.max_user_instances=2280
   $ sudo sysctl fs.inotify.max_user_watches=1255360

6. Kubeflow 1.9.0 version install
   $ git clone https://github.com/kubeflow/manifests.git
   $ cd manifests/
   $ while ! kustomize build example | kubectl apply -f -; do echo "Retrying to apply resources"; sleep 20; done

7. Ui 접속 확인
   $ kubectl port-forward svc/istio-ingressgateway -n istio-system 8080:80