모든 노드 적용
-
도커 설치 및 상태 확인
[root@matser ~]# yum install epel-release -y && sleep 1 && \ curl -fsSL https://get.docker.com -o get-docker.sh && sleep 1 && \ sh ./get-docker.sh && sleep 1 && \ systemctl enable --now docker.service [root@master ~]# systemctl status docker ● docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; preset: disabled) Active: active (running) since Sat 2025-03-08 10:14:38 KST; 1min 23s ago TriggeredBy: ● docker.socket Docs: <https://docs.docker.com> Main PID: 21210 (dockerd) Tasks: 10 Memory: 26.5M CPU: 551ms CGroup: /system.slice/docker.service └─21210 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock 3월 08 10:14:38 master.example.com dockerd[21210]: time="2025-03-08T10:14:38.000331707+09:00" level=info msg="Starting up" 3월 08 10:14:38 master.example.com dockerd[21210]: time="2025-03-08T10:14:38.003032180+09:00" level=info msg="OTEL tracing is not configured, using no> 3월 08 10:14:38 master.example.com dockerd[21210]: time="2025-03-08T10:14:38.067889084+09:00" level=info msg="Loading containers: start." 3월 08 10:14:38 master.example.com dockerd[21210]: time="2025-03-08T10:14:38.780793459+09:00" level=info msg="Loading containers: done." 3월 08 10:14:38 master.example.com dockerd[21210]: time="2025-03-08T10:14:38.814133770+09:00" level=info msg="Docker daemon" commit=bbd0a17 containerd> 3월 08 10:14:38 master.example.com dockerd[21210]: time="2025-03-08T10:14:38.814486146+09:00" level=info msg="Initializing buildkit" 3월 08 10:14:38 master.example.com dockerd[21210]: time="2025-03-08T10:14:38.880762259+09:00" level=info msg="Completed buildkit initialization" 3월 08 10:14:38 master.example.com dockerd[21210]: time="2025-03-08T10:14:38.895951252+09:00" level=info msg="Daemon has completed initialization" 3월 08 10:14:38 master.example.com dockerd[21210]: time="2025-03-08T10:14:38.896231430+09:00" level=info msg="API listen on /run/docker.sock" 3월 08 10:14:38 master.example.com systemd[1]: Started Docker Application Container Engine. -
python 3.9 설치
[root@master ~]# yum -y install python39 && \ rm -f /usr/bin/python3 && \ ln -s /usr/bin/python3.9 /usr/bin/python3
마스터 노드 작업
-
pip3 upgrade
[root@master ~]# yum -y install python3-pip wget git vim sshpass && sleep 1 && python3 -m pip install --upgrade pip -
Kubespray 이용한 kubernetes 설치
kubespraykubernetes 설치를 도와주는 자동화 도구로,
ansible을 통해 구축하고자하는 설정값을 설정 후 실행하면 kuberentes cluster 구축을 자동으로 해주는 도구- kubernetes에서 제공하며, github을 통해 오픈소스로 공개되어있음
[root@master ~]# wget https://github.com/kubernetes-sigs/kubespray/archive/refs/tags/v2.23.2.tar.gz --2025-03-08 10:23:47-- <https://github.com/kubernetes-sigs/kubespray/archive/refs/tags/v2.23.2.tar.gz> Resolving github.com (github.com)... 20.200.245.247 Connecting to github.com (github.com)|20.200.245.247|:443... connected. HTTP request sent, awaiting response... 302 Found Location: <https://codeload.github.com/kubernetes-sigs/kubespray/tar.gz/refs/tags/v2.23.2> [following] --2025-03-08 10:23:47-- <https://codeload.github.com/kubernetes-sigs/kubespray/tar.gz/refs/tags/v2.23.2> Resolving codeload.github.com (codeload.github.com)... 20.200.245.246 Connecting to codeload.github.com (codeload.github.com)|20.200.245.246|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 2594075 (2.5M) [application/x-gzip] Saving to: `v2.23.2.tar.gz' v2.23.2.tar.gz 100%[=========================================================================>] 2.47M --.-KB/s in 0.1s 2025-03-08 10:23:48 (24.3 MB/s) - `v2.23.2.tar.gz' saved [2594075/2594075] [root@master ~]# tar xzf v2.23.2.tar.gz [root@master ~]# mv kubespray-2.23.2 kubespray -
kubespray 설치를 위한 필수 패키지 설치
[root@master ~/kubespray]# cd kubespray && cat requirements.txt && python3 -m pip install -r requirements.txt ansible==7.6.0 cryptography==41.0.1 jinja2==3.1.2 jmespath==1.0.1 MarkupSafe==2.1.3 netaddr==0.8.0 pbr==5.11.1 ruamel.yaml==0.17.31 ruamel.yaml.clib==0.2.7 ··· -
hosts yaml 파일 생성
- 참고 !!LINUX!!
- cp : 대상 복사
# 복사 대상이 파일 1개가 아닌 해당 디렉터리(하위에 있는 파일 모두 포함)를 복사할 경우, 옵션 `-r` cp -r [복사 대상] [복사 결과] # 파일 또는 디렉터리를 복사할 때 복사 대상의 소유자(계정), 그룹, 권한 등의 정보까지 복사할 경우, 옵션 `-p` cp -p [복사 대상] [복사 결과] # 복사 대상 파일이 이미 해당 위치에 있을 때 파일을 지우고 강제로 복사할 경우, 옵션 `-f` cp -f [복사 대상] [복사 결과] - declare : 변수 선언
:: declare를 명령창에 쓰면 해당 서버의 전역 환경변수로 선언된 변수 값들이 출력됨```bash # 각 이름을 배열 변수로 선언할 경우, 옵션 `-a` declare -a {변수명}=(배열) ``` - shell script ${arr[@]} : arr 배열의 모든 원소 출력
[root@master ~/kubespray]# cp -rfp inventory/sample inventory/mycluster declare -a IPS=(192.168.45.60 192.168.45.61 192.168.45.62 192.168.45.63) [root@master ~/kubespray]# CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]} DEBUG: Adding group all DEBUG: Adding group kube_control_plane DEBUG: Adding group kube_node DEBUG: Adding group etcd DEBUG: Adding group k8s_cluster DEBUG: Adding group calico_rr DEBUG: adding host node1 to group all DEBUG: adding host node2 to group all DEBUG: adding host node3 to group all DEBUG: adding host node4 to group all DEBUG: adding host node1 to group etcd DEBUG: adding host node2 to group etcd DEBUG: adding host node3 to group etcd DEBUG: adding host node1 to group kube_control_plane DEBUG: adding host node2 to group kube_control_plane DEBUG: adding host node1 to group kube_node DEBUG: adding host node2 to group kube_node DEBUG: adding host node3 to group kube_node DEBUG: adding host node4 to group kube_node
- cp : 대상 복사
- 참고 !!LINUX!!
-
hosts yaml 파일 수정
[root@master ~/kubespray]# cat << EOF > inventory/mycluster/hosts.yaml all: hosts: master: ansible_host: 192.168.45.60 ip: 192.168.45.60 access_ip: 192.168.45.60 node1: ansible_host: 192.168.45.61 ip: 192.168.45.61 access_ip: 192.168.45.61 node2: ansible_host: 192.168.45.62 ip: 192.168.45.62 access_ip: 192.168.45.62 node3: ansible_host: 192.168.45.63 ip: 192.168.45.63 access_ip: 192.168.45.63 children: kube_control_plane: hosts: master: kube_node: hosts: node1: node2: node3: etcd: hosts: master: k8s_cluster: children: kube_control_plane: kube_node: calico_rr: hosts: {} EOF -
container runtime 변경
[root@master ~/kubespray]# vi inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml ---229gg 226 ## Container runtime 227 ## docker for docker, crio for cri-o and containerd for containerd. 228 ## Default: containerd 229 container_manager: docker ＃ containerd ▶ docker -
ansible 플레이
[root@master ~/kubespray]# ansible-playbook \ -i inventory/mycluster/hosts.yaml \ --become --become-user=root \ cluster.yml -
kube 설치 확인
- control-plane
다른 노드들에게 명령을 내리고 관리하는 노드 - worker nodes
실제 컨테이너를 구동하는 노드
[root@master ~/kubespray]# kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.7", GitCommit:"07a61d861519c45ef5c89bc22dda289328f29343", GitTreeState:"clean", BuildDate:"2023-10-18T11:42:32Z", GoVersion:"go1.20.10", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.7", GitCommit:"07a61d861519c45ef5c89bc22dda289328f29343", GitTreeState:"clean", BuildDate:"2023-10-18T11:33:23Z", GoVersion:"go1.20.10", Compiler:"gc", Platform:"linux/amd64"}
[root@master ~/kubespray]# kubectl get node
NAME STATUS ROLES AGE VERSION
master Ready control-plane 59m v1.27.7
node1 Ready <none> 58m v1.27.7
node2 Ready <none> 58m v1.27.7
node3 Ready <none> 58m v1.27.7
[root@master ~/kubespray]# kubectl get node -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
master Ready control-plane 59m v1.27.7 192.168.45.60 <none> CentOS Stream 9 5.14.0-452.el9.x86_64 docker://28.0.1
node1 Ready <none> 59m v1.27.7 192.168.45.61 <none> CentOS Stream 9 5.14.0-452.el9.x86_64 docker://28.0.1
node2 Ready <none> 59m v1.27.7 192.168.45.62 <none> CentOS Stream 9 5.14.0-452.el9.x86_64 docker://28.0.1
node3 Ready <none> 59m v1.27.7 192.168.45.63 <none> CentOS Stream 9 5.14.0-452.el9.x86_64 docker://28.0.1설치 확인을 위한 간단한 실습
-
실습 파일 복제
[root@master ~/kubespray]# cd && git clone <https://github.com/ncs10322/kube.git> 'kube'에 복제합니다... remote: Enumerating objects: 284, done. remote: Counting objects: 100% (97/97), done. remote: Compressing objects: 100% (72/72), done. remote: Total 284 (delta 39), reused 57 (delta 21), pack-reused 187 (from 1) 오브젝트를 받는 중: 100% (284/284), 1.37 MiB | 19.69 MiB/s, 완료. 델타를 알아내는 중: 100% (92/92), 완료. [root@master ~]# ls anaconda-ks.cfg get-docker.sh kube kube-manifests kubespray v2.23.2.tar.gz 공개 다운로드 문서 바탕화면 비디오 사진 서식 음악 -
dockerhub 이미지 다운로드 제한 방지를 위해 dockerhub 계정 secret 생성
[root@master ~]# kubectl create secret docker-registry mysecret --docker-username={dockerhub ID} --docker-password={dockerhub PW} --docker-email={dockerhub EMAIL} secret/mysecret created -
생성한 secret 관련 serviceaccount 패치
[root@master ~]# kubectl patch -n default serviceaccount/default -p '{"imagePullSecrets":[{"name": "mysecret"}]}' serviceaccount/default patched
실습 :: deploy 생성 및 삭제
pod : container를 가진 작업이 구동되는 공간 단위
[root@master ~]# kubectl create deploy webserver --image=httpd --replicas=3
deployment.apps/webserver created
[root@master ~]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
webserver-75dcc5d576-8z7cg 1/1 Running 0 15s 10.233.75.1 node2 <none> <none>
webserver-75dcc5d576-b5tc9 1/1 Running 0 15s 10.233.102.130 node1 <none> <none>
webserver-75dcc5d576-mnnmp 1/1 Running 0 15s 10.233.71.2 node3 <none> <none>실습 :: deploy 삭제
[root@master ~]# kubectl delete deploy webserver
deployment.apps "webserver" deleted
[root@master ~]# kubectl get pod
No resources found in default namespace.