파드에서 실행중인 애플리케이션을 네트워크 서비스로 노출시키는 방법
쿠버네티스 내부에서만 포드들에 접근할 때 사용한다. 외부로 포드를 노출하지 않기 때문에 쿠버네티스 클러스터 내부에서만 사용하는 포드에 적합하다.
파드에 접근할 수 있는 포트를 클러스터의 모든 노드에 동일하게 개방한다. 따라서, 외부에서 포드에 접근할 수 있는 서비스 타입이다.
접근할 수 있는 포트는 랜덤하게 정해지지만, 특정 포트로 접근하도록 설정할 수 있다.
클라우드 플랫폼에서 제공하는 로드 밸런서를 동적으로 프로비저닝해 포드에 연결한다.
NodePort 타입과 마찬가지로 외부에서 포드에 접근할 수 있는 서비스 타입이다. 그렇지만 일반적으로 AWS, GCP 등과 같은 클라우드 플랫폼 환경에서만 사용할 수 있다.
apiVersion: apps/v1
kind: Deployment
metadata:
name: deploy-cndk
spec:
replicas: 3
selector:
matchLabels:
app: deploy-websrv
template:
metadata:
labels:
app: deploy-websrv
spec:
containers:
- name: cndk-websrv
image: gcr.io/google-samples/kubernetes-bootcamp:v1
ports:
- containerPort: 8080
# kubectl apply -f pod.yaml
deployment.apps/deploy-cndk created
# watch -d 'kubectl get pods,svc,ep -o wide'
Every 2.0s: kubectl get pods,svc,ep -o wide master: Sun Jul 18 03:47:37 2021
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/deploy-cndk-6cbc8c75db-fjd4v 1/1 Running 0 4m27s 172.16.189.102 worker2 <none> <none>
pod/deploy-cndk-6cbc8c75db-nssqs 1/1 Running 0 4m27s 172.16.182.38 worker3 <none> <none>
pod/deploy-cndk-6cbc8c75db-zs2lg 1/1 Running 0 4m28s 172.16.235.136 worker1 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 12m <none>
NAME ENDPOINTS AGE
endpoints/kubernetes 192.168.1.211:6443 12m
nodeName 에는 마스터 노드의 이름을 입력
apiVersion: v1
kind: Pod
metadata:
name: netshoot-pod
spec:
**nodeName: master**
containers:
- name: netshoot-pod
image: nicolaka/netshoot
command: ["tail"]
args: ["-f", "/dev/null"]
# kubectl apply -f podnet.yaml
pod/netshoot-pod created
# watch -d 'kubectl get pods,svc,ep -o wide'
Every 2.0s: kubectl get pods,svc,ep -o wide master: Sun Jul 18 03:48:42 2021
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/deploy-cndk-6cbc8c75db-fjd4v 1/1 Running 0 5m32s 172.16.189.102 worker2 <none> <none>
pod/deploy-cndk-6cbc8c75db-nssqs 1/1 Running 0 5m32s 172.16.182.38 worker3 <none> <none>
pod/deploy-cndk-6cbc8c75db-zs2lg 1/1 Running 0 5m33s 172.16.235.136 worker1 <none> <none>
**pod/netshoot-pod 1/1 Running 0 15s 172.16.219.70 master <none> <none>**
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 13m <none>
NAME ENDPOINTS AGE
endpoints/kubernetes 192.168.1.211:6443 13m
# kubectl exec -it netshoot-pod -- zsh
dP dP dP
88 88 88
88d888b. .d8888b. d8888P .d8888b. 88d888b. .d8888b. .d8888b. d8888P
88' `88 88ooood8 88 Y8ooooo. 88' `88 88' `88 88' `88 88
88 88 88. ... 88 88 88 88 88. .88 88. .88 88
dP dP `88888P' dP `88888P' dP dP `88888P' `88888P' dP
Welcome to Netshoot! (github.com/nicolaka/netshoot)
apiVersion: v1
kind: Service
metadata:
name: svc-clusterip
spec:
ports:
- name: svc-webport
**port: 9000
targetPort: 8080**
selector:
app: deploy-websrv
**type: ClusterIP**
# kubectl apply -f svc-clusterip.yaml
service/svc-clusterip created
# watch -d 'kubectl get pods,svc,ep -o wide'
Every 2.0s: kubectl get pods,svc,ep -o wide master: Sun Jul 18 03:57:19 2021
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/deploy-cndk-6cbc8c75db-fjd4v 1/1 Running 0 14m 172.16.189.102 worker2 <none> <none>
pod/deploy-cndk-6cbc8c75db-nssqs 1/1 Running 0 14m 172.16.182.38 worker3 <none> <none>
pod/deploy-cndk-6cbc8c75db-zs2lg 1/1 Running 0 14m 172.16.235.136 worker1 <none> <none>
pod/netshoot-pod 1/1 Running 0 8m52s 172.16.219.70 master <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 22m <none>
**service/svc-clusterip ClusterIP 10.104.187.17 <none> 9000/TCP 29s app=deploy-websrv**
NAME ENDPOINTS AGE
endpoints/kubernetes 192.168.1.211:6443 22m
**endpoints/svc-clusterip 172.16.182.38:8080,172.16.189.102:8080,172.16.235.136:8080 29s**
[root@master dkos-w05 (⎈ |kube:default)]# curl 10.104.187.17:9000
Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-nssqs | v=1
[root@master dkos-w05 (⎈ |kube:default)]# curl 10.104.187.17:9000
Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-fjd4v | v=1
[root@master dkos-w05 (⎈ |kube:default)]# curl 10.104.187.17:9000
Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-zs2lg | v=1
[root@master dkos-w05 (⎈ |kube:default)]# curl 10.104.187.17:9000
Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-zs2lg | v=1
[root@master dkos-w05 (⎈ |kube:default)]# curl 10.104.187.17:9000
Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-nssqs | v=1
[root@master dkos-w05 (⎈ |kube:default)]#
생성한 서비스(ClusterIP)를 통해 클러스터 내부에서 3개의 파드로 접속이 가능하다.
하지만, 클러스터 외부에서는 접속할 수 없다는 단점이 있다.
이를 극복하기 위해, NodePort 서비스 타입을 사용해본다.
# kubectl delete deploy,svc --all
deployment.apps "deploy-cndk" deleted
service "kubernetes" deleted
service "svc-clusterip" deleted
apiVersion: apps/v1
kind: Deployment
metadata:
name: deploy-nginx11
spec:
replicas: 3
selector:
matchLabels:
app: deploy-nginx11
template:
metadata:
labels:
app: deploy-nginx11
spec:
containers:
- name: deploy-nginx11
image: nginx:1.11
# kubectl apply -f nginx11.yaml
deployment.apps/deploy-nginx11 created
# watch -d 'kubectl get pods,svc,ep -o wide'
Every 2.0s: kubectl get pods,svc,ep -o wide master: Sun Jul 18 04:27:14 2021
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/deploy-nginx11-656d5495b7-9rndv 1/1 Running 0 94s 172.16.235.137 worker1 <none> <none>
pod/deploy-nginx11-656d5495b7-km865 1/1 Running 0 94s 172.16.189.103 worker2 <none> <none>
pod/deploy-nginx11-656d5495b7-xw8sk 1/1 Running 0 94s 172.16.182.39 worker3 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 10m <none>
NAME ENDPOINTS AGE
endpoints/kubernetes 192.168.1.211:6443 10m
위 실습에서 사용한 podnet.yaml 생성하거나, 삭제하지 않았다면 그대로 사용
# watch -d 'kubectl get pods,svc,ep -o wide'
Every 2.0s: kubectl get pods,svc,ep -o wide master: Sun Jul 18 04:27:53 2021
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/deploy-nginx11-656d5495b7-9rndv 1/1 Running 0 2m13s 172.16.235.137 worker1 <none> <none>
pod/deploy-nginx11-656d5495b7-km865 1/1 Running 0 2m13s 172.16.189.103 worker2 <none> <none>
pod/deploy-nginx11-656d5495b7-xw8sk 1/1 Running 0 2m13s 172.16.182.39 worker3 <none> <none>
**pod/netshoot-pod 1/1 Running 0 31m 172.16.219.70 master <none> <none>**
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 11m <none>
NAME ENDPOINTS AGE
endpoints/kubernetes 192.168.1.211:6443 11m
apiVersion: v1
kind: Service
metadata:
name: svc-nodeport
spec:
ports:
- name: svc-webport
port: 9000
targetPort: 80
**** ports:
- containerPort: 8080
selector:
app: deploy-nginx11
**type: NodePort**
# kubectl apply -f svc-nodeport.yaml
service/svc-nodeport created
# watch -d 'kubectl get pods,svc,ep -o wide'
Every 2.0s: kubectl get pods,svc,ep -o wide master: Sun Jul 18 04:30:09 2021
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/deploy-nginx11-656d5495b7-9rndv 1/1 Running 0 4m30s 172.16.235.137 worker1 <none> <none>
pod/deploy-nginx11-656d5495b7-km865 1/1 Running 0 4m30s 172.16.189.103 worker2 <none> <none>
pod/deploy-nginx11-656d5495b7-xw8sk 1/1 Running 0 4m30s 172.16.182.39 worker3 <none> <none>
pod/netshoot-pod 1/1 Running 0 41m 172.16.219.70 master <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 13m <none>
**service/svc-nodeport NodePort 10.97.24.204 <none> 9000:30000/TCP 23s app=deploy-nginx11**
NAME ENDPOINTS AGE
endpoints/kubernetes 192.168.1.211:6443 13m
**endpoints/svc-nodeport 172.16.182.39:80,172.16.189.103:80,172.16.235.137:80 22s**
# curl localhost:30000
# curl 192.168.1.211:30000
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
... 생략 ...
# curl 10.97.24.204:9000
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
... 생략 ...
# Worker Node 1
# curl 192.168.1.212:30000
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
... 생략 ...
# Worker Node 2
# curl 192.168.1.213:30000
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
... 생략 ...
# Worker Node 3
# curl 192.168.1.214:30000
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
... 생략 ...
<서비스IP내부포트(9000)> 을 이용해 접속이 가능하다.
<마스터노드:외부포트(30000)> 를 이용해 접속이 가능하다.
뿐만 아니라, <워커노드:외부포트(30000)> 를 이용해 접속이 가능하다.
# kubectl delete deploy,svc --all
deployment.apps "deploy-nginx11" deleted
service "kubernetes" deleted
service "svc-nodeport" deleted
외부 클라이언트 접속 시, kube-proxy에 바로 접근하는것이 아니라, 그 앞단에 위치하여 Load Balancer에 접근한다.
Cloud 사업자의 경우, 각각 구현한 LoadBalancer가 존재하며, 온프레미스의 경우 MetalLB와 같은 LoadBalancer를 사용하여 구현한다.
현 실습 환경이 온프레미스이므로, MetalLB를 사용해 실습한다.
# kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.10.2/manifests/namespace.yaml
namespace/metallb-system created
# kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.10.2/manifests/metallb.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/controller created
podsecuritypolicy.policy/speaker created
serviceaccount/controller created
serviceaccount/speaker created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
role.rbac.authorization.k8s.io/config-watcher created
role.rbac.authorization.k8s.io/pod-lister created
role.rbac.authorization.k8s.io/controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/config-watcher created
rolebinding.rbac.authorization.k8s.io/pod-lister created
rolebinding.rbac.authorization.k8s.io/controller created
daemonset.apps/speaker created
deployment.apps/controller created
# kubectl get namespaces
NAME STATUS AGE
default Active 32d
ingress-nginx Active 6d16h
kube-node-lease Active 32d
kube-public Active 32d
kube-system Active 32d
**metallb-system Active 44s**
# kubectl get all -n metallb-system
NAME READY STATUS RESTARTS AGE
pod/controller-6b78bff7d9-9nl82 1/1 Running 0 46s
pod/speaker-68jzn 1/1 Running 0 46s
pod/speaker-6xcvb 1/1 Running 0 46s
pod/speaker-l9b8w 1/1 Running 0 46s
pod/speaker-rpctf 1/1 Running 0 46s
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/speaker 4 4 4 4 4 kubernetes.io/os=linux 46s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/controller 1/1 1 1 46s
NAME DESIRED CURRENT READY AGE
replicaset.apps/controller-6b78bff7d9 1 1 1 46s
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: default
protocol: layer2
addresses:
- 192.168.1.221-192.168.1.229
# kubectl apply -f metallb-config.yaml
configmap/config created
위 실습에서 사용한 pod.yaml
템플릿을 그대로 사용
# kubectl apply -f pod.yaml
deployment.apps/deploy-cndk created
# watch -d 'kubectl get pods,svc,ep -o wide'
Every 2.0s: kubectl get pods,svc,ep -o wide master: Sun Jul 18 05:18:51 2021
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
**pod/deploy-cndk-6cbc8c75db-cscvj 1/1 Running 0 8s 172.16.235.139 worker1 <none> <none>
pod/deploy-cndk-6cbc8c75db-h5htx 1/1 Running 0 8s 172.16.182.40 worker3 <none> <none>
pod/deploy-cndk-6cbc8c75db-jsbp7 1/1 Running 0 8s 172.16.189.104 worker2 <none> <none>**
pod/netshoot-pod 1/1 Running 0 90m 172.16.219.70 master <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 30m <none>
NAME ENDPOINTS AGE
endpoints/kubernetes 192.168.1.211:6443 30m
apiVersion: v1
kind: Service
metadata:
name: svc-mtlb-1
spec:
ports:
- name: svc-mtlb-webport
port: 9000
targetPort: 8080
nodePort: 30001
selector:
app: deploy-websrv
type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
name: svc-mtlb-2
spec:
ports:
- name: svc-mtlb-webport
port: 9000
targetPort: 8080
nodePort: 30002
selector:
app: deploy-websrv
type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
name: svc-mtlb-3
spec:
ports:
- name: svc-mtlb-webport
port: 9000
targetPort: 8080
nodePort: 30003
selector:
app: deploy-websrv
type: LoadBalancer
# kubectl apply -f mtlb.yaml
service/svc-mtlb-1 created
service/svc-mtlb-2 created
service/svc-mtlb-3 created
# watch -d 'kubectl get pods,svc,ep -o wide'
Every 2.0s: kubectl get pods,svc,ep -o wide master: Sun Jul 18 05:32:34 2021
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/deploy-cndk-6cbc8c75db-h5htx 1/1 Running 0 13m 172.16.182.40 worker3 <none> <none>
pod/deploy-cndk-6cbc8c75db-jsbp7 1/1 Running 0 13m 172.16.189.104 worker2 <none> <none>
pod/deploy-cndk-6cbc8c75db-pbgzz 1/1 Running 0 3m26s 172.16.182.41 worker3 <none> <none>
pod/netshoot-pod 1/1 Running 0 104m 172.16.219.70 master <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 44m <none>
**service/svc-mtlb-1 LoadBalancer 10.99.211.249 192.168.1.221 9000:30001/TCP 8s app=deploy-websrv
service/svc-mtlb-2 LoadBalancer 10.104.210.238 192.168.1.222 9000:30002/TCP 8s app=deploy-websrv
service/svc-mtlb-3 LoadBalancer 10.97.123.170 192.168.1.223 9000:30003/TCP 8s app=deploy-websrv**
NAME ENDPOINTS AGE
endpoints/kubernetes 192.168.1.211:6443 44m
**endpoints/svc-mtlb-1 172.16.182.40:8080,172.16.182.41:8080,172.16.189.104:8080 8s
endpoints/svc-mtlb-2 172.16.182.40:8080,172.16.182.41:8080,172.16.189.104:8080 8s
endpoints/svc-mtlb-3 172.16.182.40:8080,172.16.182.41:8080,172.16.189.104:8080 8s**
# for i in {1..100}; do curl -s 192.168.1.221:9000 ; done | sort | uniq -c | sort -nr
42 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-jsbp7 | v=1
34 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-h5htx | v=1
24 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-pbgzz | v=1
# for i in {1..100}; do curl -s 192.168.1.222:9000 ; done | sort | uniq -c | sort -nr
36 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-h5htx | v=1
35 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-pbgzz | v=1
29 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-jsbp7 | v=1
# for i in {1..100}; do curl -s 192.168.1.223:9000 ; done | sort | uniq -c | sort -nr
42 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-jsbp7 | v=1
32 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-pbgzz | v=1
26 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-h5htx | v=1
# for i in {1..100}; do curl -s 192.168.1.211:30001 ; done | sort | uniq -c | sort -nr
38 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-h5htx | v=1
34 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-pbgzz | v=1
28 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-jsbp7 | v=1
# for i in {1..100}; do curl -s 192.168.1.211:30002 ; done | sort | uniq -c | sort -nr
38 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-h5htx | v=1
33 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-jsbp7 | v=1
29 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-pbgzz | v=1
# for i in {1..100}; do curl -s 192.168.1.211:30003 ; done | sort | uniq -c | sort -nr
35 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-h5htx | v=1
34 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-jsbp7 | v=1
31 Hello Kubernetes bootcamp! | Running on: deploy-cndk-6cbc8c75db-pbgzz | v=1
이번에는 NodePort와 다르게
metallb-config.yaml
에서 정의한 주소 범위로 서비스의 External-IP 가 생성되었다.