컨테이너 오케스트레이션을 위한 Kubernetes (22.05.19)

박민선·2022년 5월 20일

Service & DNS & Ingress

Service Type
1) nodeport 클러스터 외부
2) loadbalancer 클러스터 외부
3) cluster ip 클러스터 내부

Service - ClusterIP

myweb-svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: myweb-svc
spec:
  selector: # 파드 셀렉터
    app: web
  ports:
    - port: 80 # 서비스 포트
      targetPort: 8080 # 타켓(파드 포트)

port 클라이언트가 접속하는 포트
targetport 컨테이너 파드 접속포트

kubectl create -f .

kubectl get svc myweb-svc

kubectl describe svc myweb-svc

kubectl get endpoint myweb-svc

kubectl run nettool -it --image ghcr.io/c1t1d0s7/network-multitool

> curl x.x.x.x(서비스 리소스의 ClusterIP)
> host myweb-svc
> curl myweb-svc

curl 확인을 위해
apt update; apt install curl
curl x.x.x.x (10.233.28.39)

Session Affinity

세션 고정
default 값은 none (ClientIP 변경가능)
meyweb-svc-ses.yaml

apiVersion: v1
kind: Service
metadata:
  name: myweb-svc-ses
spec:
  type: ClusterIP
  sessionAffinity: ClientIP
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080

Named Port

myweb-rs-named.yaml

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: myweb-rs-named
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
      env: dev
  template:
    metadata:
      labels:
        app: web
        env: dev
    spec:
      containers:
        - name: myweb
          image: ghcr.io/c1t1d0s7/go-myweb
          ports:
            - containerPort: 8080
              protocol: TCP
              name: web8080                #콘테이너 네임을 적을 수 있음

myweb-svc-named.yaml

apiVersion: v1
kind: Service
metadata:
  name: myweb-svc-named
spec:
  type: ClusterIP
  selector:
    app: web
  ports:
    - port: 80
      targetPort: web8080

Multi Port

myweb-rs-multi.yaml

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: myweb-rs-multi
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
      env: dev
  template:
    metadata:
      labels:
        app: web
        env: dev
    spec:
      containers:
        - name: myweb
          image: ghcr.io/c1t1d0s7/go-myweb
          ports:
            - containerPort: 8080
              protocol: TCP
            - containerPort: 8443
              protocol: TCP

http 80
https 443

myweb-svc-multi.yaml

apiVersion: v1
kind: Service
metadata:
  name: myweb-svc-multi
spec:
  type: ClusterIP
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080
      name: http
    - port: 443
      targetPort: 8443
      name: https

Service Discovery

환경 변수를 이용한 SD

모든 파드는 실행 시 현재 시점의 서비스 목록을 환경 변수 제공

# env | grep MYWEB
MYWEB_SVC_PORT_80_TCP_PORT=80
MYWEB_SVC_PORT_80_TCP_PROTO=tcp
MYWEB_SVC_PORT_80_TCP=tcp://10.233.3.182:80
MYWEB_SVC_SERVICE_HOST=10.233.3.182
MYWEB_SVC_PORT=tcp://10.233.3.182:80
MYWEB_SVC_SERVICE_PORT=80
MYWEB_SVC_PORT_80_TCP_ADDR=10.233.3.182

DNS를 이용한 SD

kube-dns(coredns-X 파드)

Service 생성하면 해당 이름으로 FQDN을 DNS 서버에 등록

[서비스 이름].[네임스페이스].[오브젝트 타입].[도메인]

myweb-svc.default.svc.cluster.local

host myweb-svc
host myweb-svc.default
(nameSpace까지는 써주는거 권장)

마지막 점 (RootHint)
myweb-svc.default.svc.cluster.local (.) <-

nodelocal DNS

nodelocal DNS 캐시 사용
Pod --dns--> 169.254.25.10(node-cache): DNS Cache Server --> coredns SVC(kube-system NS) -> coredns POD

nodelocal DNS 캐시 사용 X
Pod --dns--> coredns SVC(kube-system NS) -> coredns POD

Service - NodePort

svc.spec.type

ClusterIP: 클러스터 내에서 사용하는 LB
NodePort: 클러스터 외부에서 접근하는 포인트
LoadBalancer: 클러스터 외부에서 접근하는 LB

system port 0~1023 (root권한)
registered port 1024~49151
dynamic/private port 49152~65535 (client사용)

NodePort의 범위: 30000-32767

myweb-svc-np.yaml

apiVersion: v1
kind: Service
metadata:
  name: myweb-svc-np
spec:
  type: NodePort
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080
      nodePort: 31313

nodePort = nodePort + ClusterIP
nodePort를 지정안하면 자동으로 지정

Service - LoadBalancer

LoadBalancer : L4 LB (MetalLB)
ingress : L7 LB (Nginx)

myweb-svc-lb.yaml

apiVersion: v1
kind: Service
metadata:
  name: myweb-svc-lb
spec:
  type: LoadBalancer
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080
      nodePort: 31313

Kubernetes는 외부에 LB를 생성할 수 없음
but, METALLB 를 사용하면 가능
(내부 pod을 이용하여)

Metallb - Addon

~/kubespray/inventory/mycluster/group_vars/k8s-cluster/addons.yml

...
139 metallb_enabled: true
140 metallb_speaker_enabled: true
141 metallb_ip_range:
142   - "192.168.100.240-192.168.100.249"
...
168 metallb_protocol: "layer2"
...

~/kubespray/inventory/mycluster/group_vars/k8s-cluster/k8s-cluster.yml

129 kube_proxy_strict_arp: true

ansible-playbook -i inventory/mycluster/inventory.ini cluster.yml -b

LoadBalancer = (외부)LB + NP + ClusterIP

MetalLb는 2가지 모드가 있음
1. Layer2 가 default값 - 소규모
2. BGP (L3 SW를 사용) - 대규모

(METALLB 참고링크)
https://metallb.universe.tf/

Service - ExternalName

클러스터 내부에서 클러스터 외부의 특정 서비스에 접속하기 위해 DNS CNAME을 설정

apiVersion: v1
kind: Service
metadata:
  name: weather-ext-svc
spec:
  type: ExternalName
  externalName: www.naver.com

kubectl replac -f weather-ext-svc.yaml   #[네임(주소)이 변경 되었을때]

curl -s 'wttr.in/Seoul'
curl -s 'wttr.in/Seoul?format=1'
curl -s 'wttr.in/Seoul?format=2'
(서울날씨가 뜸!)

Ingress

L7 LB = ALB

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myweb-ing
spec:
  rules:
    - host: '*.encore.xyz'
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: myweb-svc-np
                port:
                  number: 80

도메인 없이 테스트 하는 법

방법1

curl --resolve www.encore.xyz:80:192.168.100.100 http://www.encore.xyz

방법2
/etc/hosts

...
192.168.100.100 www.encore.xyz  #빈곳에 작성

curl http://www.encore.xyz

방법3

https://nip.io/
https://sslip.io/

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myweb-ing
spec:
  rules:
    - host: '*.nip.io'
    ...

kubectl replace -f myweb-ing.yaml

curl http://192-168-100-100.nip.io

인그레스 예제

hello:one 이미지
Dockerfile

FROM httpd
COPY index.html /usr/local/apache2/htdocs/index.html

index.html

<h1> Hello One </h1>

hello:two 이미지
Dockerfile

FROM httpd
COPY index.html /usr/local/apache2/htdocs/index.html

index.html

<h1> Hello Two </h1>

docker image build X/hello:one
docker image build X/hello:two

docker login

docker push X/hello:one
docker push X/hello:two

RS
one-rs.yaml

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: one-rs
spec:
  replicas: 3
  selector:
    matchLabels:
      app: hello-one
  template:
    metadata:
      labels:
        app: hello-one
    spec:
      containers:
        - name: hello-one
          image: c1t1d0s7/hello:one
          ports:
            - containerPort: 80
              protocol: TCP

two-rs.yaml

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: two-rs
spec:
  replicas: 3
  selector:
    matchLabels:
      app: hello-two
  template:
    metadata:
      labels:
        app: hello-two
    spec:
      containers:
        - name: hello-two
          image: c1t1d0s7/hello:two
          ports:
            - containerPort: 80
              protocol: TCP

one-svc-np.yaml

apiVersion: v1
kind: Service
metadata:
  name: one-svc-np
spec:
  type: NodePort
  selector:
    app: hello-one
  ports:
    - port: 80
      targetPort: 80

two-svc-np.yaml

apiVersion: v1
kind: Service
metadata:
  name: two-svc-np
spec:
  type: NodePort
  selector:
    app: hello-two
  ports:
    - port: 80
      targetPort: 80

hello-ing.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hello-ing
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: / # URL 재작성, /one -> /, /two -> /
spec:
  rules:
    - host: '*.nip.io'
      http:
        paths:
          - path: /one
            pathType: Prefix
            backend:
              service:
                name: one-svc-np
                port:
                  number: 80
          - path: /two
            pathType: Prefix
            backend:
              service:
                name: two-svc-np
                port:
                  number: 80

kubectl create -f .

Readiness Probe

파드의 헬스체크를 통해 서비스의 엔드포인트 리소스에 타겟 등록

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: myweb-rs
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
      env: dev
  template:
    metadata:
      labels:
        app: web
        env: dev
    spec:
      containers:
        - name: myweb
          image: ghcr.io/c1t1d0s7/go-myweb:alpine
          ports:
            - containerPort: 8080
              protocol: TCP
          readinessProbe:
            exec:
              command:
                - ls
                - /tmp/ready

apiVersion: v1
kind: Service
metadata:
  name: myweb-svc-lb
spec:
  type: LoadBalancer
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080

kubectl create -f .

watch -n1 -d kubectl get po,svc,ep

kubectl exec <POD> -- touch /tmp/ready

박민선

클라우드신생아

이전 포스트

컨테이너 오케스트레이션을 위한 Kubernetes (22.05.18)

다음 포스트