본 글은 gasida님의 AEWS 강의 및 AWS Docs, 연관 Blog 등을 참고하여 작성하였습니다.
CNI
- Kubernetes에서 CNI(Container Network Interface)는 Pod 간 통신, 네트워크 정책, IP 할당을 담당하는 핵심 구성 요소
- CNI가 없거나, 어떤 CNI를 선택하느냐에 따라 네트워크 구조, 성능, 운영 방식이 크게 달라짐
- 호스트 네트워크를 그대로 사용하는 등의 경우를 제외하면 CNI는 서비스를 위한 핵심요소
Calilco CNI
- 대표적인 Overlay / BGP 기반 CNI
- 네트워크 + NetworkPolicy를 함께 제공
AWS VPC CNI
- Pod에 VPC 내의 IP를 할당해주는 플러그인, Pod에 VPC IP를 직접 할당
- AWS 네트워크를 그대로 사용
Calico VS VPC CNI
- 파드간 통신 시 일반적으로 K8S CNI는 오버레이(VXLAN, IP-IP 등) 통신
- AWS VPC CNI는 동일 대역으로 직접 통신
EKS Networking 실습
1. 테라폼 환경 생성
# 테라폼 실행
terraform init
terraform plan
nohup sh -c "terraform apply -auto-approve" > create.log 2>&1 &
tail -f create.log
# 자격증명 설정
terraform output -raw configure_kubectl
aws eks --region ap-northeast-2 update-kubeconfig --name myeks
aws eks --region ap-northeast-2 update-kubeconfig --name myeks
kubectl config rename-context $(cat ~/.kube/config | grep current-context | awk '{print $2}') myeks
2. 기본 네트워크 정보 확인
네트워크 기본 정보 확인
# 파드 상세 정보 확인
kubectl get daemonset aws-node --namespace kube-system -owide
kubectl describe daemonset aws-node --namespace kube-system
# kube-proxy config 확인 : 모드 iptables 사용
kubectl describe cm -n kube-system kube-proxy-config
...
mode: "iptables"
...
kubectl describe cm -n kube-system kube-proxy-config | grep iptables: -A5
iptables:
masqueradeAll: false
masqueradeBit: 14
minSyncPeriod: 0s
syncPeriod: 30s
# aws-node 데몬셋 env 확인
kubectl get ds aws-node -n kube-system -o json | jq '.spec.template.spec.containers[0].env'
# 노드 IP 확인
aws ec2 describe-instances --query "Reservations[*].Instances[*].{PublicIPAdd:PublicIpAddress,PrivateIPAdd:PrivateIpAddress,InstanceName:Tags[?Key=='Name']|[0].Value,Status:State.Name}" --filters Name=instance-state-name,Values=running --output table
# 파드 IP 확인
kubectl get pod -n kube-system -o=custom-columns=NAME:.metadata.name,IP:.status.podIP,STATUS:.status.phase
# 파드 이름 확인
kubectl get pod -A -o name
# 파드 갯수 확인
kubectl get pod -A -o name | wc -l
노드내 IP 주소 확인
보조 IPv4 주소를 coredns 파드가 사용하는지 확인
# coredns 파드 IP 정보 확인
kubectl get pod -n kube-system -l k8s-app=kube-dns -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-d487b6fcb-7x24g 1/1 Running 0 13m 192.168.1.63 ip-192-168-1-36.ap-northeast-2.compute.internal <none> <none>
coredns-d487b6fcb-vb248 1/1 Running 0 13m 192.168.3.133 ip-192-168-3-208.ap-northeast-2.compute.internal <none> <none>
Network-Multitool 디플로이먼트 생성
# [터미널1~3] 노드 모니터링
ssh ec2-user@$N1
watch -d "ip link | egrep 'ens|eni' ;echo;echo "[ROUTE TABLE]"; route -n | grep eni"
ssh ec2-user@$N2
watch -d "ip link | egrep 'ens|eni' ;echo;echo "[ROUTE TABLE]"; route -n | grep eni"
ssh ec2-user@$N3
watch -d "ip link | egrep 'ens|eni' ;echo;echo "[ROUTE TABLE]"; route -n | grep eni"
# Network-Multitool 디플로이먼트 생성
cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: netshoot-pod
spec:
replicas: 3
selector:
matchLabels:
app: netshoot-pod
template:
metadata:
labels:
app: netshoot-pod
spec:
containers:
- name: netshoot-pod
image: praqma/network-multitool
ports:
- containerPort: 80
- containerPort: 443
env:
- name: HTTP_PORT
value: "80"
- name: HTTPS_PORT
value: "443"
terminationGracePeriodSeconds: 0
EOF
# 파드 이름 변수 지정
PODNAME1=$(kubectl get pod -l app=netshoot-pod -o jsonpath='{.items[0].metadata.name}')
PODNAME2=$(kubectl get pod -l app=netshoot-pod -o jsonpath='{.items[1].metadata.name}')
PODNAME3=$(kubectl get pod -l app=netshoot-pod -o jsonpath='{.items[2].metadata.name}')
echo $PODNAME1 $PODNAME2 $PODNAME3
# 파드 확인
kubectl get pod -o wide
kubectl get pod -o=custom-columns=NAME:.metadata.name,IP:.status.podIP
# 노드에 라우팅 정보 확인
for i in $N1 $N2 $N3; do echo ">> node $i <<"; ssh ec2-user@$i sudo ip -c route; echo; done
# 노드 네트워크 인터페이스 정보 확인
ssh ec2-user@$N3
----------------
ip -br -c addr show
ip -c link
ip -c addr
ip route # 혹은 route -n
# 네임스페이스 정보 출력 -t net(네트워크 타입)
sudo lsns -t net
exit
----------------
# 테스트용 파드 접속(exec) 후 Shell 실행
kubectl exec -it $PODNAME1 -- bash
# 아래부터는 pod-1 Shell 에서 실행 : 네트워크 정보 확인
----------------------------
ip -c addr
ip -c route
route -n
ping -c 1 <pod-2 IP>
ps
cat /etc/resolv.conf
exit
----------------------------
# 파드2 Shell 실행
kubectl exec -it $PODNAME2 -- ip -c addr
# 파드3 Shell 실행
kubectl exec -it $PODNAME3 -- ip -br -c addr
3. 노드 간 파드 통신
- AWS VPC CNI 경우 별도의 오버레이(Overlay) 통신 기술 없이, VPC Native 하게 파드간 직접 통신이 가능하다
# 파드 IP 변수 지정
PODIP1=$(kubectl get pod -l app=netshoot-pod -o jsonpath='{.items[0].status.podIP}')
PODIP2=$(kubectl get pod -l app=netshoot-pod -o jsonpath='{.items[1].status.podIP}')
PODIP3=$(kubectl get pod -l app=netshoot-pod -o jsonpath='{.items[2].status.podIP}')
echo $PODIP1 $PODIP2 $PODIP3
# 파드1 Shell 에서 파드2로 ping 테스트
kubectl exec -it $PODNAME1 -- ping -c 2 $PODIP2
kubectl exec -it $PODNAME1 -- curl -s http://$PODIP2
kubectl exec -it $PODNAME1 -- curl -sk https://$PODIP2
# 파드2 Shell 에서 파드3로 ping 테스트
kubectl exec -it $PODNAME2 -- ping -c 2 $PODIP3
# 파드3 Shell 에서 파드1로 ping 테스트
kubectl exec -it $PODNAME3 -- ping -c 2 $PODIP1
# 워커 노드 EC2 : TCPDUMP 확인
## For Pod to external (outside VPC) traffic, we will program iptables to SNAT using Primary IP address on the Primary ENI.
sudo tcpdump -i any -nn icmp
sudo tcpdump -i ens5 -nn icmp
sudo tcpdump -i ens6 -nn icmp
sudo tcpdump -i eniYYYYYYYY -nn icmp
# routing policy database management 확인
ip rule
# routing table management 확인
ip route show table local
ip route show table main
ip route show table 2
4. 파드에서 외부 통신
- 파드에서 외부 통신 흐름 : iptable 에 SNAT 을 통하여 노드의 eth0(ens5) IP로 변경되어서 외부와 통신됨
# pod-1 Shell 에서 외부로 ping
kubectl exec -it $PODNAME1 -- ping -c 1 www.google.com
kubectl exec -it $PODNAME1 -- ping -i 0.1 www.google.com
kubectl exec -it $PODNAME1 -- ping -i 0.1 8.8.8.8
# 워커 노드 EC2 : TCPDUMP 확인
sudo tcpdump -i any -nn icmp
sudo tcpdump -i ens5 -nn icmp
# 퍼블릭IP 확인
for i in $N1 $N2 $N3; do echo ">> node $i <<"; ssh ec2-user@$i curl -s ipinfo.io/ip; echo; echo; done
# 작업용 EC2 : pod-1 Shell 에서 외부 접속 확인 - 공인IP는 어떤 주소인가?
## The right way to check the weather - 링크
for i in $PODNAME1 $PODNAME2 $PODNAME3; do echo ">> Pod : $i <<"; kubectl exec -it $i -- curl -s ipinfo.io/ip; echo; echo; done
kubectl exec -it $PODNAME1 -- curl -s wttr.in/seoul
kubectl exec -it $PODNAME1 -- curl -s wttr.in/seoul?format=3
kubectl exec -it $PODNAME1 -- curl -s wttr.in/Moon
kubectl exec -it $PODNAME1 -- curl -s wttr.in/:help
# 워커 노드 EC2
## 출력된 결과를 보고 어떻게 빠져나가는지 고민해보자!
ip rule
ip route show table main
sudo iptables -L -n -v -t nat
sudo iptables -t nat -S
# 파드가 외부와 통신시에는 아래 처럼 'AWS-SNAT-CHAIN-0' 룰(rule)에 의해서 SNAT 되어서 외부와 통신!
# 참고로 뒤 IP는 eth0(ENI 첫번째)의 IP 주소이다
# --random-fully 동작 - 링크1 링크2
sudo iptables -t nat -S | grep 'A AWS-SNAT-CHAIN'
-A AWS-SNAT-CHAIN-0 ! -d 192.168.0.0/16 -m comment --comment "AWS SNAT CHAIN" -j RETURN
-A AWS-SNAT-CHAIN-0 ! -o vlan+ -m comment --comment "AWS, SNAT" -m addrtype ! --dst-type LOCAL -j SNAT --to-source 192.168.1.251 --random-fully
## 아래 'mark 0x4000/0x4000' 매칭되지 않아서 RETURN 됨!
-A KUBE-POSTROUTING -m mark ! --mark 0x4000/0x4000 -j RETURN
-A KUBE-POSTROUTING -j MARK --set-xmark 0x4000/0x0
-A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -j MASQUERADE --random-fully
...
# 카운트 확인 시 AWS-SNAT-CHAIN-0에 매칭되어, 목적지가 192.168.0.0/16 아니고 외부 빠져나갈때 SNAT 192.168.1.251(EC2 노드1 IP) 변경되어 나간다!
sudo iptables -t filter --zero; sudo iptables -t nat --zero; sudo iptables -t mangle --zero; sudo iptables -t raw --zero
watch -d 'sudo iptables -v --numeric --table nat --list AWS-SNAT-CHAIN-0; echo ; sudo iptables -v --numeric --table nat --list KUBE-POSTROUTING; echo ; sudo iptables -v --numeric --table nat --list POSTROUTING'
# conntrack 확인 : EC2 메타데이터 주소(169.254.169.254) 제외 출력
for i in $N1 $N2 $N3; do echo ">> node $i <<"; ssh ec2-user@$i sudo conntrack -L -n |grep -v '169.254.169'; echo; done
conntrack v1.4.5 (conntrack-tools):
icmp 1 28 src=172.30.66.58 dst=8.8.8.8 type=8 code=0 id=34392 src=8.8.8.8 dst=172.30.85.242 type=0 code=0 id=50705 mark=128 use=1
tcp 6 23 TIME_WAIT src=172.30.66.58 dst=34.117.59.81 sport=58144 dport=80 src=34.117.59.81 dst=172.30.85.242 sport=80 dport=44768 [ASSURED] mark=128 use=1
- Pod IP → Node → iptables SNAT → Node IP → 인터넷
- Pod가 패킷 생성
- Node로 올라옴
- POSTROUTING 진입
- iptables 룰 검사
- SNAT 적용
- 외부로 나감
5. AWS VPC CNI 설정 변경
# aws-node DaemonSet의 env 확인
kubectl get ds aws-node -n kube-system -o json | jq '.spec.template.spec.containers[0].env'
...
{
"name": "WARM_ENI_TARGET",
"value": "1"
},
...
# 노드 정보 확인 : 노드 중 1대는 eni 가 1개만 배치됨!
for i in $N1 $N2 $N3; do echo ">> node $i <<"; ssh ec2-user@$i sudo ip -c addr; echo; done
for i in $N1 $N2 $N3; do echo ">> node $i <<"; ssh ec2-user@$i sudo ip -c route; echo; done
# IpamD debugging commands https://github.com/aws/amazon-vpc-cni-k8s/blob/master/docs/troubleshooting.md
for i in $N1 $N2 $N3; do echo ">> node $i <<"; ssh ec2-user@$i curl -s http://localhost:61679/v1/enis | jq; echo; done
# add-on
addons = {
coredns = {
most_recent = true
}
kube-proxy = {
most_recent = true
}
vpc-cni = {
most_recent = true
before_compute = true
configuration_values = jsonencode({
env = {
#WARM_ENI_TARGET = "1" # 현재 ENI 외에 여유 ENI 1개를 항상 확보
WARM_IP_TARGET = "5" # 현재 사용 중인 IP 외에 여유 IP 5개를 항상 유지, 설정 시 WARM_ENI_TARGET 무시됨
MINIMUM_IP_TARGET = "10" # 노드 시작 시 최소 확보해야 할 IP 총량 10개
#ENABLE_PREFIX_DELEGATION = "true"
#WARM_PREFIX_TARGET = "1" # PREFIX_DELEGATION 사용 시, 1개의 여유 대역(/28) 유지
}
})
}
}- eks.tf 수정
# 모니터링
watch -d kubectl get pod -n kube-system -l k8s-app=aws-node # aws-node 데몬셋 파드 확인
watch -d eksctl get addon --cluster myeks # addon 확인
# 적용
terraform plan
terraform apply -auto-approve
6. 노드에 파드 생성 갯수 제한
# kube-ops-view
helm repo add geek-cookbook https://geek-cookbook.github.io/charts/
helm install kube-ops-view geek-cookbook/kube-ops-view --version 1.2.2 --set service.main.type=NodePort,service.main.ports.http.nodePort=30000 --set env.TZ="Asia/Seoul" --namespace kube-system
# 확인
kubectl get deploy,pod,svc,ep -n kube-system -l app.kubernetes.io/instance=kube-ops-view
# kube-ops-view 접속
open "http://$N1:30000/#scale=1.5"
open "http://$N1:30000/#scale=1.3"
- 최대 파드 생성 갯수 :
(Number of network interfaces for the instance type × (the number of IP addressess per network interface - 1)) + 2
## 디플로이먼트 생성
cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
ports:
- containerPort: 80
EOF
# 파드 확인
kubectl get pod -o wide
kubectl get pod -o=custom-columns=NAME:.metadata.name,IP:.status.podIP
# 파드 증가 테스트 >> 파드 정상 생성 확인, 워커 노드에서 eth, eni 갯수 확인
kubectl scale deployment nginx-deployment --replicas=8
# 파드 증가 테스트 >> 파드 정상 생성 확인, 워커 노드에서 eth, eni 갯수 확인 >> 어떤일이 벌어졌는가?
kubectl scale deployment nginx-deployment --replicas=30
# 파드 생성 실패!
kubectl events
kubectl get pods | grep Pending
nginx-deployment-7fb7fd49b4-d4bk9 0/1 Pending 0 3m37s
nginx-deployment-7fb7fd49b4-qpqbm 0/1 Pending 0 3m37s
...
7. Prefix delegation
# add-on
addons = {
coredns = {
most_recent = true
}
kube-proxy = {
most_recent = true
}
vpc-cni = {
most_recent = true
before_compute = true
configuration_values = jsonencode({
env = {
#WARM_ENI_TARGET = "1" # 현재 ENI 외에 여유 ENI 1개를 항상 확보
#WARM_IP_TARGET = "5" # 현재 사용 중인 IP 외에 여유 IP 5개를 항상 유지, 설정 시 WARM_ENI_TARGET 무시됨
#MINIMUM_IP_TARGET = "10" # 노드 시작 시 최소 확보해야 할 IP 총량 10개
ENABLE_PREFIX_DELEGATION = "true"
#WARM_PREFIX_TARGET = "1" # PREFIX_DELEGATION 사용 시, 1개의 여유 대역(/28) 유지
}
})
}
}
8. IRSA 생성 & AWS LBC 설치
# IRSA 생성 : cloudforamtion 를 통해 IAM Role 생성
CLUSTER_NAME=myeks
eksctl get iamserviceaccount --cluster $CLUSTER_NAME
kubectl get serviceaccounts -n kube-system aws-load-balancer-controller
eksctl create iamserviceaccount \
--cluster=$CLUSTER_NAME \
--namespace=kube-system \
--name=aws-load-balancer-controller \
--attach-policy-arn=arn:aws:iam::$ACCOUNT_ID:policy/AWSLoadBalancerControllerIAMPolicy \
--override-existing-serviceaccounts \
--approve
# 확인
eksctl get iamserviceaccount --cluster $CLUSTER_NAME
NAMESPACE NAME ROLE ARN
kube-system aws-load-balancer-controller arn:aws:iam::911283464785:role/eksctl-myeks-addon-iamserviceaccount-kube-sys-Role1-n7mN9x019mGE
# k8s 에 SA 확인
# Inspecting the newly created Kubernetes Service Account, we can see the role we want it to assume in our pod.
kubectl get serviceaccounts -n kube-system aws-load-balancer-controller -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::911283464785:role/eksctl-myeks-addon-iamserviceaccount-kube-sys-Role1-cNSJN97wewS7
...
# Helm Chart Repository 추가
helm repo add eks https://aws.github.io/eks-charts
helm repo update
# Helm Chart - AWS Load Balancer Controller 설치
# https://artifacthub.io/packages/helm/aws/aws-load-balancer-controller
# https://github.com/aws/eks-charts/blob/master/stable/aws-load-balancer-controller/values.yaml
helm install aws-load-balancer-controller eks/aws-load-balancer-controller -n kube-system --version 3.1.0 \
--set clusterName=$CLUSTER_NAME \
--set serviceAccount.name=aws-load-balancer-controller \
--set serviceAccount.create=false
# 확인
helm list -n kube-system
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
aws-load-balancer-controller kube-system 1 2026-03-19 14:06:42.3664 +0900 KST deployed aws-load-balancer-controller-3.1.0 v3.1.0 
# 파드 상태 실패 확인
kubectl get pod -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller
NAME READY STATUS RESTARTS AGE
aws-load-balancer-controller-7875649799-vz9jj 0/1 Error 2 (26s ago) 43s
aws-load-balancer-controller-7875649799-wn799 0/1 CrashLoopBackOff 2 (5s ago) 43s
# 로그 확인 : vpc id 정보 획득 실패!
kubectl logs -n kube-system deployment/aws-load-balancer-controller
Found 2 pods, using pod/aws-load-balancer-controller-7875649799-vz9jj
{"level":"info","ts":"2026-03-19T06:06:48Z","msg":"version","GitVersion":"v3.1.0","GitCommit":"250024dbcc7a428cfd401c949e04de23c167d46e","BuildDate":"2026-02-24T18:21:40+0000"}
{"level":"error","ts":"2026-03-19T06:06:53Z","logger":"setup","msg":"unable to initialize AWS cloud","error":"failed to get VPC ID: failed to fetch VPC ID from instance metadata: error in fetching vpc id through ec2 metadata: get mac metadata: operation error ec2imds: GetMetadata, canceled, context deadline exceeded"}
- 정상화를 위한 AWS 워커 노드들 인스턴스 메타데이터 옵션 수정 : 1 → 2 로 변경
# 디플로이먼트 리스타트!
kubectl rollout restart -n kube-system deploy aws-load-balancer-controller
# 파드 상태 확인
kubectl get pod -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller
NAME READY STATUS RESTARTS AGE
aws-load-balancer-controller-86c9674468-cprsv 1/1 Running 0 35s
aws-load-balancer-controller-86c9674468-r2kbq 1/1 Running 0 16s
# crd 확인
kubectl get crd | grep -E 'elb|gateway'
albtargetcontrolconfigs.elbv2.k8s.aws 2026-03-17T02:04:08Z
ingressclassparams.elbv2.k8s.aws 2026-03-17T02:04:08Z
targetgroupbindings.elbv2.k8s.aws 2026-03-17T02:04:08Z
listenerruleconfigurations.gateway.k8s.aws 2026-03-17T02:04:09Z
loadbalancerconfigurations.gateway.k8s.aws 2026-03-17T02:04:09Z
targetgroupconfigurations.gateway.k8s.aws 2026-03-17T02:04:09Z
kubectl explain ingressclassparams.elbv2.k8s.aws
kubectl explain ingressclassparams.elbv2.k8s.aws.spec
kubectl explain ingressclassparams.elbv2.k8s.aws.spec.listeners
kubectl explain targetgroupbindings.elbv2.k8s.aws.spec
kubectl explain albtargetcontrolconfigs.elbv2.k8s.aws.spec
# AWS Load Balancer Controller 확인
kubectl get deployment -n kube-system aws-load-balancer-controller
kubectl describe deploy -n kube-system aws-load-balancer-controller
kubectl describe deploy -n kube-system aws-load-balancer-controller | grep 'Service Account'
Service Account: aws-load-balancer-controller
# 클러스터롤, 롤 확인
kubectl describe clusterrolebindings.rbac.authorization.k8s.io aws-load-balancer-controller-rolebinding
kubectl describe clusterroles.rbac.authorization.k8s.io aws-load-balancer-controller-role
...
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
targetgroupbindings.elbv2.k8s.aws [] [] [create delete get list patch update watch]
events [] [] [create patch]
configmaps [] [] [get delete create update]
ingresses [] [] [get list patch update watch]
services [] [] [get list patch update watch]
ingresses.extensions [] [] [get list patch update watch]
services.extensions [] [] [get list patch update watch]
ingresses.networking.k8s.io [] [] [get list patch update watch]
services.networking.k8s.io [] [] [get list patch update watch]
globalaccelerators.aga.k8s.aws [] [] [get list patch watch]
listenerruleconfigurations.gateway.k8s.aws [] [] [get list watch patch]
loadbalancerconfigurations.gateway.k8s.aws [] [] [get list watch patch]
targetgroupconfigurations.gateway.k8s.aws [] [] [get list watch patch]
gatewayclasses.gateway.networking.k8s.io [] [] [get list watch patch]
gateways.gateway.networking.k8s.io [] [] [get list watch patch]
endpoints [] [] [get list watch]
namespaces [] [] [get list watch]
nodes [] [] [get list watch]
pods [] [] [get list watch]
...(생략)
ingresses/status [] [] [update patch]
pods/status [] [] [update patch]
services/status [] [] [update patch]
targetgroupbindings/status [] [] [update patch]
ingresses.elbv2.k8s.aws/status [] [] [update patch]
pods.elbv2.k8s.aws/status [] [] [update patch]
services.elbv2.k8s.aws/status [] [] [update patch]
targetgroupbindings.elbv2.k8s.aws/status [] [] [update patch]
...(생략) 
9. 서비스/파드 배포 테스트 with NLB
# 모니터링
watch -d kubectl get pod,svc,ep,endpointslices
# 디플로이먼트 & 서비스 생성
cat << EOF > echo-service-nlb.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: deploy-echo
spec:
replicas: 2
selector:
matchLabels:
app: deploy-websrv
template:
metadata:
labels:
app: deploy-websrv
spec:
terminationGracePeriodSeconds: 0
containers:
- name: aews-websrv
image: k8s.gcr.io/echoserver:1.10 # open https://registry.k8s.io/v2/echoserver/tags/list
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: svc-nlb-ip-type
annotations:
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
service.beta.kubernetes.io/aws-load-balancer-healthcheck-port: "8080"
service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
spec:
allocateLoadBalancerNodePorts: false # K8s 1.24+ 무의미한 NodePort 할당 차단
ports:
- port: 80
targetPort: 8080
protocol: TCP
type: LoadBalancer
selector:
app: deploy-websrv
EOF
kubectl apply -f echo-service-nlb.yaml
# 확인
aws elbv2 describe-load-balancers --query 'LoadBalancers[*].State.Code' --output text
kubectl get deploy,pod
kubectl get svc,ep,ingressclassparams,targetgroupbindings
kubectl get targetgroupbindings -o json
10. 서비스/파드 배포 테스트 with Ingress (L7 : HTTP)
- Ingress : 클러스터 내부의 서비스(ClusterIP, NodePort, Loadbalancer)를 외부로 노출(HTTP/HTTPS) - Web Proxy 역할
- AWS Load Balancer Controller + Ingress (ALB) IP 모드 동작 with AWS VPC CNI
# 게임 파드와 Service, Ingress 배포
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
name: game-2048
---
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: game-2048
name: deployment-2048
spec:
selector:
matchLabels:
app.kubernetes.io/name: app-2048
replicas: 2
template:
metadata:
labels:
app.kubernetes.io/name: app-2048
spec:
containers:
- image: public.ecr.aws/l6m2t8p7/docker-2048:latest
imagePullPolicy: Always
name: app-2048
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
namespace: game-2048
name: service-2048
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
type: NodePort
selector:
app.kubernetes.io/name: app-2048
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: game-2048
name: ingress-2048
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
spec:
ingressClassName: alb
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: service-2048
port:
number: 80
EOF
# 모니터링
watch -d kubectl get pod,ingress,svc,ep,endpointslices -n game-2048
# 생성 확인
kubectl get ingressclass
kubectl get ingress,svc,ep,pod -n game-2048
kubectl get-all -n game-2048
kubectl get targetgroupbindings -n game-2048
# ALB 생성 확인
aws elbv2 describe-load-balancers --query 'LoadBalancers[?contains(LoadBalancerName, `k8s-game2048`) == `true`]' | jq
ALB_ARN=$(aws elbv2 describe-load-balancers --query 'LoadBalancers[?contains(LoadBalancerName, `k8s-game2048`) == `true`].LoadBalancerArn' | jq -r '.[0]')
aws elbv2 describe-target-groups --load-balancer-arn $ALB_ARN
TARGET_GROUP_ARN=$(aws elbv2 describe-target-groups --load-balancer-arn $ALB_ARN | jq -r '.TargetGroups[0].TargetGroupArn')
aws elbv2 describe-target-health --target-group-arn $TARGET_GROUP_ARN | jq
# Ingress 확인
kubectl describe ingress -n game-2048 ingress-2048
kubectl get ingress -n game-2048 ingress-2048 -o jsonpath="{.status.loadBalancer.ingress[*].hostname}{'\n'}"
# 게임 접속 : ALB 주소로 웹 접속
kubectl get ingress -n game-2048 ingress-2048 -o jsonpath='{.status.loadBalancer.ingress[0].hostname}' | awk '{ print "Game URL = http://"$1 }'
# 파드 IP 확인
kubectl get pod -n game-2048 -owide
11. ExternalDNS
- ExternalDNS : K8S 서비스/인그레스/GatewayAPI 생성 시 도메인을 설정하면, AWS(Route 53), Azure(DNS), GCP(Cloud DNS) 에 A 레코드(TXT 레코드)로 자동 생성/삭제
# 정책 파일 작성
cat << EOF > externaldns_controller_policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"route53:ChangeResourceRecordSets",
"route53:ListResourceRecordSets",
"route53:ListTagsForResources"
],
"Resource": [
"arn:aws:route53:::hostedzone/*"
]
},
{
"Effect": "Allow",
"Action": [
"route53:ListHostedZones"
],
"Resource": [
"*"
]
}
]
}
EOF
# IAM 정책 생성
aws iam create-policy \
--policy-name ExternalDNSControllerPolicy \
--policy-document file://externaldns_controller_policy.json
# 확인
ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text)
aws iam get-policy --policy-arn arn:aws:iam::$ACCOUNT_ID:policy/ExternalDNSControllerPolicy | jq
# IRSA 생성 : cloudforamtion 를 통해 IAM Role 생성
CLUSTER_NAME=myeks
eksctl create iamserviceaccount \
--cluster=$CLUSTER_NAME \
--namespace=kube-system \
--name=external-dns \
--attach-policy-arn=arn:aws:iam::$ACCOUNT_ID:policy/ExternalDNSControllerPolicy \
--override-existing-serviceaccounts \
--approve
# 확인
eksctl get iamserviceaccount --cluster $CLUSTER_NAME
NAMESPACE NAME ROLE ARN
kube-system aws-load-balancer-controller arn:aws:iam::911283464785:role/eksctl-myeks-addon-iamserviceaccount-kube-sys-Role1-2REDIfJ9sfGa
kube-system external-dns arn:aws:iam::911283464785:role/eksctl-myeks-addon-iamserviceaccount-kube-sys-Role1-F76qLAoueSgl
# k8s 에 SA 확인
# Inspecting the newly created Kubernetes Service Account, we can see the role we want it to assume in our pod.
kubectl get serviceaccounts -n kube-system external-dns -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::911283464785:role/eksctl-myeks-addon-iamserviceaccount-kube-sys-Role1-F76qLAoueSgl
...
# 자신의 도메인 변수 지정
MyDomain=<자신의 도메인>
MyDomain=gasida.link
# 설정 파일 작성
cat << EOF > external-dns-values.yaml
provider: aws
# 위에서 생성한 ServiceAccount와 연동
serviceAccount:
create: false
name: external-dns
# 필터링 설정 (보안상 권장)
# 특정 도메인만 관리하도록 제한 (예: example.com)
domainFilters:
- $MyDomain
# 레코드 업데이트 정책
# sync: 쿠버네티스에서 삭제되면 Route 53에서도 삭제 (주의 필요)
# upsert-only: 생성/수정만 하고 삭제는 수동으로 (안전함)
policy: sync
# 리소스 감지 대상
sources:
- service
- ingress
# (선택) 텍스트 레코드에 식별자 추가 (여러 클러스터가 동일 도메인 관리 시 충돌 방지)
txtOwnerId: "stduy-myeks-cluster"
registry: txt
# 로그 레벨
logLevel: info
EOF
# Helm 레포지토리 추가 및 업데이트
helm repo add external-dns https://kubernetes-sigs.github.io/external-dns/
helm repo update
# 차트 설치
helm install external-dns external-dns/external-dns \
-n kube-system \
-f external-dns-values.yaml
# 확인
helm list -n kube-system
kubectl get pod -l app.kubernetes.io/name=external-dns -n kube-system
# 로그 모니터링
kubectl logs deploy/external-dns -n kube-system -f
# 터미널1 (모니터링)
watch -d 'kubectl get pod,svc'
kubectl logs deploy/external-dns -n kube-system -f
혹은
kubectl stern -l app.kubernetes.io/name=external-dns -n kube-system
# 테트리스 디플로이먼트 배포
cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: tetris
labels:
app: tetris
spec:
replicas: 1
selector:
matchLabels:
app: tetris
template:
metadata:
labels:
app: tetris
spec:
containers:
- name: tetris
image: bsord/tetris
---
apiVersion: v1
kind: Service
metadata:
name: tetris
annotations:
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
#service.beta.kubernetes.io/aws-load-balancer-healthcheck-port: "80"
spec:
selector:
app: tetris
ports:
- port: 80
protocol: TCP
targetPort: 80
type: LoadBalancer
EOF
# 배포 확인
kubectl get deploy,svc,ep tetris
# NLB에 ExternanDNS 로 도메인 연결
kubectl annotate service tetris "external-dns.alpha.kubernetes.io/hostname=tetris.$MyDomain"
while true; do aws route53 list-resource-record-sets --hosted-zone-id "${MyDnzHostedZoneId}" --query "ResourceRecordSets[?Type == 'A']" | jq ; date ; echo ; sleep 1; done
# Route53에 A레코드 확인
aws route53 list-resource-record-sets --hosted-zone-id "${MyDnzHostedZoneId}" --query "ResourceRecordSets[?Type == 'A']" | jq
# 확인
dig +short tetris.$MyDomain @8.8.8.8
dig +short tetris.$MyDomain
# 도메인 체크
echo -e "My Domain Checker Site1 = https://www.whatsmydns.net/#A/tetris.$MyDomain"
echo -e "My Domain Checker Site2 = https://dnschecker.org/#A/tetris.$MyDomain"
# 웹 접속 주소 확인 및 접속
echo -e "Tetris Game URL = http://tetris.$MyDomain"
12. Gateway API
사전준비
# LBC > v2.13.0 버전 이상
kubectl describe pod -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller | grep Image: | uniq
Image: public.ecr.aws/eks/aws-load-balancer-controller:v3.1.0
# Installation of Gateway API CRDs # --server-side=true
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.3.0/standard-install.yaml # [REQUIRED] # Standard Gateway API CRDs
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.3.0/experimental-install.yaml # [OPTIONAL: Used for L4 Routes] # Experimental Gateway API CRDs
kubectl get crd | grep gateway.networking
backendtlspolicies.gateway.networking.k8s.io 2026-03-19T08:37:54Z
gatewayclasses.gateway.networking.k8s.io 2026-03-19T08:37:03Z
gateways.gateway.networking.k8s.io 2026-03-19T08:37:04Z
grpcroutes.gateway.networking.k8s.io 2026-03-19T08:37:04Z
httproutes.gateway.networking.k8s.io 2026-03-19T08:37:04Z
referencegrants.gateway.networking.k8s.io 2026-03-19T08:37:05Z
tcproutes.gateway.networking.k8s.io 2026-03-19T08:37:55Z
tlsroutes.gateway.networking.k8s.io 2026-03-19T08:37:56Z
udproutes.gateway.networking.k8s.io 2026-03-19T08:37:56Z
xbackendtrafficpolicies.gateway.networking.x-k8s.io 2026-03-19T08:37:56Z
xlistenersets.gateway.networking.x-k8s.io 2026-03-19T08:37:56Z
kubectl api-resources | grep gateway.networking
gatewayclasses gc gateway.networking.k8s.io/v1 false GatewayClass
gateways gtw gateway.networking.k8s.io/v1 true Gateway
grpcroutes gateway.networking.k8s.io/v1 true GRPCRoute
httproutes gateway.networking.k8s.io/v1 true HTTPRoute
referencegrants refgrant gateway.networking.k8s.io/v1beta1 true ReferenceGrant
tcproutes gateway.networking.k8s.io/v1alpha2 true TCPRoute
tlsroutes gateway.networking.k8s.io/v1alpha2 true TLSRoute
udproutes gateway.networking.k8s.io/v1alpha2 true UDPRoute
backendtlspolicies btlspolicy gateway.networking.k8s.io/v1alpha3 true BackendTLSPolicy
xbackendtrafficpolicies xbtrafficpolicy gateway.networking.x-k8s.io/v1alpha1 true XBackendTrafficPolicy
xlistenersets lset gateway.networking.x-k8s.io/v1alpha1 true XListenerSet
kubectl explain gatewayclasses.gateway.networking.k8s.io.spec
kubectl explain gateways.gateway.networking.k8s.io.spec
kubectl explain httproutes.gateway.networking.k8s.io.spec
# Installation of LBC Gateway API specific CRDs
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/refs/heads/main/config/crd/gateway/gateway-crds.yaml
kubectl get crd | grep gateway.k8s.aws
listenerruleconfigurations.gateway.k8s.aws 2026-03-19T06:05:41Z
loadbalancerconfigurations.gateway.k8s.aws 2026-03-19T06:05:42Z
targetgroupconfigurations.gateway.k8s.aws 2026-03-19T06:05:41Z
kubectl api-resources | grep gateway.k8s.aws
listenerruleconfigurations gateway.k8s.aws/v1beta1 true ListenerRuleConfiguration
loadbalancerconfigurations gateway.k8s.aws/v1beta1 true LoadBalancerConfiguration
targetgroupconfigurations gateway.k8s.aws/v1beta1 true TargetGroupConfiguration
kubectl explain loadbalancerconfigurations.gateway.k8s.aws.spec
kubectl explain listenerruleconfigurations.gateway.k8s.aws.spec
kubectl explain targetgroupconfigurations.gateway.k8s.aws.spec
# 설치 정보 확인
helm list -n kube-system
helm get values -n kube-system aws-load-balancer-controller # helm values 에 Args 및 활성화 값이 현재는 없음
kubectl describe deploy -n kube-system aws-load-balancer-controller | grep Args: -A2
Args:
--cluster-name=myeks
--ingress-class=alb
# 모니터링
kubectl get pod -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller --watch
# deployment 에 feature flag를 활성화 : By default, the LBC will not listen to Gateway API CRDs.
KUBE_EDITOR="nano" kubectl edit deploy -n kube-system aws-load-balancer-controller
...
- args:
- --cluster-name=myeks
- --ingress-class=alb
- --feature-gates=NLBGatewayAPI=true,ALBGatewayAPI=true
...
# 확인
kubectl describe deploy -n kube-system aws-load-balancer-controller | grep Args: -A3
Args:
--cluster-name=myeks
--ingress-class=alb
--feature-gates=NLBGatewayAPI=true,ALBGatewayAPI=true
# deployment 에 gateway api 지원 설정 : external-dns-values.yaml 파일 편집 -> 아래 추가
--------------------------
# 리소스 감지 대상
sources:
- service
- ingress
- gateway-httproute
- gateway-grpcroute
- gateway-tlsroute
- gateway-tcproute
- gateway-udproute
--------------------------
# ExternalDNS 에 gateway api 지원 설정
helm upgrade -i external-dns external-dns/external-dns -n kube-system -f external-dns-values.yaml
# 확인
kubectl describe deploy -n kube-system external-dns | grep Args: -A15
Args:
--log-level=info
--log-format=text
--interval=1m
--source=service
--source=ingress
--source=gateway-httproute
--source=gateway-grpcroute
--source=gateway-tlsroute
--source=gateway-tcproute
--source=gateway-udproute
--policy=sync
--registry=txt
--txt-owner-id=stduy-myeks-cluster
--domain-filter=gasida.link
--provider=aws
샘플 애플리케이션 배포 및 gateway api(externaldns) 를 사용한 http 접속
# loadbalancerconfigurations 생성
kubectl explain loadbalancerconfigurations.gateway.k8s.aws.spec
...
scheme <string>
enum: internal, internet-facing
scheme defines the type of LB to provision. If unspecified, it will be
automatically inferred.
...
cat << EOF | kubectl apply -f -
apiVersion: gateway.k8s.aws/v1beta1
kind: LoadBalancerConfiguration
metadata:
name: lbc-config
namespace: default
spec:
scheme: internet-facing
EOF
# 확인
kubectl get loadbalancerconfiguration -owide
NAME AGE
lbc-config 8s
# gatewayclasses 생성
kubectl explain gatewayclasses.spec
kubectl explain gatewayclasses.spec.parametersRef
cat << EOF | kubectl apply -f -
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: aws-alb
spec:
controllerName: gateway.k8s.aws/alb
parametersRef:
group: gateway.k8s.aws
kind: LoadBalancerConfiguration
name: lbc-config
namespace: default
EOF
# gatewayclasses 확인 : 약어 gc
kubectl get gatewayclasses -o wide # k get gc
NAME CONTROLLER ACCEPTED AGE DESCRIPTION
aws-alb gateway.k8s.aws/alb True 63s
# gateways 생성
kubectl explain gateways.spec
cat << EOF | kubectl apply -f -
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: alb-http
spec:
gatewayClassName: aws-alb
listeners:
- name: http
protocol: HTTP
port: 80
EOF
# gateways 확인 : 약어 gtw
kubectl get gateways # k get gtw
NAME CLASS ADDRESS PROGRAMMED AGE
alb-http aws-alb k8s-default-albhttp-8d7d6da11f-126923743.ap-northeast-2.elb.amazonaws.com Unknown 24s
# ALB 생성 확인
aws elbv2 describe-load-balancers | jq
aws elbv2 describe-target-groups
# 로그 모니터링
kubectl logs -l app.kubernetes.io/name=aws-load-balancer-controller -n kube-system -f
혹은
kubectl stern -l app.kubernetes.io/name=aws-load-balancer-controller -n kube-system
# 게임 파드와 Service 배포
cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment-2048
spec:
selector:
matchLabels:
app.kubernetes.io/name: app-2048
replicas: 2
template:
metadata:
labels:
app.kubernetes.io/name: app-2048
spec:
containers:
- image: public.ecr.aws/l6m2t8p7/docker-2048:latest
imagePullPolicy: Always
name: app-2048
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: service-2048
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
type: ClusterIP
selector:
app.kubernetes.io/name: app-2048
EOF
# 모니터링
watch -d kubectl get pod,ingress,svc,ep,endpointslices
# 생성 확인
kubectl get svc,ep,pod
# TargetGroupConfiguration 생성
kubectl explain httproutes.gateway.k8s.aws.spec
kubectl explain targetgroupconfigurations.gateway.k8s.aws.spec.defaultConfiguration
cat << EOF | kubectl apply -f -
apiVersion: gateway.k8s.aws/v1beta1
kind: TargetGroupConfiguration
metadata:
name: backend-tg-config
spec:
targetReference:
name: service-2048
defaultConfiguration:
targetType: ip
protocol: HTTP
EOF
# 확인
kubectl get targetgroupconfigurations -owide
NAME SERVICE-NAME AGE
backend-tg-config backend 22s
# ALB 확인
aws elbv2 describe-load-balancers | jq
aws elbv2 describe-target-groups | jq
# 서비스 도메인명 변수 지정
GWMYDOMAIN=<각자 자신의 도메인명>
GWMYDOMAIN=gwapi.gasida.link
# httproute 생성
kubectl explain httproutes.spec
kubectl explain httproutes.spec.parentRefs
kubectl explain httproutes.spec.hostnames
kubectl explain httproutes.spec.rules
cat << EOF | kubectl apply -f -
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: alb-http-route
spec:
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: alb-http
sectionName: http
hostnames:
- $GWMYDOMAIN
rules:
- backendRefs:
- name: service-2048
port: 80
EOF
# 확인
kubectl get httproute
NAME HOSTNAMES AGE
alb-http-route ["gwapi.gasida.link"] 18s
# ALB 확인
aws elbv2 describe-load-balancers | jq
aws elbv2 describe-target-groups | jq
# 로그 모니터링
kubectl logs -l app.kubernetes.io/name=aws-load-balancer-controller -n kube-system -f
혹은
kubectl stern -l app.kubernetes.io/name=aws-load-balancer-controller -n kube-system
# 확인
dig +short $GWMYDOMAIN @8.8.8.8
dig +short $GWMYDOMAIN
# 도메인 체크
echo -e "My Domain Checker Site1 = https://www.whatsmydns.net/#A/$GWMYDOMAIN"
echo -e "My Domain Checker Site2 = https://dnschecker.org/#A/$GWMYDOMAIN"
# 웹 접속 주소 확인 및 접속
echo -e "GW Api Sample URL = http://$GWMYDOMAIN"
참조
https://docs.aws.amazon.com/eks/latest/userguide/managing-vpc-cni.html
https://github.com/aws/amazon-vpc-cni-k8s
https://github.com/aws/amazon-vpc-cni-k8s/blob/master/docs/cni-proposal.md
https://docs.aws.amazon.com/eks/latest/best-practices/networking.html
https://github.com/Praqma/Network-MultiTool
클라우드 왕초보