IPAM
IPAM이란?
- IP Address Management
- 네트워크에서 IP 주소를 자동으로 할당하고 관리하는 시스템
- 이것을 CILIUM이 EBPF로 빠르게 처리 관리한다.
- 빠른 Pod 생성: 기존 DHCP 방식보다 IP 할당이 훨씬 빠름
- IP 충돌 방지: 자동으로 중복되지 않는 IP 할당
- 확장성: 대규모 클러스터에서도 효율적으로 동작
- 네트워크 정책 통합: Cilium의 보안 정책과 seamless하게 연동
- 멀티 클러스터 지원: 여러 클러스터 간 IP 관리 가능
실습으로 확인해보기 1
Kubernetes Host Scope
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl cluster-info dump | grep -m 2 -E "cluster-cidr|service-cluster-ip-range"
"--service-cluster-ip-range=10.96.0.0/16",
"--cluster-cidr=10.244.0.0/16",누가 ipam 관리하는지 확인 -> k8s
(⎈|HomeLab:N/A) root@k8s-ctr:~# cilium config view | grep ^ipam
ipam kubernetes노드별 파드에 할당되는 IPAM(PodCIDR) 정보 확인
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.podCIDR}{"\n"}{end}'
k8s-ctr 10.244.0.0/24
k8s-w1 10.244.1.0/24파드 정보 : 상태, 파드 IP 확인
kubectl get ciliumendpoints.cilium.io -A
NAMESPACE NAME SECURITY IDENTITY ENDPOINT STATE IPV4 IPV6
cilium-monitoring grafana-5c69859d9-2czgt 9298 ready 10.244.0.1
cilium-monitoring prometheus-6fc896bc5d-969vp 15227 ready 10.244.0.32
kube-system coredns-674b8bbfcf-5ww5d 56341 ready 10.244.0.97
kube-system coredns-674b8bbfcf-bklsp 56341 ready 10.244.0.231
kube-system hubble-relay-5dcd46f5c-tqcq9 26396 ready 10.244.0.109
kube-system hubble-ui-76d4965bb6-sjv5w 1204 ready 10.244.0.65
local-path-storage local-path-provisioner-74f9666bc9-84q8f 305 ready 10.244.0.164샘플 pod 띄우기
(⎈|HomeLab:N/A) root@k8s-ctr:~# k get po
NAME READY STATUS RESTARTS AGE
curl-pod 1/1 Running 0 94s
webpod-697b545f57-vgldc 1/1 Running 0 95s
webpod-697b545f57-xlswp 1/1 Running 0 95spod Ednpoint 정도만 확인
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get endpointslices -l app=webpod
NAME ADDRESSTYPE PORTS ENDPOINTS AGE
webpod-f7sd2 IPv4 80 10.244.0.90,10.244.1.69 3m25s통신 확인
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it curl-pod -- curl webpod | grep Hostname
Hostname: webpod-697b545f57-vgldc
kubectl exec -it curl-pod -- sh -c 'while true; do curl -s webpod | grep Hostname; sleep 1; done'hubble 확인
(⎈|HomeLab:N/A) root@k8s-ctr:~# hubble observe -f --protocol tcp --pod curl-pod
주요 해석 :
Aug 2 15:20:17.530: default/curl-pod (ID:23342) <> 10.96.62.19:80 (world) pre-xlate-fwd TRACED (TCP)
-> Nat 일어나기전 source IP 추적 (10.96.62.19) cluster IP 확인
-> 그리고 지금은 Default 이므로 Socker LB에 의해서 Destination IP로 변환 확인
Aug 2 15:20:17.530: default/curl-pod (ID:23342) <> default/webpod-697b545f57-vgldc:80 (ID:30094) post-xlate-fwd TRANSLATED (TCP)
pre-xlate-fwd , TRACED : NAT (IP 변환) 전 , 추적 중인 flow
post-xlate-fwd , TRANSLATED : NAT 후의 흐름 , NAT 변환이 일어났음
tcpdump 도 한번 확인
(⎈|HomeLab:N/A) root@k8s-ctr:~# tcpdump -i eth1 tcp port 80 -nn
00:33:00.798141 IP 10.244.1.69.80 > 10.244.0.178.48636: Flags [S.], seq 463285656, ack 4084563403, win 65160, options [mss 1460,sackOK,TS val 1527186764 ecr 2488876476,nop,wscale 7], length 0
00:33:00.798818 IP 10.244.0.178.48636 > 10.244.1.69.80: Flags [.], ack 1, win 502, options [nop,nop,TS val 2488876478 ecr 1527186764], length 0
00:33:00.799292 IP 10.244.0.178.48636 > 10.244.1.69.80: Flags [P.], seq 1:71, ack 1, win 502, options [nop,nop,TS val 2488876479 ecr 1527186764], length 70: HTTP: GET / HTTP/1.1
00:33:00.801462 IP 10.244.1.69.80 > 10.244.0.178.48636: Flags [.], ack 71, win 509, options [nop,nop,TS val 1527186767 ecr 2488876479], length 0
00:33:00.812844 IP 10.244.1.69.80 > 10.244.0.178.48636: Flags [P.], seq 1:322, ack 71, win 509, options [nop,nop,TS val 1527186778 ecr 2488876479], length 321: HTTP: HTTP/1.1 200 OK
IPAM 을 cilium이 하도록 변경 실습
Cluster Scopre 로 설정 변경
helm upgrade cilium cilium/cilium --namespace kube-system --reuse-values \
--set ipam.mode="cluster-pool" --set ipam.operator.clusterPoolIPv4PodCIDRList={"172.20.0.0/16"} --set ipv4NativeRoutingCIDR=172.20.0.0/16
error 발생
(⎈|HomeLab:kube-system) root@k8s-ctr:~# helm upgrade cilium cilium/cilium --namespace kube-system --reuse-values --set ipam.mode="cluster-pool" --set ipam.operator.clusterPoolIPv4PodCIDRList={"172.20.0.0/16"} --set ipv4NativeRoutingCIDR=172.20.0.0/16
Error: UPGRADE FAILED: template: cilium/templates/cilium-operator/deployment.yaml:145:26: executing "cilium/templates/cilium-operator/deployment.yaml" at <.Values.k8sServiceHostRef.name>: nil pointer evaluating interface {}.name
(⎈|HomeLab:kube-system) root@k8s-ctr:~#
-> --reuse-values 빼고 실행하니 되었는데, version이 1.18로 올라감
(⎈|HomeLab:kube-system) root@k8s-ctr:~# helm upgrade cilium cilium/cilium --namespace kube-system --set ipam.mode="cluster-pool" --set ipam.operator.clusterPoolIPv4PodCIDRList={"172.20.0.0/16"} --set ipv4NativeRoutingCIDR=172.20.0.0/16
Release "cilium" has been upgraded. Happy Helming!
NAME: cilium
LAST DEPLOYED: Sun Aug 3 00:47:31 2025
NAMESPACE: kube-system
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
You have successfully installed Cilium with Hubble.
Your release version is 1.18.0.
For any further help, visit https://docs.cilium.io/en/v1.18/gettinghelp
(⎈|HomeLab:kube-system) root@k8s-ctr:~#
(⎈|HomeLab:kube-system) root@k8s-ctr:~# kubectl -n kube-system rollout restart deploy/cilium-operator
deployment.apps/cilium-operator restarted
(⎈|HomeLab:kube-system) root@k8s-ctr:~# kubectl -n kube-system rollout restart ds/cilium
daemonset.apps/cilium restarted상태 확인
(⎈|HomeLab:kube-system) root@k8s-ctr:~# kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.podCIDR}{"\n"}{end}'
k8s-ctr 10.244.0.0/24
k8s-w1 10.244.1.0/24
-> 안바뀜 (원하는 172.20.#.# 대역으로 )
(⎈|HomeLab:kube-system) root@k8s-ctr:~#
(⎈|HomeLab:kube-system) root@k8s-ctr:~# cilium config view | grep ^ipam
ipam cluster-pool
-> ipam 바뀜
(⎈|HomeLab:kube-system) root@k8s-ctr:~# kubectl get ciliumendpoints.cilium.io -A
NAMESPACE NAME SECURITY IDENTITY ENDPOINT STATE IPV4 IPV6
cilium-monitoring grafana-5c69859d9-2czgt 9298 ready 10.244.0.1
cilium-monitoring prometheus-6fc896bc5d-969vp 15227 ready 10.244.0.32
default curl-pod 23342 ready 10.244.0.178
default webpod-697b545f57-vgldc 30094 ready 10.244.1.69
default webpod-697b545f57-xlswp 30094 ready 10.244.0.90
kube-system coredns-674b8bbfcf-5ww5d 56341 ready 10.244.0.97
kube-system coredns-674b8bbfcf-bklsp 56341 ready 10.244.0.231
local-path-storage local-path-provisioner-74f9666bc9-84q8f 305 ready 10.244.0.164
-> pod들도 기존 pod 할당되어 동작중..어쩌면 당연함..안정성을 고려한다면.
(⎈|HomeLab:kube-system) root@k8s-ctr:~# kubectl delete ciliumnode k8s-w1
ciliumnode.cilium.io "k8s-w1" deleted
kubectl -n kube-system rollout restart ds/cilium
(⎈|HomeLab:kube-system) root@k8s-ctr:~# kubectl get ciliumnode -o json | grep podCIDRs -A2
"podCIDRs": [
"10.244.0.0/24"
],
-- "podCIDRs": [
"172.20.0.0/24"
],
-> 지운 노드만 바뀐 셋팅으로 되었다!
동일하게 나머지 노드도 작업하고 ep 확인
(⎈|HomeLab:kube-system) root@k8s-ctr:~# kubectl get ciliumendpoints.cilium.io -A # 파드 IP 변경 되는가?
NAMESPACE NAME SECURITY IDENTITY ENDPOINT STATE IPV4 IPV6
kube-system coredns-674b8bbfcf-vbj54 56341 ready 172.20.0.115
(⎈|HomeLab:kube-system) root@k8s-ctr:~# ip -c route
default via 10.0.2.2 dev eth0 proto dhcp src 10.0.2.15 metric 100
10.0.2.0/24 dev eth0 proto kernel scope link src 10.0.2.15 metric 100
10.0.2.2 dev eth0 proto dhcp scope link src 10.0.2.15 metric 100
10.0.2.3 dev eth0 proto dhcp scope link src 10.0.2.15 metric 100
10.10.0.0/16 via 192.168.10.200 dev eth1 proto static
172.20.0.0/24 via 172.20.1.56 dev cilium_host proto kernel src 172.20.1.56 mtu 1450
172.20.1.0/24 via 172.20.1.56 dev cilium_host proto kernel src 172.20.1.56
172.20.1.56 dev cilium_host proto kernel scope link
-> static routing에 추가도 되었네요?하지만 pod들은 running 중인 상태라 기존 ip
(⎈|HomeLab:kube-system) root@k8s-ctr:~# kubectl get pod -A -owide | grep 10.244.
cilium-monitoring grafana-5c69859d9-2czgt 0/1 Running 0 99m 10.244.0.1 k8s-ctr
cilium-monitoring prometheus-6fc896bc5d-969vp 1/1 Running 0 99m 10.244.0.32 k8s-ctr
default curl-pod 1/1 Running 0 73m 10.244.0.178 k8s-ctr
default webpod-697b545f57-vgldc 1/1 Running 0 73m 10.244.1.69 k8s-w1
default webpod-697b545f57-xlswp 1/1 Running 0 73m 10.244.0.90 k8s-ctr
local-path-storage local-path-provisioner-74f9666bc9-84q8f 1/1 Running 0 99m 10.244.0.164 k8s-ctr
전부 다시 restart 해주니 바뀌었음 (일부만 확인해보자)
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
curl-pod 1/1 Running 0 19s 172.20.1.197 k8s-ctr
webpod-697b545f57-mfnbg 1/1 Running 0 43s 172.20.1.38 k8s-ctr
webpod-697b545f57-nmzq2 1/1 Running 0 56s 172.20.0.101 k8s-w1
(⎈|HomeLab:default) root@k8s-ctr:~# (⎈|HomeLab:default) root@k8s-ctr:~# k get po -owide
(⎈|HomeLab:default) root@k8s-ctr:~#
Hostname: webpod-697b545f57-mfnbg
