Istio Hands-on Study - 4주차 Observability (EKS+Ingress-Gateway/Gateway-API 배포)
Istio Hands-on Study
목록 보기
4/10
가시다(gasida) 님이 진행하는 Istio Hands-on Study 1기 과정을 참여하여 정리한 글입니다.
4주차는 Observability 주제로 학습을 진행하였습니다.
4주차 과제는 개인 준비한 과제로 대체합니다.
Istio 프로젝트는 2024년 8월에 In-Cluster IstioOperator의 사용 중단을 발표하였으며,
이 기능은 Istio 1.24 버전부터 완전히 제거되었습니다.
IstioOperator 지원이 중단되어 최신 버전의 경우 더 이상 IstioOperator 기반으로 EKS에서 NLB로 Ingress 구성할 수 없어서 다른 방법을 찾는 과정으로 아래 내용을 정리하였습니다.
🛠️ Terraform을 이용하여 EKS v1.32 생성
🚚 Helm 이용하여 istio-base(CRD)와 istiod, istio-ingress-gateway 배포
📇 External DNS Controller와 📧 ACM 연동하여 개인 도메인으로 https 통신 환경을 구성
📕 bookinfo 배포하는 과정에서 문제해결한 내용 포함 🍎.
✅ 최신 경향인 Kubernetes Gateway API 방식으로 외부접속 구성 방법도 포함
⁉️ Q & A : 실습을 통해 아래 질문들에 대한 답변을 얻을 수 있습니다.
1. ingress-gateway, k8s-gateway 배포 후 NLB가 internal로 생성되어 외부에서 접속할 수 없습니다.
2. 80,443 Port에 대한 TargetGroup이 unhealthy 상태 입니다.
3. elb 생성 후 route 53 hostedZone에 자동 등록은 어떻게 하나요?
4. EKS 생성 시 마다 봉투 암호화 위해 고객 관리형 암호화키(KMS)가 매번 신규 생생되어 비용이 증가합니다.
5. 암호화 통신(https)을 적용하고 싶습니다.
6. 개인 도메인을 사용하고 싶은데 gateway 설정은 어떻게 하나요?
1. EKS v1.32 & Istio v1.25.2 배포
1.1 준비사항
- terraform, kubectl, iam accesskey, route 53, acm 인증서는
각자 사전 준비 또는 이해하고 있다고 전제하고 설명은 생략하겠습니다. - 실습한 환경은 macOS에서 AWS 콘솔과 AWS CLI로 진행합니다
1.2 Terraform 코드 내용
- variables.tf
- cluster_name, my_hosted_zone_id, my_domain, acm_arn 내용 각자 상황에 맞게 수정
cat << EOF > variables.tf
###########################
# Variable
###########################
variable "cluster_name" {
description = "The name of the EKS cluster"
type = string
default = "istio-sejkim"
}
variable "cluster_version" {
description = "The version of the EKS cluster"
type = string
default = "1.32"
}
# External DNS
variable "attach_external_dns_policy" {
description = "Determines whether to attach the External DNS IAM policy to the role"
type = bool
default = true
}
variable "my_hosted_zone_id" {
description = "MyDnsHostedZoneId"
type = string
default = "arn:aws:route53:::hostedzone/Z0756*****C36UV"
}
variable "acm_arn" {
description = "ACM ARN"
type = string
default = "arn:aws:acm:ap-northeast-2:1**********3:certificate/415404eb-e2e2-4744-b2e4-1108735b5903"
}
variable "ingress_name" {
description = "Ingress Name"
type = string
default = "ing-bookinfo"
}
variable "my_domain" {
description = "My Domain"
type = string
default = "ksj7279.click"
}
EOF- data.tf
cat << EOF > data.tf
##########################################
# local variables to be used in the module
##########################################
locals {
name = basename(path.cwd)
region = "ap-northeast-2"
vpc_cidr = "10.0.0.0/16"
azs = slice(data.aws_availability_zones.available.names, 0, 3)
istio_chart_url = "https://istio-release.storage.googleapis.com/charts"
istio_chart_version = "1.25.2"
tags = {
Blueprint = local.name
GithubRepo = "github.com/aws-ia/terraform-aws-eks-blueprints"
}
}
##########################################
# Data sources to be used in the module
##########################################
data "aws_ecrpublic_authorization_token" "token" {
provider = aws.virginia
}
data "aws_availability_zones" "available" {
# Do not include local zones
filter {
name = "opt-in-status"
values = ["opt-in-not-required"]
}
}
# 기 생성된 KMS Key 재사용 하고자 할 경우
data "aws_kms_key" "by_key_arn" {
key_id = "arn:aws:kms:ap-northeast-2:1**********3:key/2233822a-1cd4-4672-9f06-4c5df03defa9"
}
EOF- provider.tf
cat << EOF > provider.tf
###########################
# Provider
###########################
terraform {
required_version = ">= 1.3"
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 5.34"
}
helm = {
source = "hashicorp/helm"
version = ">= 2.9"
}
kubernetes = {
source = "hashicorp/kubernetes"
version = ">= 2.20"
}
}
}
provider "aws" {
region = local.region
}
# Required for public ECR where Karpenter artifacts are hosted
provider "aws" {
region = "us-east-1"
alias = "virginia"
}
provider "kubernetes" {
host = module.eks.cluster_endpoint
cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)
exec {
api_version = "client.authentication.k8s.io/v1beta1"
command = "aws"
# This requires the awscli to be installed locally where Terraform is executed
args = ["eks", "get-token", "--cluster-name", module.eks.cluster_name]
}
}
provider "helm" {
kubernetes {
host = module.eks.cluster_endpoint
cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)
exec {
api_version = "client.authentication.k8s.io/v1beta1"
command = "aws"
# This requires the awscli to be installed locally where Terraform is executed
args = ["eks", "get-token", "--cluster-name", module.eks.cluster_name]
}
}
}
EOF- vpc.tf
cat << EOF > vpc.tf
###########################
# VPC
###########################
module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "~> 5.0"
name = local.name
cidr = local.vpc_cidr
azs = local.azs
private_subnets = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 4, k)]
public_subnets = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k + 48)]
enable_nat_gateway = true
single_nat_gateway = true
public_subnet_tags = {
"kubernetes.io/role/elb" = 1
}
private_subnet_tags = {
"kubernetes.io/role/internal-elb" = 1
# Tags subnets for Karpenter auto-discovery
"karpenter.sh/discovery" = var.cluster_name
}
tags = local.tags
}
EOF- eks.tf
cat << EOF > eks.tf
###########################
# EKS Cluster
###########################
module "eks" {
source = "terraform-aws-modules/eks/aws"
version = "~> 20.11"
cluster_name = var.cluster_name
cluster_version = var.cluster_version
cluster_endpoint_public_access = true
# Give the Terraform identity admin access to the cluster
# which will allow resources to be deployed into the cluster
enable_cluster_creator_admin_permissions = true
vpc_id = module.vpc.vpc_id
subnet_ids = module.vpc.private_subnets
# KMS Key가 없는 경우 true로 설정하면 AWS가 관리하는 KMS Key를 사용합니다.
# KMS Key가 있는 경우 false로 설정하면 사용자가 관리하는 KMS Key를 사용합니다.
create_kms_key = false
cluster_encryption_config = {
provider_key_arn = tostring(data.aws_kms_key.by_key_arn.arn)
resources = ["secrets"]
}
eks_managed_node_groups = {
istio-sejkim-ng = {
instance_types = ["t3.medium"]
min_size = 3
max_size = 6
desired_size = 3
}
}
# EKS K8s API cluster needs to be able to talk with the EKS worker nodes with port 15017/TCP and 15012/TCP which is used by Istio
# Istio in order to create sidecar needs to be able to communicate with webhook and for that network passage to EKS is needed.
node_security_group_additional_rules = {
ingress_15017 = {
description = "Cluster API - Istio Webhook namespace.sidecar-injector.istio.io"
protocol = "TCP"
from_port = 15017
to_port = 15017
type = "ingress"
source_cluster_security_group = true
}
ingress_15012 = {
description = "Cluster API to nodes ports/protocols"
protocol = "TCP"
from_port = 15012
to_port = 15012
type = "ingress"
source_cluster_security_group = true
}
}
tags = local.tags
}
###########################
# EKS Blueprints Addons
###########################
resource "kubernetes_namespace_v1" "istio_system" {
metadata {
name = "istio-system"
}
}
module "eks_blueprints_addons" {
source = "aws-ia/eks-blueprints-addons/aws"
version = "~> 1.21"
cluster_name = module.eks.cluster_name
cluster_endpoint = module.eks.cluster_endpoint
cluster_version = module.eks.cluster_version
oidc_provider_arn = module.eks.oidc_provider_arn
eks_addons = {
coredns = {
most_recent = true
}
vpc-cni = {
most_recent = true
}
kube-proxy = {
most_recent = true
}
}
# This is required to expose Istio Ingress Gateway
enable_aws_load_balancer_controller = true
# Karpenter 배포 시 AWS ECR 접근을 위해 AWS ECR Public에 대한 인증 토큰을 사용합니다.
# Karpenter는 AWS ECR Public에 대한 인증 토큰을 사용하여 Karpenter의 Helm Chart를 다운로드합니다. 필요시 주석을 해제합니다.
# enable_karpenter = true
# karpenter = {
# repository_username = data.aws_ecrpublic_authorization_token.token.user_name
# repository_password = data.aws_ecrpublic_authorization_token.token.password
# }
enable_kube_prometheus_stack = false
enable_metrics_server = true
enable_external_dns = true
external_dns_route53_zone_arns = [var.my_hosted_zone_id]
helm_releases = {
istio-base = {
chart = "base"
chart_version = local.istio_chart_version
repository = local.istio_chart_url
name = "istio-base"
namespace = kubernetes_namespace_v1.istio_system.metadata[0].name
}
istiod = {
chart = "istiod"
chart_version = local.istio_chart_version
repository = local.istio_chart_url
name = "istiod"
namespace = kubernetes_namespace_v1.istio_system.metadata[0].name
set = [
{
name = "meshConfig.accessLogFile"
value = "/dev/stdout"
},
{
name = "meshConfig.enableGatewayAPI"
value = "true"
}
]
}
}
tags = local.tags
}
EOF- output.tf
output "configure_kubectl" {
description = "Configure kubectl: make sure you're logged in with the correct AWS profile and run the following command to update your kubeconfig"
value = "aws eks --region ${local.region} update-kubeconfig --name ${module.eks.cluster_name}"
}1.3 Terraform 코드 실행
# 환경변수 설정
export CLUSTER_NAME=istio-sejkim
export AWS_DEFAULT_PROFILE=istio # ~/.aws/credentials 에 등록된 profile 이름
# Terraform 초기화
terraform init
# VPC 생성
terraform apply -target='module.vpc' --auto-approve
# EKS Cluster v1.32 생성
terraform apply -target='module.eks' --auto-approve
# kubectl 실행을 위한 kubeconfig 인증서 생성
aws eks --region ap-northeast-2 update-kubeconfig --name $CLUSTER_NAME
(옵션, 이전 실습환경 기 존재 시)
kubectl config delete-context istio
kubectl config rename-context arn:aws:eks:ap-northeast-2:1**********3:cluster/istio-sejkim istio
Context "arn:aws:eks:ap-northeast-2:1**********3:cluster/istio-sejkim" renamed to "istio".
# istio-system 네임스페이스 생성
terraform apply -target='kubernetes_namespace_v1.istio_system' --auto-approve
# add-ons 설치(cni, coredns, kube-proxy, alb-ingress-controller, external-dns 등)
# & Istio 설치(istio-base, istiod)
terraform apply -target='module.eks_blueprints_addons' --auto-approve
# 최종 잔여 코드 실행
terraform apply --auto-approve1.4 Terraform 코드 실행결과
- Terraform state list
data.aws_availability_zones.available
data.aws_ecrpublic_authorization_token.token
data.aws_kms_key.by_key_arn
kubernetes_namespace_v1.istio_system
module.eks.data.aws_caller_identity.current[0]
module.eks.data.aws_iam_policy_document.assume_role_policy[0]
module.eks.data.aws_iam_policy_document.custom[0]
module.eks.data.aws_iam_session_context.current[0]
module.eks.data.aws_partition.current[0]
module.eks.data.tls_certificate.this[0]
module.eks.aws_cloudwatch_log_group.this[0]
module.eks.aws_ec2_tag.cluster_primary_security_group["Blueprint"]
module.eks.aws_ec2_tag.cluster_primary_security_group["GithubRepo"]
module.eks.aws_eks_access_entry.this["cluster_creator"]
module.eks.aws_eks_access_policy_association.this["cluster_creator_admin"]
module.eks.aws_eks_cluster.this[0]
module.eks.aws_iam_openid_connect_provider.oidc_provider[0]
module.eks.aws_iam_policy.cluster_encryption[0]
module.eks.aws_iam_policy.custom[0]
module.eks.aws_iam_role.this[0]
module.eks.aws_iam_role_policy_attachment.cluster_encryption[0]
module.eks.aws_iam_role_policy_attachment.custom[0]
module.eks.aws_iam_role_policy_attachment.this["AmazonEKSClusterPolicy"]
module.eks.aws_iam_role_policy_attachment.this["AmazonEKSVPCResourceController"]
module.eks.aws_security_group.cluster[0]
module.eks.aws_security_group.node[0]
module.eks.aws_security_group_rule.cluster["ingress_nodes_443"]
module.eks.aws_security_group_rule.node["egress_all"]
module.eks.aws_security_group_rule.node["ingress_15012"]
module.eks.aws_security_group_rule.node["ingress_15017"]
module.eks.aws_security_group_rule.node["ingress_cluster_443"]
module.eks.aws_security_group_rule.node["ingress_cluster_4443_webhook"]
module.eks.aws_security_group_rule.node["ingress_cluster_6443_webhook"]
module.eks.aws_security_group_rule.node["ingress_cluster_8443_webhook"]
module.eks.aws_security_group_rule.node["ingress_cluster_9443_webhook"]
module.eks.aws_security_group_rule.node["ingress_cluster_kubelet"]
module.eks.aws_security_group_rule.node["ingress_nodes_ephemeral"]
module.eks.aws_security_group_rule.node["ingress_self_coredns_tcp"]
module.eks.aws_security_group_rule.node["ingress_self_coredns_udp"]
module.eks.time_sleep.this[0]
module.eks_blueprints_addons.data.aws_caller_identity.current
module.eks_blueprints_addons.data.aws_eks_addon_version.this["coredns"]
module.eks_blueprints_addons.data.aws_eks_addon_version.this["kube-proxy"]
module.eks_blueprints_addons.data.aws_eks_addon_version.this["vpc-cni"]
module.eks_blueprints_addons.data.aws_iam_policy_document.aws_load_balancer_controller[0]
module.eks_blueprints_addons.data.aws_iam_policy_document.external_dns[0]
module.eks_blueprints_addons.data.aws_partition.current
module.eks_blueprints_addons.data.aws_region.current
module.eks_blueprints_addons.aws_cloudformation_stack.usage_telemetry[0]
module.eks_blueprints_addons.aws_eks_addon.this["coredns"]
module.eks_blueprints_addons.aws_eks_addon.this["kube-proxy"]
module.eks_blueprints_addons.aws_eks_addon.this["vpc-cni"]
module.eks_blueprints_addons.helm_release.this["istio-base"]
module.eks_blueprints_addons.helm_release.this["istiod"]
module.eks_blueprints_addons.random_bytes.this
module.eks_blueprints_addons.time_sleep.this
module.vpc.aws_default_network_acl.this[0]
module.vpc.aws_default_route_table.default[0]
module.vpc.aws_default_security_group.this[0]
module.vpc.aws_eip.nat[0]
module.vpc.aws_internet_gateway.this[0]
module.vpc.aws_nat_gateway.this[0]
module.vpc.aws_route.private_nat_gateway[0]
module.vpc.aws_route.public_internet_gateway[0]
module.vpc.aws_route_table.private[0]
module.vpc.aws_route_table.public[0]
module.vpc.aws_route_table_association.private[0]
module.vpc.aws_route_table_association.private[1]
module.vpc.aws_route_table_association.private[2]
module.vpc.aws_route_table_association.public[0]
module.vpc.aws_route_table_association.public[1]
module.vpc.aws_route_table_association.public[2]
module.vpc.aws_subnet.private[0]
module.vpc.aws_subnet.private[1]
module.vpc.aws_subnet.private[2]
module.vpc.aws_subnet.public[0]
module.vpc.aws_subnet.public[1]
module.vpc.aws_subnet.public[2]
module.vpc.aws_vpc.this[0]
module.eks.module.eks_managed_node_group["istio-sejkim-ng"].data.aws_caller_identity.current
module.eks.module.eks_managed_node_group["istio-sejkim-ng"].data.aws_iam_policy_document.assume_role_policy[0]
module.eks.module.eks_managed_node_group["istio-sejkim-ng"].data.aws_partition.current
module.eks.module.eks_managed_node_group["istio-sejkim-ng"].aws_eks_node_group.this[0]
module.eks.module.eks_managed_node_group["istio-sejkim-ng"].aws_iam_role.this[0]
module.eks.module.eks_managed_node_group["istio-sejkim-ng"].aws_iam_role_policy_attachment.this["AmazonEC2ContainerRegistryReadOnly"]
module.eks.module.eks_managed_node_group["istio-sejkim-ng"].aws_iam_role_policy_attachment.this["AmazonEKSWorkerNodePolicy"]
module.eks.module.eks_managed_node_group["istio-sejkim-ng"].aws_iam_role_policy_attachment.this["AmazonEKS_CNI_Policy"]
module.eks.module.eks_managed_node_group["istio-sejkim-ng"].aws_launch_template.this[0]
module.eks_blueprints_addons.module.aws_load_balancer_controller.data.aws_caller_identity.current[0]
module.eks_blueprints_addons.module.aws_load_balancer_controller.data.aws_iam_policy_document.assume[0]
module.eks_blueprints_addons.module.aws_load_balancer_controller.data.aws_iam_policy_document.this[0]
module.eks_blueprints_addons.module.aws_load_balancer_controller.data.aws_partition.current[0]
module.eks_blueprints_addons.module.aws_load_balancer_controller.aws_iam_policy.this[0]
module.eks_blueprints_addons.module.aws_load_balancer_controller.aws_iam_role.this[0]
module.eks_blueprints_addons.module.aws_load_balancer_controller.aws_iam_role_policy_attachment.this[0]
module.eks_blueprints_addons.module.aws_load_balancer_controller.helm_release.this[0]
module.eks_blueprints_addons.module.external_dns.data.aws_caller_identity.current[0]
module.eks_blueprints_addons.module.external_dns.data.aws_iam_policy_document.assume[0]
module.eks_blueprints_addons.module.external_dns.data.aws_iam_policy_document.this[0]
module.eks_blueprints_addons.module.external_dns.data.aws_partition.current[0]
module.eks_blueprints_addons.module.external_dns.aws_iam_policy.this[0]
module.eks_blueprints_addons.module.external_dns.aws_iam_role.this[0]
module.eks_blueprints_addons.module.external_dns.aws_iam_role_policy_attachment.this[0]
module.eks_blueprints_addons.module.external_dns.helm_release.this[0]
module.eks_blueprints_addons.module.metrics_server.helm_release.this[0]
module.eks.module.eks_managed_node_group["istio-sejkim-ng"].module.user_data.null_resource.validate_cluster_service_cidr-
VPC
-
IAM Role
-
Policy
-
EKS Cluster
-
K8s 정보 확인
# 노드
kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
ip-10-0-21-216.ap-northeast-2.compute.internal Ready <none> 7m21s v1.32.3-eks-473151a 10.0.21.216 <none> Amazon Linux 2023.7.20250414 6.1.132-147.221.amzn2023.x86_64 containerd://1.7.27
ip-10-0-32-39.ap-northeast-2.compute.internal Ready <none> 7m22s v1.32.3-eks-473151a 10.0.32.39 <none> Amazon Linux 2023.7.20250414 6.1.132-147.221.amzn2023.x86_64 containerd://1.7.27
ip-10-0-5-219.ap-northeast-2.compute.internal Ready <none> 7m20s v1.32.3-eks-473151a 10.0.5.219 <none> Amazon Linux 2023.7.20250414 6.1.132-147.221.amzn2023.x86_64 containerd://1.7.27
# 네임스페이스
kubectl get ns
NAME STATUS AGE
default Active 12m
istio-system Active 4s
kube-node-lease Active 12m
kube-public Active 12m
kube-system Active 12m- Helm 설치 목록
helm list -A
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
aws-load-balancer-controller kube-system 1 2025-05-02 23:53:45.559345 +0900 KST deployed aws-load-balancer-controller-1.7.1 v2.7.1
external-dns external-dns 1 2025-05-02 23:53:45.126132 +0900 KST deployed external-dns-1.14.3 0.14.0
istio-base istio-system 1 2025-05-02 23:54:40.089807 +0900 KST deployed base-1.25.2 1.25.2
istiod istio-system 1 2025-05-02 23:54:43.28031 +0900 KST deployed istiod-1.25.2 1.25.2
metrics-server kube-system 1 2025-05-02 23:53:13.337095 +0900 KST deployed metrics-server-3.12.0 0.7.0 - istio 설치목록
kubectl -n istio-system get deploy,pod,svc
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/istiod 1/1 1 1 3m41s
NAME READY STATUS RESTARTS AGE
pod/istiod-5945c7b655-cxlc8 1/1 Running 0 3m41s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/istiod ClusterIP 172.20.197.240 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 3m41s1.5 Sample Bookinfo 배포
# Bookinfo Namespace 생성
kubectl create namespace bookinfo
kubectl label namespace bookinfo istio-injection=enabled
kubectl -n bookinfo apply -f https://raw.githubusercontent.com/istio/istio/release-1.25/samples/bookinfo/platform/kube/bookinfo.yaml
kubectl -n bookinfo get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
details ClusterIP 172.20.117.176 <none> 9080/TCP 34s
productpage ClusterIP 172.20.27.64 <none> 9080/TCP 33s
ratings ClusterIP 172.20.173.190 <none> 9080/TCP 34s
reviews ClusterIP 172.20.135.194 <none> 9080/TCP 34s
kubectl -n bookinfo get pods
NAME READY STATUS RESTARTS AGE
details-v1-54ffdd5947-chchd 2/2 Running 0 66s
productpage-v1-d49bb79b4-m4n9g 2/2 Running 0 65s
ratings-v1-856f65bcff-ch592 2/2 Running 0 66s
reviews-v1-848b8749df-xdd95 2/2 Running 0 65s
reviews-v2-5fdf9886c7-c7hks 2/2 Running 0 65s
reviews-v3-bb6b8ddc7-8rx74 2/2 Running 0 65s1.6 Istio-Ingress-Gateway 배포
- istio-ingress-gateway.yaml
- annotations에 external-dns, internet-facing, acm 등 추가 설정
USER-SUPPLIED VALUES:
labels:
istio: ingressgateway
service:
annotations:
external-dns.alpha.kubernetes.io/hostname: ing-bookinfo.ksj7279.click
service.beta.kubernetes.io/aws-load-balancer-attributes: load_balancing.cross_zone.enabled=true
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:ap-northeast-2:1**********3:certificate/415404eb-e2e2-4744-b2e4-1108735b5903
service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-TLS-1-2-Ext-2018-06
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
service.beta.kubernetes.io/aws-load-balancer-type: external
ports:
- name: status-port
port: 15021
targetPort: 15021
- name: http2
port: 80
targetPort: 8080
- name: https
port: 443
targetPort: 8080
type: LoadBalancer
autoscaling:
minReplicas: 3 # 서비스 가용성을 위해 1->3으로 변경- helm 배포
# Istall instio-ingress-gateway with helm
helm repo add istio https://istio-release.storage.googleapis.com/charts
helm repo update
helm install istio-ingress-gateway istio/gateway -n istio-system -f istio-ingress-gateway.yaml --version 1.25.2
Update Complete. ⎈Happy Helming!⎈
NAME: istio-ingress-gateway
LAST DEPLOYED: Sat May 3 15:27:22 2025
NAMESPACE: istio-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
"istio-ingress-gateway" successfully installed!
To learn more about the release, try:
$ helm status istio-ingress-gateway -n istio-system
$ helm get all istio-ingress-gateway -n istio-system
# Wait for Istio to be ready
kubectl -n istio-system wait --for=condition=Ready pods --all --timeout=300s
# Check istio-ingress-gateway
kubectl -n istio-system get all
NAME READY STATUS RESTARTS AGE
pod/istio-ingress-gateway-5d8886464b-bhc67 1/1 Running 0 31s
pod/istio-ingress-gateway-5d8886464b-qrzdp 1/1 Running 0 46s
pod/istio-ingress-gateway-5d8886464b-sw97g 1/1 Running 0 30s
pod/istiod-5945c7b655-wf7gf 1/1 Running 0 3h59m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/istio-ingress-gateway LoadBalancer 172.20.181.39 k8s-istiosys-istioing-baa587af2c-add60fad882dcb93.elb.ap-northeast-2.amazonaws.com 15021:30467/TCP,80:31476/TCP,443:32485/TCP 46s
service/istiod ClusterIP 172.20.2.105 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 5h19m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/istio-ingress-gateway 3/3 3 3 46s
deployment.apps/istiod 1/1 1 1 5h19m
NAME DESIRED CURRENT READY AGE
replicaset.apps/istio-ingress-gateway-5d8886464b 3 3 3 46s
replicaset.apps/istiod-5945c7b655 1 1 1 5h19m
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
horizontalpodautoscaler.autoscaling/istio-ingress-gateway Deployment/istio-ingress-gateway cpu: <unknown>/80% 3 5 3 46s
horizontalpodautoscaler.autoscaling/istiod Deployment/istiod cpu: 2%/80% 1 5 1 5h19m- gateway-vs-bookinfo.yaml
cat << EOF > gateway-vs-bookinfo.yaml
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
name: bookinfo-gateway
spec:
# The selector matches the ingress gateway pod labels.
# If you installed Istio using Helm following the standard documentation, this would be "istio=ingress"
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 8080 🛠️ TargetGroup 통신 Port (80, 443)
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
name: bookinfo
spec:
hosts:
- "*"
gateways:
- bookinfo-gateway
http:
- match:
- uri:
exact: /productpage
- uri:
prefix: /static
- uri:
exact: /login
- uri:
exact: /logout
- uri:
prefix: /api/v1/products
route:
- destination:
host: productpage
port:
number: 9080
EOF
# gateway & virtual service 배포
kubectl -n bookinfo apply -f gateway-vs-bookinfo.yaml
gateway.networking.istio.io/bookinfo-gateway created
virtualservice.networking.istio.io/bookinfo created
# gateway & virtual service 확인
kubectl get gw,vs -n bookinfo
NAME AGE
gateway.networking.istio.io/bookinfo-gateway 15s
NAME GATEWAYS HOSTS AGE
virtualservice.networking.istio.io/bookinfo ["bookinfo-gateway"] ["*"] 15s- Load Balancer 상태 확인
# svc istio-ingress-gateway
kubectl -n istio-system get svc istio-ingress-gateway
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-ingress-gateway LoadBalancer 172.20.65.175 k8s-istiosys-istioing-9ae0140110-633135a822670363.elb.ap-northeast-2.amazonaws.com 15021:32239/TCP,80:32175/TCP,443:31932/TCP 102m
<# istio-ingress-gateway Service
kubectl neat get -- svc -n istio-system istio-ingress-gateway -o yaml
apiVersion: v1
kind: Service
metadata:
annotations:
external-dns.alpha.kubernetes.io/hostname: ing-bookinfo.ksj7279.click
meta.helm.sh/release-name: istio-ingress-gateway
meta.helm.sh/release-namespace: istio-system
service.beta.kubernetes.io/aws-load-balancer-attributes: load_balancing.cross_zone.enabled=true
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:ap-northeast-2:17**********3:certificate/415404eb-e2e2-4744-b2e4-1108735b5903
service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-TLS-1-2-Ext-2018-06
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
service.beta.kubernetes.io/aws-load-balancer-type: external
labels:
app: istio-ingress-gateway
app.kubernetes.io/instance: istio-ingress-gateway
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: istio-ingress-gateway
app.kubernetes.io/part-of: istio
app.kubernetes.io/version: 1.25.2
helm.sh/chart: gateway-1.25.2
istio: ingressgateway
istio.io/dataplane-mode: none
name: istio-ingress-gateway
namespace: istio-system
spec:
clusterIP: 172.20.65.175
clusterIPs:
- 172.20.65.175
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
loadBalancerClass: service.k8s.aws/nlb
ports:
- name: status-port
nodePort: 32239
port: 15021
- name: http2
nodePort: 32175
port: 80
targetPort: 8080 ⛔️ default 80, TargetGroup Unhealthy 발생해소 위해 변경한 값
- name: https
nodePort: 31932
port: 443
targetPort: 8080 ⛔️ default 443, TargetGroup Unhealthy 발생해소 위해 변경한 값
selector:
app: istio-ingress-gateway
istio: ingressgateway
type: LoadBalancer-
LoadBalancer 확인 내용
-
Listener 구성 (443/https 적용)
-
TargetGroup 정보
-
Route 53 Hosted Zone에 자동 추가 됨
-
bookinfo 페이지 https://개인도메인/productpage URL로 접속
-
(선택) istio-ingress-gateway minReplicas 3을 2으로 조정 시
kubectl patch hpa istio-ingress-gateway -n istio-system -p '{"spec": {"minReplicas": 2}}'
horizontalpodautoscaler.autoscaling/istio-ingress-gateway patched1.7 Istio-Ingress-Gateway 삭제
- gateway, virtualservice 내용 제거
# gateway & virtual service 삭제
kubectl -n bookinfo delete -f gateway-vs-bookinfo.yaml
gateway.networking.istio.io "bookinfo-gateway" deleted
virtualservice.networking.istio.io "bookinfo" deleted
# Helm으로 배포된 것 istio-ingress-gateway 삭제전 확인
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
istio-base istio-system 1 2025-05-03 10:03:08.615509 +0900 KST deployed base-1.25.2 1.25.2
istio-ingress-gateway istio-system 1 2025-05-03 15:27:22.23463 +0900 KST deployed gateway-1.25.2 1.25.2
istiod istio-system 2 2025-05-03 11:04:36.618247 +0900 KST deployed istiod-1.25.2 1.25.2
# istio-ingress-gateway 삭제
helm -n istio-system uninstall istio-ingress-gateway
release "istio-ingress-gateway" uninstalled
# 확인
kubectl -n istio get all
NAME READY STATUS RESTARTS AGE
pod/istiod-5945c7b655-wf7gf 1/1 Running 0 4h8m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/istiod ClusterIP 172.20.2.105 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 5h28m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/istiod 1/1 1 1 5h28m
NAME DESIRED CURRENT READY AGE
replicaset.apps/istiod-5945c7b655 1 1 1 5h28m
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
horizontalpodautoscaler.autoscaling/istiod Deployment/istiod cpu: 0%/80% 1 5 1 5h28m- Load Balancer 가 자동 삭제 됨
1.8 Kubernetes Gateway API 배포
- Gateway API CRDs 미설치된 경우 설치
kubectl get crd gateways.gateway.networking.k8s.io &> /dev/null || \
{ kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd?ref=v1.2.1" | kubectl apply -f -; }
customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/grpcroutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io created- Gateway, httpRoutes 배포
cat <<EOF > gateway-httproute-bookinfo.yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: istio-gateway
namespace: bookinfo
annotations:
external-dns.alpha.kubernetes.io/hostname: "gwa-bookinfo.ksj7279.click"
service.beta.kubernetes.io/aws-load-balancer-type: "external"
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"
service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: "ELBSecurityPolicy-TLS-1-2-Ext-2018-06"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:ap-northeast-2:1**********3:certificate/415404eb-e2e2-4744-b2e4-1108735b5903"
spec:
gatewayClassName: istio
listeners:
- name: http
port: 80
protocol: HTTP
allowedRoutes:
namespaces:
from: Same
- name: https
port: 443
protocol: HTTP
allowedRoutes:
namespaces:
from: Same
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: bookinfo
namespace: bookinfo
spec:
parentRefs:
- name: istio-gateway
hostnames:
- "gwa-bookinfo.ksj7279.click"
rules:
- matches:
- path:
type: Exact
value: /productpage
- path:
type: PathPrefix
value: /static
- path:
type: Exact
value: /login
- path:
type: Exact
value: /logout
- path:
type: PathPrefix
value: /api/v1/products
backendRefs:
- name: productpage
port: 9080
EOF
kubectl -n bookinfo apply -f gateway-httproute-bookinfo.yaml
gateway.gateway.networking.k8s.io/istio-gateway created
httproute.gateway.networking.k8s.io/bookinfo created
kubectl -n bookinfo get gtw,httproute
NAME CLASS ADDRESS PROGRAMMED AGE
gateway.gateway.networking.k8s.io/istio-gateway istio k8s-bookinfo-istiogat-986e4027bc-77bff472515b16e7.elb.ap-northeast-2.amazonaws.com True 75s
NAME HOSTNAMES AGE
httproute.gateway.networking.k8s.io/bookinfo ["gwa-bookinfo.ksj7279.click"] 75s
kubectl -n bookinfo get svc istio-gateway-istio
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-gateway-istio LoadBalancer 172.20.114.134 k8s-bookinfo-istiogat-986e4027bc-77bff472515b16e7.elb.ap-northeast-2.amazonaws.com 15021:30367/TCP,80:30882/TCP,443:30697/TCP 2m54s- kubernetes gateway pod 상세정보
kubectl neat get -- deploy istio-gateway-istio -n bookinfo -o yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
external-dns.alpha.kubernetes.io/hostname: gwa-bookinfo.ksj7279.click
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:ap-northeast-2:1**********3:certificate/415404eb-e2e2-4744-b2e4-1108735b5903
service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-TLS-1-2-Ext-2018-06
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
service.beta.kubernetes.io/aws-load-balancer-type: external
labels:
gateway.istio.io/managed: istio.io-gateway-controller
gateway.networking.k8s.io/gateway-name: istio-gateway
name: istio-gateway-istio
namespace: bookinfo
spec:
progressDeadlineSeconds: 600
replicas: 1 🏥 안정성을 위해 2 이상으로 수정
revisionHistoryLimit: 10
selector:
matchLabels:
gateway.networking.k8s.io/gateway-name: istio-gateway
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
annotations:
external-dns.alpha.kubernetes.io/hostname: gwa-bookinfo.ksj7279.click
istio.io/rev: default
prometheus.io/path: /stats/prometheus
prometheus.io/port: "15020"
prometheus.io/scrape: "true"
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:ap-northeast-2:1**********3:certificate/415404eb-e2e2-4744-b2e4-1108735b5903
service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-TLS-1-2-Ext-2018-06
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
service.beta.kubernetes.io/aws-load-balancer-type: external
creationTimestamp: null
labels:
gateway.istio.io/managed: istio.io-gateway-controller
gateway.networking.k8s.io/gateway-name: istio-gateway
service.istio.io/canonical-name: istio-gateway-istio
service.istio.io/canonical-revision: latest
sidecar.istio.io/inject: "false"
spec:
containers:
- args:
- proxy
- router
- --domain
- $(POD_NAMESPACE).svc.cluster.local
- --proxyLogLevel
- warning
- --proxyComponentLogLevel
- misc:error
- --log_output_level
- default:info
env:
- name: PILOT_CERT_PROVIDER
value: istiod
- name: CA_ADDR
value: istiod.istio-system.svc:15012
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: INSTANCE_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
- name: SERVICE_ACCOUNT
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.serviceAccountName
- name: HOST_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.hostIP
- name: ISTIO_CPU_LIMIT
valueFrom:
resourceFieldRef:
divisor: "0"
resource: limits.cpu
- name: PROXY_CONFIG
value: |
{}
- name: ISTIO_META_POD_PORTS
value: '[]'
- name: ISTIO_META_APP_CONTAINERS
- name: GOMEMLIMIT
valueFrom:
resourceFieldRef:
divisor: "0"
resource: limits.memory
- name: GOMAXPROCS
valueFrom:
resourceFieldRef:
divisor: "0"
resource: limits.cpu
- name: ISTIO_META_CLUSTER_ID
value: Kubernetes
- name: ISTIO_META_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: ISTIO_META_INTERCEPTION_MODE
value: REDIRECT
- name: ISTIO_META_WORKLOAD_NAME
value: istio-gateway-istio
- name: ISTIO_META_OWNER
value: kubernetes://apis/apps/v1/namespaces/bookinfo/deployments/istio-gateway-istio
- name: ISTIO_META_MESH_ID
value: cluster.local
- name: TRUST_DOMAIN
value: cluster.local
image: docker.io/istio/proxyv2:1.25.2
imagePullPolicy: IfNotPresent
name: istio-proxy
ports:
- containerPort: 15020
name: metrics
protocol: TCP
- containerPort: 15021
name: status-port
protocol: TCP
- containerPort: 15090
name: http-envoy-prom
protocol: TCP
readinessProbe:
failureThreshold: 4
httpGet:
path: /healthz/ready
port: 15021
scheme: HTTP
periodSeconds: 15
successThreshold: 1
timeoutSeconds: 1
resources:
limits:
cpu: "2"
memory: 1Gi
requests:
cpu: 100m
memory: 128Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
privileged: false
readOnlyRootFilesystem: true
runAsGroup: 1337
runAsNonRoot: true
runAsUser: 1337
startupProbe:
failureThreshold: 30
httpGet:
path: /healthz/ready
port: 15021
scheme: HTTP
initialDelaySeconds: 1
periodSeconds: 1
successThreshold: 1
timeoutSeconds: 1
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/workload-spiffe-uds
name: workload-socket
- mountPath: /var/run/secrets/credential-uds
name: credential-socket
- mountPath: /var/run/secrets/workload-spiffe-credentials
name: workload-certs
- mountPath: /var/run/secrets/istio
name: istiod-ca-cert
- mountPath: /var/lib/istio/data
name: istio-data
- mountPath: /etc/istio/proxy
name: istio-envoy
- mountPath: /var/run/secrets/tokens
name: istio-token
- mountPath: /etc/istio/pod
name: istio-podinfo
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
sysctls:
- name: net.ipv4.ip_unprivileged_port_start
value: "0"
serviceAccount: istio-gateway-istio
serviceAccountName: istio-gateway-istio
terminationGracePeriodSeconds: 30
volumes:
- name: workload-socket
- name: credential-socket
- name: workload-certs
- emptyDir:
medium: Memory
name: istio-envoy
- name: istio-data
- downwardAPI:
defaultMode: 420
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.labels
path: labels
- fieldRef:
apiVersion: v1
fieldPath: metadata.annotations
path: annotations
name: istio-podinfo
- name: istio-token
projected:
defaultMode: 420
sources:
- serviceAccountToken:
audience: istio-ca
expirationSeconds: 43200
path: istio-token
- configMap:
defaultMode: 420
name: istio-ca-root-cert
name: istiod-ca-cert-
LoadBalancer
-
Listenr 구성
-
TargetGroup
-
External DNS에 의해 Rout 53 HostedZone에 자동 추가 됨
-
gwa-bookinfo.ksj7279.click/productpage 로 접속
2. Observability
2.1 Add-ons 배포
- Observability Add-ons 배포
for ADDON in kiali jaeger prometheus grafana
do
ADDON_URL="https://raw.githubusercontent.com/istio/istio/release-1.20/samples/addons/$ADDON.yaml"
kubectl apply --server-side -f $ADDON_URL
done- 확인
kubectl get pods,svc -n istio-system
NAME READY STATUS RESTARTS AGE
pod/grafana-84f968c4c9-ks76r 1/1 Running 0 30m
pod/istiod-5945c7b655-wf7gf 1/1 Running 0 7h23m
pod/jaeger-69c844bd48-2nnwz 1/1 Running 0 30m
pod/kiali-fc474f545-gqzgl 1/1 Running 0 30m
pod/prometheus-6798d6559c-q6hhw 2/2 Running 0 30m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/grafana ClusterIP 172.20.15.216 <none> 3000/TCP 30m
service/istiod ClusterIP 172.20.2.105 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 8h
service/jaeger-collector ClusterIP 172.20.13.164 <none> 14268/TCP,14250/TCP,9411/TCP,4317/TCP,4318/TCP 30m
service/kiali ClusterIP 172.20.2.123 <none> 20001/TCP,9090/TCP 30m
service/prometheus ClusterIP 172.20.84.38 <none> 9090/TCP 30m
service/tracing ClusterIP 172.20.62.245 <none> 80/TCP,16685/TCP 30m
service/zipkin ClusterIP 172.20.46.101 <none> 9411/TCP 30m2.2 port-forward 방식으로 접속
# Visualize Istio Mesh console using Kiali
kubectl port-forward svc/kiali 20001:20001 -n istio-system
# Get to the Prometheus UI
kubectl port-forward svc/prometheus 9090:9090 -n istio-system
# Visualize metrics in using Grafana
kubectl port-forward svc/grafana 3000:3000 -n istio-system
# Visualize application traces via Jaeger
kubectl port-forward svc/jaeger 16686:16686 -n istio-system2.3 kubernetes gateway api로 add-ons 접속
- gateway, httproute manifest 생성
cat << EOF > gateway-httproute-observability.yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: observability-gateway
namespace: istio-system
annotations:
external-dns.alpha.kubernetes.io/hostname: "observability.ksj7279.click"
service.beta.kubernetes.io/aws-load-balancer-type: "external"
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"
service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: "ELBSecurityPolicy-TLS-1-2-Ext-2018-06"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:ap-northeast-2:170698194833:certificate/415404eb-e2e2-4744-b2e4-1108735b5903"
spec:
gatewayClassName: istio
listeners:
- name: http
port: 80
protocol: HTTP
allowedRoutes:
namespaces:
from: All
- name: https
port: 443
protocol: HTTP
allowedRoutes:
namespaces:
from: All
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: kiali-route
namespace: istio-system
spec:
parentRefs:
- name: observability-gateway
hostnames:
- "observability.ksj7279.click"
rules:
- matches:
- path:
type: PathPrefix
value: /kiali
backendRefs:
- name: kiali
port: 20001
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: prometheus-route
namespace: istio-system
spec:
parentRefs:
- name: observability-gateway
hostnames:
- "observability.ksj7279.click"
rules:
- matches:
- path:
type: PathPrefix
value: /prometheus
filters:
- type: URLRewrite
urlRewrite:
path:
type: ReplacePrefixMatch
replacePrefixMatch: /
backendRefs:
- name: prometheus
port: 9090
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: grafana-route
namespace: istio-system
spec:
parentRefs:
- name: observability-gateway
hostnames:
- "observability.ksj7279.click"
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: grafana
port: 3000
EOF- gateway-httproute-observability.yaml 배포
kubectl apply -f gateway-httproute-observability.yaml
gateway.gateway.networking.k8s.io/observability-gateway created
httproute.gateway.networking.k8s.io/kiali-route created
httproute.gateway.networking.k8s.io/prometheus-route created
httproute.gateway.networking.k8s.io/grafana-route created- 배포 확인
# gateway, httproute 확인
kubectl get gtw,httproute -n istio-system
NAME CLASS ADDRESS PROGRAMMED AGE
gateway.gateway.networking.k8s.io/observability-gateway istio k8s-istiosys-observab-7e12038dc2-b29ed8ea31dab7d4.elb.ap-northeast-2.amazonaws.com True 24m
NAME HOSTNAMES AGE
httproute.gateway.networking.k8s.io/grafana-route ["observability.ksj7279.click"] 24m
httproute.gateway.networking.k8s.io/kiali-route ["observability.ksj7279.click"] 24m
httproute.gateway.networking.k8s.io/prometheus-route ["observability.ksj7279.click"] 24m
# deployment, service 확인
kubectl get deployment,svc -n istio-system
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/grafana 1/1 1 1 33m
deployment.apps/istiod 1/1 1 1 87m
deployment.apps/jaeger 1/1 1 1 33m
deployment.apps/kiali 1/1 1 1 33m
deployment.apps/observability-gateway-istio 1/1 1 1 25m
deployment.apps/prometheus 1/1 1 1 33m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/grafana ClusterIP 172.20.182.153 <none> 3000/TCP 33m
service/istiod ClusterIP 172.20.80.47 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 87m
service/jaeger-collector ClusterIP 172.20.131.134 <none> 14268/TCP,14250/TCP,9411/TCP,4317/TCP,4318/TCP 33m
service/kiali ClusterIP 172.20.93.32 <none> 20001/TCP,9090/TCP 33m
service/observability-gateway-istio LoadBalancer 172.20.37.225 k8s-istiosys-observab-7e12038dc2-b29ed8ea31dab7d4.elb.ap-northeast-2.amazonaws.com 15021:30122/TCP,80:32021/TCP,443:31146/TCP 25m
service/prometheus ClusterIP 172.20.68.60 <none> 9090/TCP 33m
service/tracing ClusterIP 172.20.2.223 <none> 80/TCP,16685/TCP 33m
service/zipkin ClusterIP 172.20.229.192 <none> 9411/TCP 33m- NLB 확인
- 도메인 질의
nslookup observability.ksj7279.click
Server: 1.1.1.1
Address: 1.1.1.1#53
Non-authoritative answer:
Name: observability.ksj7279.click
Address: 3.37.77.5
Name: observability.ksj7279.click
Address: 43.202.209.178
Name: observability.ksj7279.click
Address: 13.124.0.185-
Kiali 접속 (https://observability.ksj7279.click/kiali)
-
Prometheus 접속 (https://observability.ksj7279.click/prometheus/graph)
-
Grafana 접속 (https://observability.ksj7279.click)
3. 실습자원 삭제
# Kiali용 Gateway, httproute 삭제
kubectl delete -f gateway-httproute-observability.yaml
# Observaility Add-ons 삭제
for ADDON in kiali jaeger prometheus grafana
do
ADDON_URL="https://raw.githubusercontent.com/istio/istio/release-1.20/samples/addons/$ADDON.yaml"
kubectl delete -f $ADDON_URL
done
# bookinfo 삭제
kubectl -n bookinfo delete -f gateway-httproute-bookinfo.yaml
kubectl delete ns bookinfo
# add-ons 삭제
terraform destroy -target='module.eks_blueprints_addons' --auto-approve
# istio-system 네임스페이스 자원 삭제
terraform destroy -target='kubernetes_namespace_v1.istio_system' --auto-approve
# eks cluster 삭제
terraform destroy -target='module.eks' --auto-approve
# vpc 삭제
terraform destroy -target='module.vpc' --auto-approve
# 최종 잔여자원 삭제
terraform destroy --auto-approve
# Route 53 HostedZone Record는 수동 삭제4. Istio Gateway API
🔄 Istio가 Gateway API를 권장하는 이유
Istio는 이전까지 Gateway, VirtualService, DestinationRule 같은 Istio CRD(Custom Resource Definition)를 사용하여 트래픽을 제어해왔습니다. 하지만 최근에는 Kubernetes Gateway API를 기본 인터페이스로 삼는 방향으로 나아가고 있습니다.
🧭 Gateway API란?
Gateway API는 Kubernetes SIG-NETWORK에서 표준화 중인 API로, Ingress보다 더 유연하고 확장 가능한 L4~L7 트래픽 관리 API입니다.
주요 리소스들:
GatewayClass: Gateway 구현체(예: Istio)를 정의.Gateway: 네트워크 진입점, 실제 LoadBalancer/NodePort와 연동됨.HTTPRoute,TCPRoute등: 트래픽 라우팅 규칙.ReferencePolicy: cross-namespace 리소스 접근 허용 정책.
💡 Istio에서 Gateway API를 사용하는 장점
| 기존 방식 (VirtualService 등) | Gateway API 방식 |
|---|---|
| Istio 전용 리소스 | Kubernetes 표준화된 리소스 |
| 제한된 재사용성 및 가시성 | 더 좋은 추상화 및 다중 팀 관리 |
| 관리를 위한 Istio 생태계에 종속 | 다양한 컨트롤러와 호환 가능 (e.g., Istio, Gloo, Kong) |
| 학습 곡선 큼 | 더 직관적인 리소스 설계 |
Istio는 1.20 이후부터 Gateway API를 정식 지원(stable)하고 있으며, 앞으로 VirtualService는 deprecated 예정입니다.
🛠️ Istio + Gateway API 구성 예시
1. GatewayClass (Istio에서 자동 생성됨)
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: istio
spec:
controllerName: istio.io/gateway-controller2. Gateway
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: my-gateway
namespace: my-namespace
spec:
gatewayClassName: istio
listeners:
- name: http
port: 80
protocol: HTTP
hostname: "*.example.com"3. HTTPRoute
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: my-route
namespace: my-namespace
spec:
parentRefs:
- name: my-gateway
rules:
- matches:
- path:
type: PathPrefix
value: /app
backendRefs:
- name: my-service
port: 8080🔐 주의할 점
ReferenceGrant를 설정하지 않으면 cross-namespace backend 참조가 안 됩니다.- Istio가 Gateway API를 제대로 인식하려면
istio-ingressgateway에 해당 기능이 활성화되어 있어야 합니다 (--set meshConfig.enableGatewayAPI=true).
📌 결론
Istio에서 Gateway API를 사용하는 건 표준 기반의 서비스 노출 방식으로 진화하는 것이며,
앞으로는 이것이 권장 경로가 될 것입니다.
기존 VirtualService/Ingress 등을 쓰고 있다면, 점진적으로 Gateway API로 전환하는 것을 고려하는 것이 좋습니다.
5. Kubernetes Gateway API vs AWS Gateway API Controller (with VPC Lattice)
🧭 개요 요약
| 항목 | Kubernetes Gateway API | AWS Gateway API Controller for EKS |
|---|---|---|
| 목적 | 쿠버네티스 표준 L4/L7 라우팅 API | AWS에서 Gateway API를 ALB로 구현 |
| 위치 | 클러스터 내부, 벤더 중립 | EKS 전용, AWS 리소스 tightly coupled |
| 컨트롤러 | Istio, Contour, Gloo 등 필요 | AWS 전용 컨트롤러 (gateway-api-controller) |
| 백엔드 연결 | K8s 서비스 | K8s 서비스, Lambda, IP 등 (Lattice 가능) |
🧱 리소스 비교
| 리소스 | 공통점 | AWS 확장 |
|---|---|---|
GatewayClass, Gateway | 기본 API 동일 | AWS는 GatewayClass → alb 사용 |
HTTPRoute, TCPRoute | 동일 | Lattice 및 ALB 대상그룹 연동 지원 |
보안 리소스 (ReferenceGrant, Policy) | 일부 수동 구성 필요 | IAM, SG, ACM으로 자동 연계 가능 |
🧩 트래픽 흐름 구조
Kubernetes Gateway API
Internet
↓
[LoadBalancer Service]
↓
[Gateway Pod (Envoy/Contour)]
↓
[HTTPRoute → K8s Service]AWS Gateway API Controller for EKS
Internet
↓
[AWS ALB (via GatewayClass=alb)]
↓
[HTTPRoute]
↓
[K8s Service or Lattice target]🔐 보안 및 IAM 통합
| 항목 | Kubernetes Gateway API | AWS Gateway API Controller |
|---|---|---|
| 인증서 관리 | cert-manager 필요 | ACM 연동 자동 |
| 인증/인가 | 외부 AuthProvider 연동 필요 (예: OIDC) | IAM 정책, 보안 그룹, TLS 자동 |
| 통합 보안 모델 | 자체 구현 필요 | AWS IAM + SG + Lattice 통합 가능 |
🔭 관측 및 모니터링
| 항목 | Kubernetes Gateway API | AWS Gateway API Controller |
|---|---|---|
| 로그 | Envoy / Fluentbit 등 수동 구성 | CloudWatch 로그 자동 수집 |
| 지표 | Prometheus, OpenTelemetry | AWS CloudWatch Metrics |
| 트레이싱 | OpenTelemetry 통합 | X-Ray 또는 CloudWatch Traces |
🧠 선택 기준
| 요구사항 | 추천 솔루션 |
|---|---|
| 멀티 클라우드, 벤더 중립 네트워크 구성 | ✅ Kubernetes Gateway API |
| EKS에서 ALB 기반 API Gateway를 간편하게 설정 | ✅ AWS Gateway API Controller |
| IAM, TLS, CloudWatch, VPC와 깊은 통합 필요 | ✅ AWS Gateway API Controller |
| 서비스 간 통신을 포함한 통합 네트워킹 구성 | ✅ VPC Lattice 연동 활용 가능 (with Gateway API) |
💡 VPC Lattice와의 관계
AWS Gateway API Controller는 VPC Lattice와 직접 통합 가능하며, HTTPRoute의 backendRef로 Lattice 서비스도 지정할 수 있습니다.
- 즉,
HTTPRoute → backendRef → Lattice service구성으로 EKS와 VPC Lattice 연동이 가능해집니다. - 이는 "Gateway API를 통한 Lattice 기반 서비스 통합"이라는 새로운 아키텍처 스타일을 가능하게 합니다.
📌 요약
| 항목 | Kubernetes Gateway API | AWS Gateway API Controller |
|---|---|---|
| 벤더 종속성 | ❌ 없음 | ✅ AWS 전용 |
| 클러스터 외부 통합 | 수동 (Istio 등 필요) | 자동 (ALB, Lattice, IAM) |
| 보안/운영 자동화 | 별도 구성 필요 | AWS 기반 자동화 우수 |
| 확장성 | 멀티 클라우드 적합 | AWS 기반 서비스 통합에 최적 |
I'm SJ
k8s gateway api 와 AWS Gateway api controller 의 차이점에 대해 일목요연하게 내용을 잘 정리해 주셔서 많은 도움이 되었습니다.