Zabbix helm chart 설치 및 구성

cloud2000·2024년 5월 18일
kubernetesobservabliity
1
post-thumbnail

Zabbix is a mature and effortless enterprise-class open source monitoring solution for network monitoring and application monitoring of millions of metrics

1. cluster 생성

  • kubekey로 우선 control-plane node 1개와 worker node 2개로 구성된 cluster를 생성한다.
  • kubesphere는 kubesphere-system, kubesphere-controls-system, kubesphere-monitoring-system로 최소한으로 설치한다.
$ vi user-cluster.yaml
apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Cluster
metadata:
  name: user-cluster
spec:
  hosts:
  - {name: node-63, address: 192.168.0.63, user: root, privateKeyPath: "/home/vagrant/infra/id_rsa"}
  - {name: node-64, address: 192.168.0.64, user: root, privateKeyPath: "/home/vagrant/infra/id_rsa"}
  - {name: node-65, address: 192.168.0.65, user: root, privateKeyPath: "/home/vagrant/infra/id_rsa"}
  - {name: node-52, address: 192.168.0.52, user: root, privateKeyPath: "/home/vagrant/infra/id_rsa"}
  roleGroups:
    etcd:
    - node-63
    control-plane:
    - node-63
    worker:
    - node-64
    - node-65
    registry:
    - node-52
  controlPlaneEndpoint:
    domain: lb.kubesphere.local
    address: ""
    port: 6443
  kubernetes:
    version: v1.29.3
    imageRepo: kubesphere
    containerManager: containerd
    clusterName: cluster.local
    autoRenewCerts:
    masqueradeAll: false
    maxPods: 110
    podPidsLimit: 10000
    nodeCidrMaskSize: 24
    proxyMode: ipvs
    featureGates:
      RotateKubeletServerCertificate: true
  etcd:
    type: kubekey
  network:
    plugin: calico
    kubePodsCIDR: 10.233.64.0/18
    kubeServiceCIDR: 10.233.0.0/18
    multusCNI:
      enabled: false
  registry:
    type: harbor
    plainHTTP: false
    privateRegistry: ""
    namespaceOverride: ""
    registryMirrors: []
    insecureRegistries: []
    auths:
  addons: []
  
---
apiVersion: installer.kubesphere.io/v1alpha1
kind: ClusterConfiguration
metadata:
  name: ks-installer
  namespace: kubesphere-system
  labels:
    version: v3.4.1
spec:
  persistence:
    storageClass: ""
  authentication:
    jwtSecret: ""
  zone: ""
  local_registry: ""
  namespace_override: ""
  etcd:
    monitoring: true
    endpointIps: 192.168.0.63
    port: 2379
    tlsEnable: true
  common:
    core:
      console:
        enableMultiLogin: true
        port: 30000
        type: NodePort
    redis:
      enabled: true
      volumeSize: 2Gi
    openldap:
      enabled: true
      volumeSize: 2Gi
    minio:
      volumeSize: 20Gi
    monitoring:
      endpoint: http://prometheus-operated.kubesphere-monitoring-system.svc:9090
      GPUMonitoring:
        enabled: false
    gpu:
      kinds:
      - resourceName: "nvidia.com/gpu"
        resourceType: "GPU"
        default: true
    es:
      enabled: false
      logMaxAge: 7
      elkPrefix: logstash
      basicAuth:
        enabled: false
        username: ""
        password: ""
      externalElasticsearchHost: ""
      externalElasticsearchPort: ""
  alerting:
    enabled: false
  auditing:
    enabled: false
  devops:
    enabled: false
  events:
    enabled: false
  logging:
    enabled: false
  metrics_server:
    enabled: true
  monitoring:
    enabled: false
    storageClass: ""
  multicluster:
    clusterRole: none
  network:
    networkpolicy:
      enabled: false
    ippool:
      type: none
    topology:
      type: none
  openpitrix:
    store:
      enabled: false
  servicemesh:
    enabled: false
  kubeedge:
    enabled: false
    
$ ./kk-v3.1.1 create cluster -f user-cluster.yaml

2. zabbix 설치

  • Data 수집 방식: Active, Passive mode, Proxy 서버 사용 예

  • chart value.yaml 파일 수정

global:
  imagePullSecrets: []
  commonLabels: {}

zabbixImageTag: ubuntu-6.0.20

postgresAccess:
  useUnifiedSecret: true
  unifiedSecretName: zabbixdb-pguser-zabbix
  unifiedSecretAutoCreate: true
  host: "zabbix-postgresql"
  port: "5432"
  user: "zabbix"
  password: "zabbix"
  database: "zabbix"

zabbixServer:
  enabled: true
  replicaCount: 1
  podAntiAffinity: true
  hostPort: false
  hostIP: 0.0.0.0
  resources: {}
  image:
    repository: zabbix/zabbix-server-pgsql
    tag: null
    pullPolicy: IfNotPresent
    pullSecrets: []
  haNodesAutoClean:
    enabled: true
    image:
      repository: postgres
      tag: 15
      pullPolicy: IfNotPresent
      pullSecrets: []
    schedule: "0 1 * * *"
    concurrencyPolicy: "Replace"
    deleteOlderThanSeconds: 3600
    resources: {}
    extraEnv: []
    extraVolumeMounts: []
    extraContainers: []
    extraInitContainers: []
    extraVolumes: []
    extraPodSpecs: {}
    securityContext: {}
    cronjobLabels: {}
  jobDBSchema:
    jobAnnotations: {}
    jobLabels: {}
    extraInitContainers: []
    extraContainers: []
    extraPodSpecs: {}
    extraVolumeMounts: []
    extraVolumes: []
    securityContext: {}
  service:
    type: ClusterIP
    externalIPs: []
    loadBalancerIP: ""
    clusterIP:
    port: 10051
    nodePort: 31051
    annotations: {}
  extraEnv: []
  deploymentAnnotations: {}
  deploymentLabels: {}
  containerAnnotations: {}
  containerLabels: {}
  extraVolumeMounts: []
  extraContainers: []
  extraInitContainers: []
  extraVolumes: []
  extraPodSpecs: {}
  securityContext: {}
  livenessProbe: {}
  readinessProbe: {}
  startupProbe: {}

postgresql:
  enabled: true
  image:
    repository: postgres
    tag: 15
    pullPolicy: IfNotPresent
    pullSecrets: []
  resources: {}
  persistence:
    enabled: false
    existingClaimName: false
    storageSize: 5Gi
  service:
    type: ClusterIP
    clusterIP:
    port: 5432
    annotations: {}
  extraRuntimeParameters:
    max_connections: 50
  extraEnv: []
  statefulSetAnnotations: {}
  statefulSetLabels: {}
  containerAnnotations: {}
  containerLabels: {}
  extraVolumeMounts: []
  extraContainers: []
  extraInitContainers: []
  extraVolumes: []
  extraPodSpecs: {}
  securityContext: {}
  livenessProbe: {}
  readinessProbe: {}
  startupProbe: {}

zabbixProxy:
  enabled: false
  replicaCount: 1
  resources: {}
  image:
    repository: zabbix/zabbix-proxy-sqlite3
    tag: null
    pullPolicy: IfNotPresent
    pullSecrets: []
  ZBX_PROXYMODE: 0
  ZBX_HOSTNAME: zabbix-proxy # This variable is unique, case sensitive hostname.
  ZBX_SERVER_HOST: zabbix-zabbix-server
  ZBX_SERVER_PORT: 10051
  ZBX_DEBUGLEVEL: 4
  ZBX_TIMEOUT: 4
  ZBX_JAVAGATEWAY_ENABLE: false
  ZBX_VMWARECACHESIZE: 128M
  service:
    type: ClusterIP
    clusterIP:
    port: 10051
    annotations: {}
  extraEnv: []
  statefulSetAnnotations: {}
  statefulSetLabels: {}
  containerAnnotations: {}
  containerLabels: {}
  extraVolumeMounts: []
  extraContainers: []
  extraInitContainers: []
  extraVolumes: []
  extraPodSpecs: {}
  securityContext: {}
  extraVolumeClaimTemplate: []
  livenessProbe: {}
  readinessProbe: {}
  startupProbe: {}

zabbixAgent:
  enabled: true				// 활성화 여부
  runAsSidecar: false		// sidecar로 기동 여부
  runAsDaemonSet: true		// daemonset으로 기동
  resources: {}
  image:
    repository: zabbix/zabbix-agent2
    tag: null
    pullPolicy: IfNotPresent
    pullSecrets: []
  ZBX_SERVER_HOST: 0.0.0.0/0			# Agent listen ip
  ZBX_SERVER_PORT: 10051				# Agent listen port
  ZBX_PASSIVE_ALLOW: true				# ZabbixServer -> Agent로 접속하여 데이터 수집 방식 활성
  ZBX_PASSIVESERVERS: 192.168.0.0/24	# Agent로 접속허용 할 IP, CIDR 대역. 예: Server=127.0.0.1,192.168.1.0/24,::1,2001:db8::/32,zabbix.example.com
  ZBX_ACTIVE_ALLOW: false				# Agent -> ZabbixServer로 접속하여 데이터 수집 방식 비활성
  ZBX_DEBUGLEVEL: 3
  ZBX_TIMEOUT: 4
  service:
    type: ClusterIP
    clusterIP:
    port: 10050
    #externalTrafficPolicy: Local
    listenOnAllInterfaces: true
    #nodePort: 31050
    annotations: {}
  hostRootFsMount: true
  extraEnv: []
  # Active check mode일 경우 ZBX_STARTAGENTS = 0 으로 설정함
  # extraEnv:
  #   - name: ZBX_STARTAGENTS
  #     value: '0'
  extraVolumeMounts: []
  daemonSetAnnotations: {}
  daemonSetLabels: {}
  deploymentLabels: {}
  containerAnnotations: {}
  containerLabels: {}
  extraContainers: []
  extraInitContainers: []
  extraVolumes: []
  extraPodSpecs: # {}
    hostNetwork: true
    
    # hostNetwork true일 경우 k8s cluster DNS를 사용할 경우에는 아래와 같이 설정함.
    # dnsPolicy: "None"
    # dnsConfig:
      # nameservers:
        # - 10.233.0.3  # This should be the IP of your cluster's DNS service (kube-dns or core-dns).
      # searches:
        # - svc.cluster.local  # This is the default search domain.  
  securityContext: {}
  
  # Active check mode일 경우 livenessProbe, startupProbe을 제거해야 하나 반영이 안됨.
  livenessProbe:
    tcpSocket:
      port: zabbix-agent
    timeoutSeconds: 3
    failureThreshold: 3
    periodSeconds: 10
    successThreshold: 1
  readinessProbe: {}
  startupProbe:
    tcpSocket:
      port: zabbix-agent
    initialDelaySeconds: 15
    periodSeconds: 5
    timeoutSeconds: 3
    failureThreshold: 5
    successThreshold: 1

zabbixWeb:
  enabled: true
  replicaCount: 1
  podAntiAffinity: true
  resources: {}
  image:
    repository: zabbix/zabbix-web-nginx-pgsql
    tag: null
    pullPolicy: IfNotPresent
    pullSecrets: []
  #samlCertsSecretName: zabbix-web-samlcerts
  service:
    type: NodePort
    externalIPs: []
    loadBalancerIP: ""
    clusterIP:
    port: 80
    #nodePort: 31080
    annotations: {}
  extraEnv: []
  deploymentAnnotations: {}
  deploymentLabels: {}
  containerAnnotations: {}
  containerLabels: {}
  extraVolumeMounts: []
  extraContainers: []
  extraInitContainers: []
  extraVolumes: []
  extraPodSpecs: {}
  securityContext: {}
  livenessProbe:
    httpGet:
      path: /
      port: zabbix-web
    initialDelaySeconds: 30
    periodSeconds: 10
    timeoutSeconds: 5
    failureThreshold: 6
    successThreshold: 1
  readinessProbe:
    httpGet:
      path: /
      port: zabbix-web
    initialDelaySeconds: 5
    periodSeconds: 10
    timeoutSeconds: 5
    failureThreshold: 6
    successThreshold: 1
  startupProbe: {}

zabbixWebService:
  enabled: true
  replicaCount: 1
  podAntiAffinity: true
  resources: {}
  image:
    repository: zabbix/zabbix-web-service
    tag: null
    pullPolicy: IfNotPresent
    pullSecrets: []
  service:
    type: ClusterIP
    clusterIP:
    port: 10053
    annotations: {}
  extraEnv: []
  deploymentAnnotations: {}
  deploymentLabels: {}
  containerAnnotations: {}
  containerLabels: {}
  extraVolumeMounts: []
  extraContainers: []
  extraInitContainers: []
  extraVolumes: []
  extraPodSpecs: {}
  securityContext: {}
  livenessProbe: {}
  readinessProbe: {}
  startupProbe: {}

zabbixJavaGateway:
  enabled: false
  replicaCount: 1
  resources: {}
  image:
    repository: zabbix/zabbix-java-gateway
    tag: null
    pullPolicy: IfNotPresent
    pullSecrets: []
  ZBX_START_POLLERS: 5
  ZBX_DEBUGLEVEL: 3
  ZBX_TIMEOUT: 3
  ZBX_JAVAGATEWAY: zabbix-java-gateway
  service:
    type: ClusterIP
    clusterIP:
    port: 10052
    listenOnAllInterfaces: true
    #nodePort: 31052
    annotations: {}
  extraEnv: []
  extraVolumeMounts: []
  deploymentAnnotations: {}
  deploymentLabels: {}
  containerAnnotations: {}
  containerLabels: {}
  extraContainers: []
  extraInitContainers: []
  extraVolumes: []
  extraPodSpecs: {}
  securityContext: {}
  livenessProbe:
    tcpSocket:
      port: zabbix-java-gw
    timeoutSeconds: 3
    failureThreshold: 3
    periodSeconds: 10
    successThreshold: 1
  readinessProbe: {}
  startupProbe:
    tcpSocket:
      port: zabbix-java-gw
    initialDelaySeconds: 15
    periodSeconds: 5
    timeoutSeconds: 3
    failureThreshold: 5
    successThreshold: 1

# Ingress configurations
ingress:
  enabled: false
  annotations: {}
  hosts:
    - host: zabbix.utcloud.io
      paths:
        - path: /
          pathType: ImplementationSpecific
  tls: 
    - utcloud-tls-star
  pathType: Prefix

# Ingress CRD object for the Traefik Ingresscontroller
ingressRoute:
  enabled: false
  annotations: {}
  entryPoints:
    - websecure
  hostName: chart-example.local

route:
  enabled: false
  hostName: chart-example.local
  tls:
    termination: edge
    #insecureEdgeTerminationPolicy: Redirect
  annotations: {}

nodeSelector: {}

tolerations: []

affinity: {}

securityContext: {}

karpenter:
  enabled: false
  clusterName: "CHANGE_HERE"
  tag: "karpenter.sh/discovery"
  resourceTags:
    Environment: testing
    Scost: zabbix
    product: zabbix
  instanceProfile:
    use: false
    name: "CHANGE_HERE"
  role:
    use: true
    name: "CHANGE_HERE"
  amiFamily: Bottlerocket
  limits:
    cpu: "2"
    memory: "8Gi"
  labels:
    karpenter: "true"
    app: "zabbix"
  weight: 10
  disruption:
    consolidationPolicy: "WhenEmpty"
    consolidateAfter: "30s"
    # You can choose to disable expiration entirely by setting the string value 'Never' here
    expireAfter: "720h"
  requirements:
    - key: "karpenter.k8s.aws/instance-category"
      operator: In
      values: ["c", "m", "r"]
    - key: "karpenter.k8s.aws/instance-cpu"
      operator: In
      values: ["2", "4", "8", "16", "32"]
    - key: "kubernetes.io/arch"
      operator: In
      values: ["amd64"]
    - key: kubernetes.io/os
      operator: In
      values: ["linux"]
    - key: "karpenter.sh/capacity-type"
      operator: In
      values: ["spot", "on-demand"]
  metadataOptions:
    httpEndpoint: enabled
    httpProtocolIPv6: disabled
    httpPutResponseHopLimit: 2
    httpTokens: required

serviceAccount:
  create: true
  name: ""
  annotations: {}
  labels: {}
  automountServiceAccountToken: true

rbac:
  create: true
  additionalRulesForClusterRole: []
$ mkdir /data/zabbix; cd /data/zabbix
$ curl -LO https://github.com/zabbix-community/helm-zabbix/releases/download/zabbix-4.3.0/zabbix-4.3.0.tgz

$ helm show values ./zabbix-4.3.0.tgz > override-values.yaml

$ vi override-values.yaml

$ helm upgrade -i -n monitoring zabbix ./zabbix-4.3.0.tgz \
 --dependency-update \
 --create-namespace \
 -f override-values.yaml

3. 설정하기

  • zabbix web에 Admin/zabbix로 로그인 후 Admin 암호를 재설정함.
  • host group, host 등록. host 등록시에 "linux by zabbix agent" template 매핑

4. 참고

  • hostNetwork: true 일 경우, k8s coredns를 사용하는 대신 Node의 전역 dns 구성을 가져온다.
    ks-installer-865fc48886-2hzrk:/kubesphere$ cat /etc/resolv.conf
    search kubesphere-system.svc.cluster.local svc.cluster.local cluster.local
    nameserver 169.254.25.10
    options ndots:5

zabbix@node-63:/var/lib/zabbix$ cat /etc/resolv.conf
nameserver 4.2.2.1
nameserver 4.2.2.2
nameserver 208.67.220.220

  • Node의 전역 DNS 설정
    root@node-63:~# cat /etc/systemd/resolved.conf
    [Resolve]
    DNS=4.2.2.1 4.2.2.2 208.67.220.220

  • 일반적인 k8s coreDns를 사용하는 Pod 예
    ks-installer-865fc48886-2hzrk:/kubesphere$ cat /etc/resolv.conf
    search kubesphere-system.svc.cluster.local svc.cluster.local cluster.local
    nameserver 169.254.25.10
    options ndots:5

oot@node-63:~# kubectl get svc -n kube-system
NAME                          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                        AGE
coredns                       ClusterIP   10.233.0.3      <none>        53/UDP,53/TCP,9153/TCP         3d4h
etcd                          ClusterIP   None            <none>        2379/TCP                       3d3h
kube-controller-manager-svc   ClusterIP   None            <none>        10257/TCP                      3d3h
kube-scheduler-svc            ClusterIP   None            <none>        10259/TCP                      3d3h
kubelet                       ClusterIP   None            <none>        10250/TCP,10255/TCP,4194/TCP   3d3h
metrics-server                ClusterIP   10.233.10.143   <none>        443/TCP                        3d3h

dnsPolicy: "None"  # Don't use the host's DNS policy.
dnsConfig:
  nameservers:
    - 10.233.0.3  # This should be the IP of your cluster's DNS service (kube-dns or core-dns).
  searches:
    - svc.cluster.local  # This is the default search domain.

참고) https://chimbu.medium.com/enable-internal-cluster-dns-resolution-for-pods-using-hostnetwork-in-kubernetes-845d714f465e

profile
cloud2000
클라우드쟁이
이전 포스트

kubekey 소스 분석

다음 포스트

PLG Stack

0개의 댓글

관련 채용 정보