Harbor 구축 (On Kubernetes)

NOHHYEONGJUN·2025년 6월 26일
CI/CDHarbor

CI/CD

목록 보기
13/15
post-thumbnail

Harbor

Harbor는 클라우드 네이티브 환경에 적합한 컨테이너 이미지 레지스트리(Container Image Registry)이다.

Docker Hub 같은 퍼블릭 레지스트리와는 달리, 프라이빗 환경에서 보안적이고 통제된 이미지 저장소를 제공한다.

 

  • 컨테이너 이미지를 안전하게 저장, 관리할 수 있는 오픈소스 레지스트리이다.

  • 취약점 스캐닝과 접근 제어 기능을 통해 컨테이너 이미지의 보안을 강화한다.

 

간단하게 Kubernetes 위에 Harbor를 배포하고 사용하는 방법을 정리해보았다.

추가로, keycloak OpenID Connect 연동도 함께 정리하였다.

 

 

1. Helm

Helm을 통해 배포하였고, 이 과정에서 커스텀 등이 필요해 Helm Repo를 직접 받은 후, 필요한 부분만 수정해 배포하였다.
이는 올바른 방법이 아니므로, 참고만 해서 진행하는 것이 좋을 것 같다.

repository 구조

.
├── Chart.yaml
├── templates
│   ├── core
│   │   └── core-cm.yaml
│   ├── harbor-logo-config.yaml
│   ├── harbor-style-config.yaml
│   ├── harbor-title-config.yaml
│   ├── ingress
│   │   └── ingress.yaml
│   ├── jobservice
│   │   └── jobservice-pvc.yaml
│   ├── patch-job.yaml
│   ├── rbac.yaml
│   └── registry
│       └── registry-pvc.yaml
└── values.yaml

 

values.yaml

externalURL: ex)www.myharbor.com
expose:
  type: ingress
  tls:
    enabled: false
  ingress:
    hosts:
      core: harbor.local
persistence:
  enabled: true
  annotations: {}
  persistentVolumeClaim:
    registry:
      existingClaim: "harbor-registry"
      storageClass: "longhorn"
      accessMode: ReadWriteOnce
      size: 150Gi
      annotations: {}
    jobservice:
      existingClaim: "harbor-jobservice"
      storageClass: "longhorn"
      accessMode: ReadWriteOnce
      size: 30Gi
      annotations: {}
    database:
      existingClaim: "database-data-harbor-database-0"
      storageClass: "longhorn"
      accessMode: ReadWriteOnce
      size: 30Gi
      annotations: {}
    redis:
      existingClaim: "data-harbor-redis-0"
      storageClass: "longhorn"
      accessMode: ReadWriteOnce
      size: 30Gi
      annotations: {}
    trivy:
      existingClaim: "data-harbor-trivy-0"
      storageClass: "longhorn"
      accessMode: ReadWriteOnce
      size: 30Gi
      annotations: {}
    chartmuseum:
      enabled: false        

# Core 설정
core:
  absoluteURL: true
  image:
    pullPolicy: IfNotPresent
  http:
    relativeurls: true
  configureUserSettings:
    enable: true
  custom:
    configMap:
      enabled: true
  reload: true 
  secret:
    secretName: "harbor-admin-secret"
    keyMappings:
      HARBOR_ADMIN_PASSWORD: "password"

# Jobservice 설정
jobservice:
  image:
    pullPolicy: IfNotPresent
  secretName: "harbor-jobservice-secret"  # 시크릿 사용
  secretKey: "secret"  
  workers:
    pool:
      workers: 20 # 워커 풀 조정 
  redis:
    namespace: "harbor_job_service"
    timeout: 3600

# Redis 설정
redis:
  type: internal
  internal:
    image:
      pullPolicy: IfNotPresent

registry:
  relativeurls: true
  
# Database 설정
database:
  type: internal
  internal:
    image:
      pullPolicy: IfNotPresent

# 공통 설정
commonName: harbor.local
imagePullPolicy: IfNotPresent

# 시크릿 참조
secretName: "harbor-admin-secret"
secretKey: "password"

updateStrategy:
  type: RollingUpdate

# 리소스 제한 설정 (최적화)
resources:
  requests:
    memory: 8Gi  # 증가
    cpu: 1000m    # 증가
  limits:
    memory: 16Gi  # 조정
    cpu: 2000m    # 조정

metrics:
  enabled: true
  core:
    path: /metrics
    port: 8001
  registry:
    path: /metrics
    port: 8001
  serviceMonitor:
    enabled: true
    labels: {}
    interval: 30s
    scrapeTimeout: 10s
    path: /metrics

 

Chart.yaml

apiVersion: v2
name: harbor-config
version: 1.0.0
dependencies:
  - name: harbor
    version: 1.16.1  
    repository: https://helm.goharbor.io

 

core-cm.yaml

apiVersion: v1
data:
  CHART_CACHE_DRIVER: redis
  CONFIG_PATH: /etc/core/app.conf
  CORE_LOCAL_URL: http://127.0.0.1:8080
  CORE_URL: http://harbor-core:80
  DATABASE_TYPE: postgresql
  EXT_ENDPOINT: ex)www.myharbor.com
  HTTPS_PROXY: ''
  HTTP_PROXY: ''
  JOBSERVICE_URL: http://harbor-jobservice
  LOG_LEVEL: info
  NO_PROXY: >-
    harbor-core,harbor-jobservice,harbor-database,harbor-registry,harbor-portal,harbor-trivy,harbor-exporter,127.0.0.1,localhost,.local,.internal
  PERMITTED_REGISTRY_TYPES_FOR_PROXY_CACHE: >-
    docker-hub,harbor,azure-acr,aws-ecr,google-gcr,quay,docker-registry,github-ghcr,jfrog-artifactory
  PORT: '8080'
  PORTAL_URL: http://harbor-portal
  POSTGRESQL_DATABASE: registry
  POSTGRESQL_HOST: harbor-database
  POSTGRESQL_MAX_IDLE_CONNS: '100'
  POSTGRESQL_MAX_OPEN_CONNS: '900'
  POSTGRESQL_PORT: '5432'
  POSTGRESQL_SSLMODE: disable
  POSTGRESQL_USERNAME: postgres
  QUOTA_UPDATE_PROVIDER: db
  REGISTRY_CONTROLLER_URL: http://harbor-registry:8080
  REGISTRY_CREDENTIAL_USERNAME: harbor_registry_user
  REGISTRY_STORAGE_PROVIDER_NAME: filesystem
  REGISTRY_URL: http://harbor-registry:5000
  TOKEN_SERVICE_URL: http://harbor-core:80/service/token
  TRIVY_ADAPTER_URL: http://harbor-trivy:8080
  WITH_TRIVY: 'true'
  _REDIS_URL_CORE: redis://harbor-redis:6379/0?idle_timeout_seconds=30
  _REDIS_URL_REG: redis://harbor-redis:6379/2?idle_timeout_seconds=30
  app.conf: |
    appname = Harbor
    runmode = prod
    enablegzip = true

    [prod]
    httpport = 8080
kind: ConfigMap
metadata:
  labels:
    app: harbor
    app.kubernetes.io/instance: harbor
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: harbor
    app.kubernetes.io/part-of: harbor
    app.kubernetes.io/version: 2.12.1
    chart: harbor
    heritage: Helm
    release: harbor
  name: harbor-core
  namespace: harbor

 

ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: harbor-ingress
  namespace: harbor
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "4096k"
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"  
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"  
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "60"  
spec:
  ingressClassName: nginx
  rules:
  - host: ex)www.myharbor.com
    http:
      paths:
      - path: /api/
        pathType: Prefix
        backend:
          service:
            name: harbor-core
            port:
              number: 80
      - path: /service/
        pathType: Prefix
        backend:
          service:
            name: harbor-core
            port:
              number: 80
      - path: /c/
        pathType: Prefix
        backend:
          service:
            name: harbor-core
            port:
              number: 80
      - path: /v2/
        pathType: Prefix
        backend:
          service:
            name: harbor-core  
            port:
              number: 80  
      - path: /
        pathType: Prefix
        backend:
          service:
            name: harbor-portal
            port:
              number: 80

 

jobservice-pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    helm.sh/resource-policy: keep
  labels:
    app: harbor
    app.kubernetes.io/component: jobservice
    app.kubernetes.io/instance: harbor
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: harbor
    app.kubernetes.io/part-of: harbor
    app.kubernetes.io/version: 2.12.1
    chart: harbor
    component: jobservice
    heritage: Helm
    release: harbor
  name: harbor-jobservice
  namespace: harbor
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: longhorn
  resources:
    requests:
      storage: 30Gi

 

registry-pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    helm.sh/resource-policy: keep
  labels:
    app: harbor
    app.kubernetes.io/component: registry
    app.kubernetes.io/instance: harbor
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: harbor
    app.kubernetes.io/part-of: harbor
    app.kubernetes.io/version: 2.12.1
    chart: harbor
    component: registry
    heritage: Helm
    release: harbor
  name: harbor-registry
  namespace: harbor
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: longhorn
  resources:
    requests:
      storage: 150Gi

 

 

2. Style Custom

Logo, Color 등을 커스텀하기 위해 추가적으로 설정하였다.
배포 시, 자동으로 설정을 변경하기 위한 설정이다.

harbor-logo-config.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: harbor-custom-logo
  namespace: harbor
  annotations:
    helm.sh/hook: post-install,post-upgrade
    helm.sh/hook-weight: "-3"
    helm.sh/hook-delete-policy: hook-succeeded
data:
  favicon.ico: |-
    <?xml version="1.0" encoding="UTF-8" standalone="no"?> .....
  harbor-logo.svg: |-
    <?xml version="1.0" encoding="UTF-8" standalone="no"?> .....

 

harbor-style-config.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: harbor-custom-style
  namespace: harbor
  annotations:
    helm.sh/hook: post-install,post-upgrade
    helm.sh/hook-weight: "-2"
    helm.sh/hook-delete-policy: hook-succeeded
data:
  dark-theme.css: |
    :root {
      --clr-color-action-50: hsl(270, 83%, 94%);
      --clr-color-action-100: hsl(270, 81%, 88%);
      --clr-color-action-200: hsl(270, 78%, 78%);
      --clr-color-action-300: hsl(270, 69%, 69%);
      --clr-color-action-400: hsl(270, 66%, 57%);
      --clr-color-action-500: hsl(270, 80%, 46%);
      --clr-color-action-600: hsl(270, 100%, 32%);
      --clr-color-action-700: hsl(270, 100%, 28%);
      --clr-color-action-800: hsl(270, 100%, 24%);
      --clr-color-action-900: hsl(270, 100%, 21%);
      --header-color: hsl(270, 100%, 25%);
    }
    .header-container {
      background-color: var(--header-color) !important;
    }
    .nav-link-container > .nav-link {
      color: #ffffff !important;
    }
    .btn-primary {
      background-color: var(--header-color) !important;
      border-color: hsl(270, 100%, 32%) !important;
      color: #ffffff !important;
    }
    .btn-primary:hover {
      background-color: hsl(270, 100%, 28%) !important;
    }
    .side-nav {
      background-color: hsl(270, 30%, 98%) !important;
    }
    .side-nav .nav-link {
      color: #000000 !important;
    }
    .harbor-container, .harbor-body {
      background-color: hsl(270, 20%, 99%) !important;
      color: #000000 !important;
    }
    .table thead th {
      background-color: var(--header-color) !important;
      color: white !important;
    }
    .table tbody td {
      color: #000000 !important;
    }
    a {
      color: var(--header-color) !important;
    }
    a:hover {
      color: hsl(270, 100%, 32%) !important;
    }
    .nav-item.active {
      background-color: var(--header-color) !important;
      color: white !important;
    }
    .search-input {
      border-color: var(--header-color) !important;
      color: #acbac3 !important;
      background-color: transparent !important;
    }
    .text-primary {
      color: #000000 !important;
    }
    .datagrid {
      color: #000000 !important;
    }
    /* 레지스트리 이름 색상 */
    .repository-name {
      color: #000000 !important;
    }

    /* 로그인 페이지 스타일 */
    .login-wrapper {
      background: linear-gradient(135deg,
        var(--header-color) 0%,
        hsl(270, 45%, 35%) 50%,
        hsl(270, 40%, 45%) 100%) !important;
    }

    /* 로그인 카드 */
    .login-wrapper .login {
      background-color: white !important;
      border-radius: 8px !important;
      box-shadow: 0 4px 12px rgba(0, 0, 0, 0.1) !important;
    }

    /* 로그인 버튼 */
    .login-wrapper .btn-primary {
      background-color: var(--header-color) !important;
      border-color: hsl(270, 100%, 32%) !important;
      width: 100% !important;
    }

    /* 로그인 입력 필드 */
    .login-wrapper input {
      border: 1px solid hsl(270, 20%, 90%) !important;
      border-radius: 4px !important;
      color: #000000 !important;
    }

    .login-wrapper input:focus {
      border-color: var(--header-color) !important;
      box-shadow: 0 0 0 2px hsla(270, 80%, 46%, 0.2) !important;
    }

    /* 로그인 배경 이미지 숨기기 */
    .login-wrapper .login-aside {
      background: none !important;
    }
    .harbor-logo {
      font-size: 0 !important;  /* 기존 텍스트 숨기기 */
    }
    .harbor-logo:after {
      content: "Harbor" !important;
      font-size: 20px !important;
      color: white !important;
      font-weight: 500 !important;
    }

    /* 브라우저 탭 타이틀 변경 */
    title {
      font-size: 0 !important;
    }
    title:after {
      content: "Harbor" !important;
      font-size: inherit !important;
    }


  light-theme.css: |
    :root {
      --clr-color-action-50: hsl(270, 83%, 94%);
      --clr-color-action-100: hsl(270, 81%, 88%);
      --clr-color-action-200: hsl(270, 78%, 78%);
      --clr-color-action-300: hsl(270, 69%, 69%);
      --clr-color-action-400: hsl(270, 66%, 57%);
      --clr-color-action-500: hsl(270, 80%, 46%);
      --clr-color-action-600: hsl(270, 100%, 32%);
      --clr-color-action-700: hsl(270, 100%, 28%);
      --clr-color-action-800: hsl(270, 100%, 24%);
      --clr-color-action-900: hsl(270, 100%, 21%);
      --header-color: hsl(270, 100%, 25%);
    }
    .header-container {
      background-color: var(--header-color) !important;
    }
    .nav-link-container > .nav-link {
      color: #ffffff !important;
    }
    .btn-primary {
      background-color: var(--header-color) !important;
      border-color: hsl(270, 100%, 32%) !important;
      color: #ffffff !important;
    }
    .btn-primary:hover {
      background-color: hsl(270, 100%, 28%) !important;
    }
    .side-nav {
      background-color: hsl(270, 30%, 98%) !important;
    }
    .side-nav .nav-link {
      color: #000000 !important;
    }
    .harbor-container, .harbor-body {
      background-color: hsl(270, 20%, 99%) !important;
      color: #000000 !important;
    }
    .table thead th {
      background-color: var(--header-color) !important;
      color: white !important;
    }
    .table tbody td {
      color: #000000 !important;
    }
    a {
      color: var(--header-color) !important;
    }
    a:hover {
      color: hsl(270, 100%, 32%) !important;
    }
    .nav-item.active {
      background-color: var(--header-color) !important;
      color: white !important;
    }
    .search-input {
      border-color: var(--header-color) !important;
      color: #acbac3 !important;
      background-color: transparent !important;
    }
    .text-primary {
      color: #000000 !important;
    }
    .datagrid {
      color: #000000 !important;
    }
    /* 레지스트리 이름 색상 */
    .repository-name {
      color: #000000 !important;
    }

    /* 로그인 페이지 스타일 */
    .login-wrapper {
      background: linear-gradient(135deg,
        var(--header-color) 0%,
        hsl(270, 45%, 35%) 50%,
        hsl(270, 40%, 45%) 100%) !important;
    }

    /* 로그인 카드 */
    .login-wrapper .login {
      background-color: white !important;
      border-radius: 8px !important;
      box-shadow: 0 4px 12px rgba(0, 0, 0, 0.1) !important;
    }

    /* 로그인 버튼 */
    .login-wrapper .btn-primary {
      background-color: var(--header-color) !important;
      border-color: hsl(270, 100%, 32%) !important;
      width: 100% !important;
    }

    /* 로그인 입력 필드 */
    .login-wrapper input {
      border: 1px solid hsl(270, 20%, 90%) !important;
      border-radius: 4px !important;
      color: #000000 !important;
    }

    .login-wrapper input:focus {
      border-color: var(--header-color) !important;
      box-shadow: 0 0 0 2px hsla(270, 80%, 46%, 0.2) !important;
    }

    /* 로그인 배경 이미지 숨기기 */
    .login-wrapper .login-aside {
      background: none !important;
    }
    .harbor-logo {
      font-size: 0 !important;  /* 기존 텍스트 숨기기 */
    }
    .harbor-logo:after {
      content: "Harbor" !important;
      font-size: 20px !important;
      color: white !important;
      font-weight: 500 !important;
    }

    /* 브라우저 탭 타이틀 변경 */
    title {
      font-size: 0 !important;
    }
    title:after {
      content: "Harbor" !important;
      font-size: inherit !important;
    }

 

harbor-title-config.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: harbor-custom-title
  namespace: harbor
  annotations:
    helm.sh/hook: post-install,post-upgrade
    helm.sh/hook-weight: "-1"
    helm.sh/hook-delete-policy: hook-succeeded
data:
  setting.json: |
    {
        "headerBgColor": {
        "darkMode": "",
        "lightMode": ""
      },
      "loginBgImg": "",
      "loginTitle": "Harbor",
      "product": {
        "name": "Harbor",
        "logo": "",
        "introduction": ""
      }
    }
  index.html: |
    <!DOCTYPE html>
    <html>
      <head>
        <meta charset="utf-8"/>
        <title>Harbor</title>
        <base href="/"/>
        <meta name="viewport" content="width=device-width, initial-scale=1"/>
        <link rel="icon" href="favicon.ico?v=2" type="image/x-icon">
    <link rel="stylesheet" href="styles.ac415221c96d2bef.css"></head>
    <body>
        <harbor-app>
            <div class="spinner spinner-lg app-loading app-loading-fixed">
                Loading...
            </div>
        </harbor-app>
    .....
    </html>

 

patch-job.yaml

apiVersion: batch/v1
kind: Job
metadata:
  name: patch-harbor-portal
  annotations:
    helm.sh/hook: post-install,post-upgrade
    helm.sh/hook-weight: "1"
    helm.sh/hook-delete-policy: hook-succeeded
spec:
  template:
    spec:
      serviceAccountName: harbor-patch-sa
      containers:
      - name: kubectl
        image: bitnami/kubectl
        command:
        - /bin/sh
        - -c
        - |
          # 현재 deployment 상태 확인
          if ! kubectl get deployment harbor-portal -n {{ .Release.Namespace }} -o json | grep -q "custom-logo"; then
            kubectl patch deployment harbor-portal -n {{ .Release.Namespace }} --type=json -p='[
              {
                "op": "add",
                "path": "/spec/template/spec/containers/0/volumeMounts/-",
                "value": {
                  "mountPath": "/usr/share/nginx/html/images/harbor-logo.svg",
                  "name": "custom-logo",
                  "subPath": "harbor-logo.svg"
                }
              },
              {
                "op": "add",
                "path": "/spec/template/spec/containers/0/volumeMounts/-",
                "value": {
                  "mountPath": "/usr/share/nginx/html/dark-theme.css",
                  "name": "custom-style",
                  "subPath": "dark-theme.css"
                }
              },
              {
                "op": "add",
                "path": "/spec/template/spec/containers/0/volumeMounts/-",
                "value": {
                  "mountPath": "/usr/share/nginx/html/light-theme.css",
                  "name": "custom-style",
                  "subPath": "light-theme.css"
                }
              },
              {
                "op": "add",
                "path": "/spec/template/spec/containers/0/volumeMounts/-",
                "value": {
                  "mountPath": "/usr/share/nginx/html/index.html",
                  "name": "custom-title",
                  "subPath": "index.html"
                }
              },
              {
                "op": "add",
                "path": "/spec/template/spec/containers/0/volumeMounts/-",
                "value": {
                  "mountPath": "/usr/share/nginx/html/setting.json",
                  "name": "custom-title",
                  "subPath": "setting.json"
                }
              },
              {
                "op": "add",
                "path": "/spec/template/spec/volumes/-",
                "value": {
                  "name": "custom-logo",
                  "configMap": {
                    "name": "harbor-custom-logo"
                  }
                }
              },
              {
                "op": "add",
                "path": "/spec/template/spec/volumes/-",
                "value": {
                  "name": "custom-style",
                  "configMap": {
                    "name": "harbor-custom-style"
                  }
                }
              },
              {
                "op": "add",
                "path": "/spec/template/spec/volumes/-",
                "value": {
                  "name": "custom-title",
                  "configMap": {
                    "name": "harbor-custom-title"
                  }
                }
              }
            ]'
          else
            echo "Custom volumes already exist. Skipping patch."
          fi
      restartPolicy: OnFailure
  backoffLimit: 4

 

rbac.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: harbor-patch-sa
  namespace: {{ .Release.Namespace }}
  annotations:
    helm.sh/hook: post-install,post-upgrade
    helm.sh/hook-weight: "0"
    helm.sh/hook-delete-policy: hook-succeeded
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: harbor-patch-role
  namespace: {{ .Release.Namespace }}
  annotations:
    helm.sh/hook: post-install,post-upgrade
    helm.sh/hook-weight: "0"
    helm.sh/hook-delete-policy: hook-succeeded
rules:
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: harbor-patch-rolebinding
  namespace: {{ .Release.Namespace }}
  annotations:
    helm.sh/hook: post-install,post-upgrade
    helm.sh/hook-weight: "0"
    helm.sh/hook-delete-policy: hook-succeeded
subjects:
- kind: ServiceAccount
  name: harbor-patch-sa
roleRef:
  kind: Role
  name: harbor-patch-role
  apiGroup: rbac.authorization.k8s.io

 

 

3. Keycloak OpenID Connect 설정

기본 인증 모드 사용 시, 모든 사용자는 Harbor로 접속 시 Keycloak 로그인페이지로 이동하게 된다.
만약 Keycloak DB 계정 (Admin)으로 접속 필요 시, URL/account/sign-in 으로 접속하면 된다.

 

 

4. 프로젝트 할당량 설정

프로젝트 생성 시, 기본적으로 할당될 크기를 설정할 수 있다.

 

 

5. 프로젝트 생성 권한 설정

프로젝트를 관리자만 생성할 수 있도록 설정한다.

 

 

결론

Harbor의 경우, 변경된 부분이 많이 존재한다.

가장 많이 쓰이는 Container Image Registry 이므로, 중요하다.

profile
NOHHYEONGJUN
Cloud/DevOps & Network Virtualization에 관심 있는 대학생입니다. 🐳
이전 포스트

Jenkins 구축 (On Kubernetes)

다음 포스트

Sonarqube 구축 (On Kubernetes)

0개의 댓글