📗 개요 - Harbor - K8S

🏳️‍🌈 [궁금한점]

• Kubernetes에 저 사양 Harbor 설치 방법

---

🔗[목차]

• 1️⃣ 준비
  • StorageClass
  • Harbor 데이터 디렉토리
  • Harbor 차트 다운로드
• 2️⃣ 설정
  • 사용자 Values 파일 생성
• 3️⃣ 설치
  • Harbor 설치 (standalone)
  • POD 확인
  • Harbor PVC 확인
  • Harbor 차트 버전
• 4️⃣ 검증
  • 서비스 포트 확인
  • DNS
  • Ingress
    • Ingress 확인
    • Ning Ingress Controller 노드 포트 확인
  • 인그레스 컨트롤러 노드 포트 확인
  • Curl Test
  • 웹포털 컨테이너 헬스체크
  • 웹포털 DNS 확인
  • Harbor 코어를 통한 헬스체크
  • Nginx Ingress Controller 에 접속해서 Harbor Portal 연결확인
  • 콘솔 접속
• 참고
  • 자주 쓰는 명령어
  • Harbor 사용 볼륨
  • externalURL 과 Ingress 차이
  • externalURL 기능
• Trivy

1️⃣ 준비

StorageClass

운영 환경에서는 CSI를 사용해야 한다.

• sc 정의 (dev-harbor-sc.yml)

apiVersion: storage.k8s.io/v1
kind: StorageClass
reclaimPolicy: Delete  
metadata:
  name: dev-harbor-local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

• sc 생성

k apply -f dev-harbor-sc.yaml

persistentvolume/dev-harbor-pv1 created

Harbor 데이터 디렉토리

• 디렉토리 생성 (slave1)

sudo mkdir /tmp/dev-harbor-data{1..3}
sudo chown -R 1001:1001 /tmp/dev-harbor-data{1..3}

ls -ld /tmp/dev-harbor-data{1..3}
drwxr-xr-x 2 1001 1001 4096 Jun  7 03:08 /tmp/dev-harbor-data1
drwxr-xr-x 2 1001 1001 4096 Jun  7 03:08 /tmp/dev-harbor-data2
drwxr-xr-x 2 1001 1001 4096 Jun  7 03:08 /tmp/dev-harbor-data3

PV 생성

• pv 정의

apiVersion: v1
kind: PersistentVolume
metadata:
  name: dev-harbor-pv1
  labels:
    type: local
spec:
  storageClassName: dev-harbor-local-storage
  persistentVolumeReclaimPolicy: Delete
  capacity:
    storage: 2Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/tmp/dev-harbor-data1"
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - {key: kubernetes.io/hostname, operator: In, values: [slave1]}     
---        
apiVersion: v1
kind: PersistentVolume
metadata:
  name: dev-harbor-pv2
  labels:
    type: local
spec:
  storageClassName: dev-harbor-local-storage
  persistentVolumeReclaimPolicy: Delete
  capacity:
    storage: 2Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/tmp/dev-harbor-data2"
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - {key: kubernetes.io/hostname, operator: In, values: [slave1]}     
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: dev-harbor-pv3
  labels:
    type: local
spec:
  storageClassName: dev-harbor-local-storage
  persistentVolumeReclaimPolicy: Delete
  capacity:
    storage: 2Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/tmp/dev-harbor-data3"
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - {key: kubernetes.io/hostname, operator: In, values: [slave1]}            

• PV 생성

k apply -f dev-harbor-pv.yaml

persistentvolume/dev-harbor-pv1 created

k get pv

NAME             CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                 STORAGECLASS               REASON   AGE
dev-harbor-pv1   2Gi        RWO            Delete           Bound    my-dev/my-harbor-registry             dev-harbor-local-storage            38m
dev-harbor-pv2   2Gi        RWO            Delete           Bound    my-dev/data-my-harbor-trivy-0         dev-harbor-local-storage            38m
dev-harbor-pv3   2Gi        RWO            Delete           Bound    my-dev/my-harbor-jobservice           dev-harbor-local-storage            41m
dev-psql-pv1     1Gi        RWO            Delete           Bound    my-dev/data-my-psql-postgresql-0      dev-psql-local-storage              82m
dev-redis-pv1    2Gi        RWO            Delete           Bound    my-dev/redis-data-my-redis-master-0   dev-redis-local-storage             9m20s

Harbor 차트 다운로드

helm repo add harbor https://helm.goharbor.io
helm repo update

helm pull harbor/harbor --untar

2️⃣ 설정

사용자 Values 파일 생성

cp values.yaml dev-harbor-values.yaml

• prd-values.yaml 파일 편집
• 아래 내용 남기고 삭제

expose:
  type: ingress
  tls:
    enabled: false
  ingress:
    hosts:
      core: myharbor.local
    controller: default
    kubeVersionOverride: ""
    className: "nginx"

externalURL: http://myharbor.local:32091

persistence:
  enabled: true
  persistentVolumeClaim:
    registry:
      storageClass: "dev-harbor-local-storage"
      accessMode: ReadWriteOnce
      size: 2Gi
    jobservice:
      jobLog:
        storageClass: "dev-harbor-local-storage"
        accessMode: ReadWriteOnce
        size: 2Gi

harborAdminPassword: "Harbor12345"
portal:
  resources:
    requests:
      cpu: 50m
      memory: 64Mi
    limits:
      cpu: 100m
      memory: 128Mi

core:
  resources:
    requests:
      cpu: 50m
      memory: 64Mi
    limits:
      cpu: 100m
      memory: 128Mi

jobservice:
  resources:
    requests:
      cpu: 50m
      memory: 64Mi
    limits:
      cpu: 100m
      memory: 128Mi

registry:
  resources:
    requests:
      cpu: 50m
      memory: 64Mi
    limits:
      cpu: 100m
      memory: 128Mi
trivy:
  enabled: false   
database:
  type: external
  external:
    host: "my-psql-postgresql-0.my-psql-postgresql-hl.my-dev.svc.cluster.local"
    port: "5432"
    username: "admin"
    password: "psql"
    coreDatabase: "psql"
redis:
  type: external
  external:
    addr: "my-redis-master-0.my-redis-headless.my-dev.svc.cluster.local:6379"
    username: ""
    password: ""

3️⃣ 설치

Harbor 설치 (standalone)

helm install my-harbor . -n my-dev --create-namespace -f dev-harbor-values.yaml

NAME: my-harbor
LAST DEPLOYED: Sat Jun  7 12:56:05 2025
NAMESPACE: my-dev
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: harbor
CHART VERSION: 26.7.2
APP VERSION: 2.13.1

Did you know there are enterprise versions of the Bitnami catalog? For enhanced secure software supply chain features, unlimited pulls from Docker, LTS support, or application customization, see Bitnami Premium or Tanzu Application Catalog. See https://www.arrow.com/globalecs/na/vendors/bitnami for more information.

** Please be patient while the chart is being deployed **

1. Get the Harbor URL:

  NOTE: It may take a few minutes for the LoadBalancer IP to be available.
        Watch the status with: 'kubectl get svc --namespace my-dev -w my-harbor'
    export SERVICE_IP=$(kubectl get svc --namespace my-dev my-harbor --template "{{ range (index .status.loadBalancer.ingress 0) }}{{ . }}{{ end }}")
    echo "Harbor URL: http://$SERVICE_IP/"

2. Login with the following credentials to see your Harbor application

  echo Username: "admin"
  echo Password: $(kubectl get secret --namespace my-dev my-harbor-core-envvars -o jsonpath="{.data.HARBOR_ADMIN_PASSWORD}" | base64 -d)

WARNING: There are "resources" sections in the chart not set. Using "resourcesPreset" is not recommended for production. For production installations, please set the following values according to your workload needs:
  - exporter.resources
  - nginx.resources
  - registry.controller.resources
  - registry.server.resources
+info https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/

POD 확인

k get po -n my-dev -l app.kubernetes.io/instance=my-harbor

NAME                                    READY   STATUS    RESTARTS      AGE
my-harbor-core-68959d694-5nsqx          1/1     Running   0             21m
my-harbor-jobservice-6d6894fcfb-v8hqv   1/1     Running   2 (21m ago)   21m
my-harbor-portal-74c8fb9b8-xdhwb        1/1     Running   0             21m
my-harbor-registry-6567555d84-67pxz     2/2     Running   0             21m

Harbor PVC 확인

k get pvc -n my-dev -l app.kubernetes.io/instance=my-harbor

NAME                   STATUS   VOLUME           CAPACITY   ACCESS MODES   STORAGECLASS               AGE
my-harbor-jobservice   Bound    dev-harbor-pv1   2Gi        RWO            dev-harbor-local-storage   25h
my-harbor-registry     Bound    dev-harbor-pv2   2Gi        RWO            dev-harbor-local-storage   25h

Harbor 차트 버전

helm ls -n my-dev -l name=my-harbor

NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
my-harbor       my-dev          1               2025-06-08 16:36:57.6176201 +0900 KST   deployed        harbor-1.17.1   2.13.1

4️⃣ 검증

서비스 포트 확인

k get svc -n my-dev -l app.kubernetes.io/instance=my-harbor

NAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
my-harbor-core         ClusterIP   10.102.252.254   <none>        80/TCP              23m
my-harbor-jobservice   ClusterIP   10.98.218.182    <none>        80/TCP              23m
my-harbor-portal       ClusterIP   10.109.209.234   <none>        80/TCP              23m
my-harbor-registry     ClusterIP   10.106.229.13    <none>        5000/TCP,8080/TCP   23m

DNS

• hosts 파일 수정 , 192.168.56.10 은 자신의 노드 포트 IP

192.168.56.10 myharbor.local

ping myharbor.local

Ping myexample.local [192.168.56.10] 32바이트 데이터 사용:
192.168.56.10의 응답: 바이트=32 시간<1ms TTL=64
192.168.56.10의 응답: 바이트=32 시간<1ms TTL=64

Ingress

Ingress 확인

k get ing -n my-dev -l app.kubernetes.io/instance=my-harbor

NAME                CLASS   HOSTS            ADDRESS          PORTS   AGE
my-harbor-ingress   nginx   myharbor.local   192.168.56.102   80      2m47s

k get ing my-harbor-ingress -n my-dev -o yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/proxy-body-size: "0"
    ingress.kubernetes.io/ssl-redirect: "true"
    meta.helm.sh/release-name: my-harbor
    meta.helm.sh/release-namespace: my-dev
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
  creationTimestamp: "2025-06-08T07:36:59Z"
  generation: 1
  labels:
    app: harbor
    app.kubernetes.io/instance: my-harbor
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: harbor
    app.kubernetes.io/part-of: harbor
    app.kubernetes.io/version: 2.13.1
    chart: harbor
    heritage: Helm
    release: my-harbor
  name: my-harbor-ingress
  namespace: my-dev
  resourceVersion: "732625"
  uid: 0bd48ad5-19e6-46f5-a07b-5dc8faff525b
spec:
  ingressClassName: nginx
  rules:
  - host: myharbor.local
    http:
      paths:
      - backend:
          service:
            name: my-harbor-core
            port:
              number: 80
        path: /api/
        pathType: Prefix
      - backend:
          service:
            name: my-harbor-core
            port:
              number: 80
        path: /service/
        pathType: Prefix
      - backend:
          service:
            name: my-harbor-core
            port:
              number: 80
        path: /v2/
        pathType: Prefix
      - backend:
          service:
            name: my-harbor-core
            port:
              number: 80
        path: /c/
        pathType: Prefix
      - backend:
          service:
            name: my-harbor-portal
            port:
              number: 80
        path: /
        pathType: Prefix
status:
  loadBalancer:
    ingress:
    - ip: 192.168.56.102

Ning Ingress Controller 노드 포트 확인

k get svc -n my-dev -l app.kubernetes.io/instance=my-harbor

NAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
my-harbor-core         ClusterIP   10.109.125.130   <none>        80/TCP              3m12s
my-harbor-jobservice   ClusterIP   10.111.215.2     <none>        80/TCP              3m12s
my-harbor-portal       ClusterIP   10.111.178.74    <none>        80/TCP              3m12s
my-harbor-registry     ClusterIP   10.110.53.193    <none>        5000/TCP,8080/TCP   3m12s

인그레스 컨트롤러 노드 포트 확인

k get svc -n my-dev -l app.kubernetes.io/instance=my-nic

NAME                                              TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
my-nic-nginx-ingress-controller                   NodePort    10.107.24.79   <none>        80:32091/TCP,443:31971/TCP   81m
my-nic-nginx-ingress-controller-default-backend   ClusterIP   10.97.1.245    <none>        80/TCP                       81m

Curl Test

k exec -it my-harbor-portal-74c8fb9b8-xdhwb -n my-dev -- bash
nginx [ / ]$ cat /etc/resolv.conf
search my-dev.svc.cluster.local svc.cluster.local cluster.local
nameserver 10.96.0.10
options ndots:5
nginx [ / ]$ curl my-harbor-portal.my-dev.svc.cluster.local
<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8"/>
        <title>Harbor</title>
        <base href="/"/>
        <meta name="viewport" content="width=device-width, initial-scale=1"/>
        <link rel="icon" type="image/x-icon" href="favicon.ico?v=2"/>
    <link rel="stylesheet" href="styles.ac415221c96d2bef.css"></head>
    <body>
        <harbor-app>
            <div class="spinner spinner-lg app-loading app-loading-fixed">
                Loading...
            </div>
        </harbor-app>
    <script src="runtime.4eab865dc31b6057.js" type="module"></script><script src="polyfills.d87db3092ff69ed9.js" type="module"></script><script src="scripts.3846d86d42cdb753.js" defer></script><script src="main.a86461846fd53284.js" type="module"></script></body>
</html>

웹포털 컨테이너 헬스체크

k exec -it my-harbor-portal-74c8fb9b8-jmsb9 -n my-dev -- bash

nginx [ / ]$ curl http://my-harbor-core/api/v2.0/health

{"components":[{"name":"core","status":"healthy"},{"name":"database","status":"healthy"},{"name":"jobservice","status":"healthy"},{"name":"portal","status":"healthy"},{"name":"redis","status":"healthy"},{"name":"registry","status":"healthy"},{"name":"registryctl","status":"healthy"}],"status":"healthy"}

웹포털 DNS 확인

curl my-harbor-portal.my-dev.svc.cluster.local
<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8"/>
        <title>Harbor</title>
        <base href="/"/>
        <meta name="viewport" content="width=device-width, initial-scale=1"/>
        <link rel="icon" type="image/x-icon" href="favicon.ico?v=2"/>
    <link rel="stylesheet" href="styles.ac415221c96d2bef.css"></head>
    <body>
        <harbor-app>
            <div class="spinner spinner-lg app-loading app-loading-fixed">
                Loading...
            </div>
        </harbor-app>
    <script src="runtime.4eab865dc31b6057.js" type="module"></script><script src="polyfills.d87db3092ff69ed9.js" type="module"></script><script src="scripts.3846d86d42cdb753.js" defer></script><script src="main.a86461846fd53284.js" type="module"></script></body>
</html>

Harbor 코어를 통한 헬스체크

kubectl exec -it my-harbor-core-68959d694-5nsqx -n my-dev -- curl http://localhost:8080/api/v2.0/health

{"components":[{"name":"core","status":"healthy"},{"name":"database","status":"healthy"},{"name":"jobservice","status":"healthy"},{"name":"portal","status":"healthy"},{"name":"redis","status":"healthy"},{"name":"registry","status":"healthy"},{"name":"registryctl","status":"healthy"}],"status":"healthy"}

Nginx Ingress Controller 에 접속해서 Harbor Portal 연결확인

I have no name!@my-nic-nginx-ingress-controller-669797d9f5-kn5hm:/etc/nginx$ curl my-harbor-portal.my-dev.svc.cluster.local
<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8"/>
        <title>Harbor</title>
        <base href="/"/>
        <meta name="viewport" content="width=device-width, initial-scale=1"/>
        <link rel="icon" type="image/x-icon" href="favicon.ico?v=2"/>
    <link rel="stylesheet" href="styles.ac415221c96d2bef.css"></head>
    <body>
        <harbor-app>
            <div class="spinner spinner-lg app-loading app-loading-fixed">
                Loading...
            </div>
        </harbor-app>
    <script src="runtime.4eab865dc31b6057.js" type="module"></script><script src="polyfills.d87db3092ff69ed9.js" type="module"></script><script src="scripts.3846d86d42cdb753.js" defer></script><script src="main.a86461846fd53284.js" type="module"></script></body>
</html>

콘솔 접속

참고

자주 쓰는 명령어

# helm 삭제 후 재 생성
helm delete my-harbor -n my-dev
helm install my-harbor . -n my-dev -f dev-harbor-values.yaml --create-namespace


# persistentVolumeReclaimPolicy: Retain 일 경우  - Released 상태의 pv Available로 변경 
rm -rf /tmp/dev-harbor-data{1..3}/*
kubectl patch pv dev-harbor-pv{1..3} -p '{"spec":{"claimRef": null}}'

# persistentVolumeReclaimPolicy: Retain 일 경우 - PV 생성
k apply -f dev-harbor-pv.yaml

# persistentVolumeReclaimPolicy: Retain 일 경우 - pv 전체 삭제
k delete -f dev-harbor-pv.yaml

# PVC 삭제
k delete pvc -n my-dev data-my-harbor-postgresql-0
k delete pvc -n my-dev data-my-harbor-trivy-0
k delete pvc -n my-dev redis-data-my-harbor-redis-master-0
k delete pvc -n my-dev my-harbor-registry
k delete pvc -n my-dev my-harbor-jobservice

Harbor 사용 볼륨

컴포넌트	목적	디폴트 mount 경로
database	Harbor DB(PostgreSQL)	/var/lib/postgresql/data
registry	이미지 저장소	/storage
jobservice	이미지 처리 결과 저장소	/var/jobservice
redis	캐시/큐 저장소	/data
trivy	취약점 DB 저장소	/home/scanner/.cache

externalURL 과 Ingress 차이

항목	설명
Ingress	트래픽을 Harbor Core에 전달함
externalURL	Harbor 자체가 "내 주소는 이것이다"라고 인식하기 위한 값

externalURL 기능

Harbor 내부 컴포넌트와 클라이언트가 통신할 때 Harbor의 도메인을 명확히 인식하기 위한 기준 주소로 사용된다.
externalURL은 단순 노출 주소가 아니라, Harbor 자체 기능 동작에 핵심적인 메타데이터이다.

기능	externalURL 영향 여부	설명
Harbor UI 링크	영향 있음	Harbor UI 내부 링크들이 이 URL 기준으로 생성됨
Docker CLI 로그인	영향 있음	docker login harbor.example.com 에 필요한 주소
Replication	영향 있음	다른 Harbor 간 동기화 시 자기 주소 식별용
Notary	영향 있음	서명 기능 활성화 시 자기 도메인 필요
Webhook	영향 있음	외부 시스템에 이벤트 전송 시 주소 기준
E-Mail	영향 있음	메일 내용 내 "로그인 링크" 등 주소 기준
Token 기반 인증	영향 있음	Harbor Core → Registry → Token 전달 시 주소 필요

Trivy

• Aqua Security에서 만든 오픈소스 도구
• 컨테이너 이미지, 파일 시스템, Git 리포지토리, Kubernetes 클러스터 등을 스캔 가능
• Harbor에 기본 내장되어 취약점 분석 기능 제공
• Harbor 내에서는 주로 다음을 담당합니다:
• 이미지 푸시 시 자동 스캔 (기본 설정)
• 수동 스캔 버튼 제공
• 이미지 레포지토리에서 취약점 레포트 제공