harbor

Nam_JU·2025년 7월 2일

k8s

목록 보기

12/14

harbor

http환경 생성후, https환경으로 변경

docker 설치

sudo dnf -y install dnf-utils
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
sudo systemctl enable --now docker
docker version
docker compose version

harbor 설치

# Harbor 설치 파일 다운로드
wget https://github.com/goharbor/harbor/releases/download/v2.11.0/harbor-online-installer-v2.11.0.tgz
tar xzvf harbor-online-installer-v2.11.0.tgz
cd harbor

# 설정 파일 복사 및 편집
cp harbor.yml.tmpl harbor.yml
vim harbor.yml

      3 # The IP address or hostname to access admin UI and registry service.
      4 # DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
      5 hostname: harbor.test.com ##변경
...
      7 # http related config
      8 http:
      9   # port for http, default is 80. If https enabled, this port will redirect to https port
     10   port: 80
     11
     12 # https related config 주석처리
     13 #https:
     14 #  # https port for harbor, default is 443
     15 #  port: 443
     16   # The path of cert and key files for nginx
     17 #  certificate: /your/certificate/path
     18 #  private_key: /your/private/key/path
     19   # enable strong ssl ciphers (default: false)
     20   # strong_ssl_ciphers: false

     44 # The initial password of Harbor admin
     45 # It only works in first time to install harbor
     46 # Remember Change the admin password from UI after launching Harbor.
     47 harbor_admin_password: cloud1234 ##변경
     48
     
     ## log위치도 확인

dns설정

vi /etc/hosts
192.168.0.68 k8s-master
192.168.0.69 k8s-worker harbor.test.com

docker - harbor registry연동 : /etc/docker/daemon.json
- docker login “dns/ip”부분을 등록하지 않으면 로그인시 connect: connection refused 에러가 뜸!

vi /etc/docker/daemon.json

[root@k8s-worker ~]# cat /etc/docker/daemon.json
{
  "insecure-registries": [
    "harbor.test.com:80",min
    "192.168.0.69:80"
  ],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

sudo systemctl restart docker

[root@k8s-worker docker]# docker info | grep -i insecure
 Insecure Registries:
[root@k8s-worker docker]#

local pc에서 vi /etc/hosts로 ip:dns등록 후 접속
ip접속

dns접속

harbor system서비스 등록

매번 harbor를 실행하기위해 ./install.sh 파일을 실행시키기 귀찮으니 systemd서비스로 관리하도록 하자

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
docker-compose --version

service파일 생성

[root@k8s-worker ~]# which docker-compose
/usr/local/bin/docker-compose

[root@k8s-worker ~]# cat /etc/systemd/system/harbor.service
[Unit]
Description=Harbor Container Registry
Documentation=https://goharbor.io/
Requires=docker.service
After=docker.service
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/opt/harbor/harbor
ExecStartPre=/usr/local/bin/docker-compose down
ExecStart=/usr/local/bin/docker-compose up -d
ExecStop=/usr/local/bin/docker-compose down
ExecReload=/usr/local/bin/docker-compose restart
TimeoutStartSec=0
Restart=on-failure
RestartSec=30

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable harbor
sudo systemctl start harbor
sudo systemctl status harbor

프로젝트 생성 및 파일 업로드, 다운

프로젝트 생성

파일 업로드

cli로 로그인

## cat etc/docker/daemon.json
[root@k8s-worker ~]# cat /etc/docker/daemon.json
{
  "insecure-registries": [
    "harbor.test.com:80",
    "192.168.0.69:80"
  ],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}
[root@k8s-worker ~]# docker login "http://harbor.test.com:80"
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[root@k8s-worker ~]# docker login "http://192.168.0.69:80"
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

테스트 파일 생성

파일 다운

[root@k8s-worker ~]# docker pull nginx:latest
latest: Pulling from library/nginx
3da95a905ed5: Pull complete
6c8e51cf0087: Pull complete
9bbbd7ee45b7: Pull complete
48670a58a68f: Pull complete
ce7132063a56: Pull complete
23e05839d684: Pull complete
ee95256df030: Pull complete
Digest: sha256:93230cd54060f497430c7a120e2347894846a81b6a5dd2110f7362c5423b4abc
Status: Downloaded newer image for nginx:latest
docker.io/library/nginx:latest

파일 확인

[root@k8s-worker ~]# docker image list
REPOSITORY                    TAG       IMAGE ID       CREATED         SIZE
nginx                         latest    9592f5595f2b   7 days ago      192MB
goharbor/redis-photon         v2.11.0   184984d263c2   13 months ago   165MB
goharbor/harbor-registryctl   v2.11.0   f1220f69df90   13 months ago   162MB
goharbor/registry-photon      v2.11.0   95046ed33f52   13 months ago   84.5MB
goharbor/nginx-photon         v2.11.0   681ba9915791   13 months ago   153MB
goharbor/harbor-log           v2.11.0   a0a812a07568   13 months ago   163MB
goharbor/harbor-jobservice    v2.11.0   bba862a3784a   13 months ago   159MB
goharbor/harbor-core          v2.11.0   2cf11c05e0e2   13 months ago   185MB
goharbor/harbor-portal        v2.11.0   ea8fda08df5b   13 months ago   162MB
goharbor/harbor-db            v2.11.0   9bd788ea0df6   13 months ago   271MB
goharbor/prepare              v2.11.0   2baf15fbf5e2   13 months ago   207MB

tar로 저장 : docker save -o nginx_250702.tar nginx:latest

[root@k8s-worker ~]# docker save -o nginx_250702.tar nginx:latest
[root@k8s-worker ~]# ls -al
합계 547180
dr-xr-x---.  7 root root      4096  7월  2 16:42 .
-rw-------   1 root root 196392960  7월  2 16:42 nginx_250702.tar

tar 파일 다시 로드: docker load -i nginx_250702.tar

이미지 tag 변경후 하버 업로드

태그 변경: docker tag nginx:latest harbor.test.com:80/jupiter/nginx:250702

[root@k8s-worker ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.0.68 k8s-master
192.168.0.69 k8s-worker harbor.test.com
[root@k8s-worker ~]# docker tag nginx:latest harbor.test.com:80/jupiter/nginx:250702
[root@k8s-worker ~]# docker images
REPOSITORY                         TAG       IMAGE ID       CREATED         SIZE
nginx                              latest    9592f5595f2b   7 days ago      192MB
harbor.test.com:80/jupiter/nginx   250702    9592f5595f2b   7 days ago      192MB
goharbor/redis-photon              v2.11.0   184984d263c2   13 months ago   165MB
goharbor/harbor-registryctl        v2.11.0   f1220f69df90   13 months ago   162MB
goharbor/registry-photon           v2.11.0   95046ed33f52   13 months ago   84.5MB
goharbor/nginx-photon              v2.11.0   681ba9915791   13 months ago   153MB
goharbor/harbor-log                v2.11.0   a0a812a07568   13 months ago   163MB
goharbor/harbor-jobservice         v2.11.0   bba862a3784a   13 months ago   159MB
goharbor/harbor-core               v2.11.0   2cf11c05e0e2   13 months ago   185MB
goharbor/harbor-portal             v2.11.0   ea8fda08df5b   13 months ago   162MB
goharbor/harbor-db                 v2.11.0   9bd788ea0df6   13 months ago   271MB
goharbor/prepare                   v2.11.0   2baf15fbf5e2   13 months ago   207MB

이미지 push : docker push harbor.test.com:80/jupiter/nginx:250702

[root@k8s-worker ~]# docker push harbor.test.com:80/jupiter/nginx:250702
The push refers to repository [harbor.test.com:80/jupiter/nginx]
07eaefc6ebf2: Pushed
de2ef8ceb76a: Pushed
e6c40b7bdc83: Pushed
f941308035cf: Pushed
81a9d30670ec: Pushed
1bf33238ab09: Pushed
1bb35e8b4de1: Pushed
250702: digest: sha256:ccde53834eab53e85b35526a647cdb714ea4521b1ddf5a07b5c8787298d13087 size: 1778

harbor https적용

[root@k8s-worker ssl]# mkdir -p /etc/harbor/ssl
[root@k8s-worker ssl]# cd /etc/harbor/ssl


[root@k8s-worker ssl]# openssl genrsa -out ca.key 4096
Generating RSA private key, 4096 bit long modulus (2 primes)
.............................................++++
..............................++++
e is 65537 (0x010001)
[root@k8s-worker ssl]# openssl req -x509 -new -nodes -key ca.key -subj "/CN=Harbor-CA" -days 3650 -out ca.crt
[root@k8s-worker ssl]# openssl genrsa -out harbor.test.com.key 4096
Generating RSA private key, 4096 bit long modulus (2 primes)
..............................................................++++
...............................................................................++++
e is 65537 (0x010001)
[root@k8s-worker ssl]# ls
ca.crt  ca.key  harbor.test.com.key


[root@k8s-worker ssl]# openssl req -new -key harbor.test.com.key -subj "/CN=harbor.test.com" -out harbor.test.com.csr
[root@k8s-worker ssl]# ls
ca.crt  ca.key  harbor.test.com.csr  harbor.test.com.key


[root@k8s-worker ssl]# openssl x509 -req -in harbor.test.com.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
> -out harbor.test.com.crt -days 3650 -extensions v3_req -extfile <(cat <<EOF
> [ v3_req ]
> subjectAltName = @alt_names
>
> [alt_names]
> DNS.1 = harbor.test.com
> EOF
> )
Signature ok
subject=CN = harbor.test.com
Getting CA Private Key

[root@k8s-worker ssl]# ls
ca.crt  ca.key  ca.srl  harbor.test.com.crt  harbor.test.com.csr  harbor.test.com.key

파일 이름	설명	쉽게 말하면
`ca.key`	인증기관(CA)의 비밀 열쇠	"사장님 도장"
`ca.crt`	인증기관(CA)의 증명서	"이 도장은 진짜 사장님 거야!"
`harbor.test.com.key`	Harbor 서버의 비밀 열쇠	"Harbor 개인 도장"
`harbor.test.com.csr`	인증서 요청서	"사장님, 제 도장 인증해주세요"
`harbor.test.com.crt`	Harbor의 인증서	"사장님이 진짜라고 확인해줬어요!"

적용

harbor 설정 변경 후 반드시 ./prepare 스크립트를 실행해야함

[root@k8s-worker harbor]# cd harbor/
[root@k8s-worker harbor]# ls
LICENSE  common  common.sh  docker-compose.yml  harbor.yml  harbor.yml.tmpl  install.sh  prepare
[root@k8s-worker harbor]# ./prepare
prepare base dir is set to /opt/harbor/harbor

systemctl daemon-reload
systemctl restart containerd
systemctl restart docker
systemctl restart harbor

[root@k8s-worker harbor]# docker login https://harbor.test.com:80
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

태그 변경후 다시 업로드 테스트

[root@k8s-worker harbor]# docker tag harbor.test.com:80/jupiter/nginx:250702 harbor.test.com:80/jupiter/nginx:250702v
[root@k8s-worker harbor]# docker images
REPOSITORY                         TAG       IMAGE ID       CREATED         SIZE
nginx                              latest    9592f5595f2b   7 days ago      192MB
harbor.test.com:80/jupiter/nginx   250702    9592f5595f2b   7 days ago      192MB
harbor.test.com:80/jupiter/nginx   250702v   9592f5595f2b   7 days ago      192MB
goharbor/redis-photon              v2.11.0   184984d263c2   13 months ago   165MB
goharbor/harbor-registryctl        v2.11.0   f1220f69df90   13 months ago   162MB
goharbor/registry-photon           v2.11.0   95046ed33f52   13 months ago   84.5MB
goharbor/nginx-photon              v2.11.0   681ba9915791   13 months ago   153MB
goharbor/harbor-log                v2.11.0   a0a812a07568   13 months ago   163MB
goharbor/harbor-jobservice         v2.11.0   bba862a3784a   13 months ago   159MB
goharbor/harbor-core               v2.11.0   2cf11c05e0e2   13 months ago   185MB
goharbor/harbor-portal             v2.11.0   ea8fda08df5b   13 months ago   162MB
goharbor/harbor-db                 v2.11.0   9bd788ea0df6   13 months ago   271MB
goharbor/prepare                   v2.11.0   2baf15fbf5e2   13 months ago   207MB
[root@k8s-worker harbor]# docker push harbor.test.com:80/jupiter/nginx:250702v
The push refers to repository [harbor.test.com:80/jupiter/nginx]
07eaefc6ebf2: Layer already exists
de2ef8ceb76a: Layer already exists
e6c40b7bdc83: Layer already exists
f941308035cf: Layer already exists
81a9d30670ec: Layer already exists
1bf33238ab09: Layer already exists
1bb35e8b4de1: Layer already exists
250702v: digest: sha256:ccde53834eab53e85b35526a647cdb714ea4521b1ddf5a07b5c8787298d13087 size: 1778

harbor 설정파일 확인

타인의 pc에서 harbor환경을 확인하는 방법

하버가 존재하는지 확인 ps -ef | grep harbor

[root@k8s-worker ~]# ps -ef | grep harbor
10000     218032  217960  0 15:46 ?        00:00:03 /home/harbor/harbor_registryctl -c /etc/registryctl/config.yml
10000     218334  218314  2 15:46 ?        00:00:21 /harbor/harbor_core
10000     218835  218806  2 15:46 ?        00:00:27 /harbor/harbor_jobservice -c /etc/jobservice/config.yml
root      232919   93404  0 16:02 pts/0    00:00:00 grep --color=auto harbor

하버 dns, port 확인 : cat /etc/hosts , cat etc/docker/daemon.json , find / -name "*harbor.yml*" , netstat -nltp

## cat /etc/hosts
192.168.0.68 k8s-master
192.168.0.69 k8s-worker harbor.test.com

## cat etc/docker/daemon.json
[root@k8s-worker ~]# cat /etc/docker/daemon.json
{
  "insecure-registries": [
    "harbor.test.com:80",
    "192.168.0.69:80"
  ],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

## find / -name "*harbor.yml*"

## netstat -nltp
tcp6       0      0 :::22                   :::*                    LISTEN      775/sshd
tcp6       0      0 :::10250                :::*                    LISTEN      823/kubelet
tcp6       0      0 :::111                  :::*                    LISTEN      1/systemd
tcp6       0      0 :::80                   :::*                    LISTEN      218502/docker-proxy
tcp6       0      0 :::10256                :::*                    LISTEN      1355/kube-proxy

docker login 테스트

/etc/docker/daemon.json에 적힌 내용이 없으면 login이 안될수 있음 확인필요

[root@k8s-worker ~]# cat /etc/docker/daemon.json
{
  "insecure-registries": [
    "harbor.test.com:80",
    "192.168.0.69:80"
  ],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

## 없는것
[root@k8s-worker ~]# docker login "http://harbor.test.com"
Authenticating with existing credentials...
Login did not succeed, error: Error response from daemon: Get "https://harbor.test.com/v2/": dial tcp 192.168.0.69:443: connect: connection refused
Username (admin): admin
Password:
Error response from daemon: Get "https://harbor.test.com/v2/": dial tcp 192.168.0.69:443: connect: connection refused

## 등록된것
[root@k8s-worker ~]# docker login "http://harbor.test.com:80"
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
## 등록된것
[root@k8s-worker ~]# docker login "http://192.168.0.69:80"
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded