Compute 노드 추가

NOHHYEONGJUN·2025년 3월 31일
openstack

OpenStack

목록 보기
21/23
post-thumbnail

0. GPU 사용 필요

구축한 오픈스택을 통해 구축한 쿠버네티스를 사용하던 도중, GPU 클러스터 사용이 필요해졌다.
그래서 별도로 있던 A6000이 장착된 서버를 Compute 노드로 추가해 사용하려고 한다.

새롭게 추가하는 노드를 com02 로 지정하겠다.

 

 

1. 미러링 사이트 변경

패키지 설치 등을 고속으로 진행하기 위해 국내 사이트로 변경한다.

sudo -s
cd /etc/apt

# 백업
cp sources.list sources.list.v1

sed -i 's/http:\/\/kr.archive.ubuntu.com/https:\/\/ftp.kaist.ac.kr/g' /etc/apt/sources.list

apt update

Get:1 https://ftp.kaist.ac.kr/ubuntu jammy InRelease [270 kB]
Get:2 https://ftp.kaist.ac.kr/ubuntu jammy-updates InRelease [128 kB] 
Get:3 https://ftp.kaist.ac.kr/ubuntu jammy-backports InRelease [127 kB]
Get:4 https://ftp.kaist.ac.kr/ubuntu jammy/main amd64 Packages [1395 kB]
Get:5 https://ftp.kaist.ac.kr/ubuntu jammy/main Translation-en [510 kB]
Get:6 https://ftp.kaist.ac.kr/ubuntu jammy/main amd64 c-n-f Metadata [30.3 kB]                                   
Get:7 https://ftp.kaist.ac.kr/ubuntu jammy/restricted amd64 Packages [129 kB]                                       
Get:8 https://ftp.kaist.ac.kr/ubuntu jammy/restricted Translation-en [18.6 kB]                              
Get:9 https://ftp.kaist.ac.kr/ubuntu jammy/restricted amd64 c-n-f Metadata [488 B]                                  
Get:10 https://ftp.kaist.ac.kr/ubuntu jammy/universe amd64 Packages [14.1 MB]                     
Get:11 https://ftp.kaist.ac.kr/ubuntu jammy/universe Translation-en [5652 kB]                                 
Get:12 https://ftp.kaist.ac.kr/ubuntu jammy/universe amd64 c-n-f Metadata [286 kB]                                  
Get:13 https://ftp.kaist.ac.kr/ubuntu jammy/multiverse amd64 Packages [217 kB]                                    
Get:14 https://ftp.kaist.ac.kr/ubuntu jammy/multiverse Translation-en [112 kB]                                    
Get:15 https://ftp.kaist.ac.kr/ubuntu jammy/multiverse amd64 c-n-f Metadata [8372 B]                                     
Get:16 https://ftp.kaist.ac.kr/ubuntu jammy-updates/main amd64 Packages [2316 kB]                                        
Get:17 https://ftp.kaist.ac.kr/ubuntu jammy-updates/main Translation-en [387 kB]                                   
Get:18 https://ftp.kaist.ac.kr/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.9 kB]     
Get:19 https://ftp.kaist.ac.kr/ubuntu jammy-updates/restricted amd64 Packages [2944 kB]
Get:20 https://ftp.kaist.ac.kr/ubuntu jammy-updates/restricted Translation-en [515 kB]
Get:21 https://ftp.kaist.ac.kr/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [612 B] 
Get:22 https://ftp.kaist.ac.kr/ubuntu jammy-updates/universe amd64 Packages [1187 kB]
Get:23 https://ftp.kaist.ac.kr/ubuntu jammy-updates/universe Translation-en [291 kB]  
Get:24 https://ftp.kaist.ac.kr/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.4 kB] 
Get:25 https://ftp.kaist.ac.kr/ubuntu jammy-updates/multiverse amd64 Packages [44.5 kB] 
Get:26 https://ftp.kaist.ac.kr/ubuntu jammy-updates/multiverse Translation-en [11.5 kB]
Get:27 https://ftp.kaist.ac.kr/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [440 B]
Get:28 https://ftp.kaist.ac.kr/ubuntu jammy-backports/main amd64 Packages [67.7 kB] 
Get:29 https://ftp.kaist.ac.kr/ubuntu jammy-backports/main Translation-en [11.1 kB] 
Get:30 https://ftp.kaist.ac.kr/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 
Get:31 https://ftp.kaist.ac.kr/ubuntu jammy-backports/restricted amd64 c-n-f Metadata [116 B]
Get:32 https://ftp.kaist.ac.kr/ubuntu jammy-backports/universe amd64 Packages [30.0 kB]
Get:33 https://ftp.kaist.ac.kr/ubuntu jammy-backports/universe Translation-en [16.6 kB]
Get:34 https://ftp.kaist.ac.kr/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B]
Get:35 https://ftp.kaist.ac.kr/ubuntu jammy-backports/multiverse amd64 c-n-f Metadata [116 B]
Hit:36 http://security.ubuntu.com/ubuntu jammy-security InRelease
Fetched 30.8 MB in 3s (10.6 MB/s)               
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
162 packages can be upgraded. Run 'apt list --upgradable' to see them.

 

 

2. 시간대 설정

시간대를 서울로 설정해준다.

timedatectl set-timezone Asia/Seoul; timedatectl

               Local time: Wed 2025-02-19 13:25:14 KST
           Universal time: Wed 2025-02-19 04:25:14 UTC
                 RTC time: Wed 2025-02-19 04:25:14
                Time zone: Asia/Seoul (KST, +0900)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no

 

 

3. 필요 유틸리티 설치 및 패키지 업그레이드

분홍색 박스가 뜨면 ok를 선택해준다.

apt install -y lrzsz net-tools zip bzip2 make gcc perl
apt upgrade -y

reboot

 

 

4. 커널 버전 자동 업데이트 금지 및 보안 업데이트 자동 수행 설정

분홍색 박스가 뜨면 ok를 선택해준다.

sudo -s
apt-mark hold $(uname -r)
apt install unattended-upgrades
dpkg-reconfigure unattended-upgrades

 

 

5. NFS 설정

기존에 설치할때 처럼 NFS를 통해 컨트롤노드에서 파일을 가져와 사용한다.

apt install -y nfs-common

showmount -e 10.0.0.1
	Export list for 10.0.0.1:
	/nfs 10.0.0.0/24

mkdir -p /nfs

# 마운트 안되었을 시
sudo mount 10.0.0.1:/nfs /nfs

 

 

6. NTP 설정

con01에 맞춰준다.

apt install chrony

vi /etc/chrony/chrony.conf
	server con01 iburst

service chrony restart
service chrony status

chronyc sources
MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
^- alphyn.canonical.com          2   6    17    26  -2808us[-2822us] +/-  122ms
^- prod-ntp-3.ntp1.ps5.cano>     2   6    17    26  +8528us[+8528us] +/-  138ms
^- prod-ntp-5.ntp1.ps5.cano>     2   6    17    27  +9053us[+9038us] +/-  137ms
^- prod-ntp-4.ntp1.ps5.cano>     2   6    17    25  +1311us[+1311us] +/-  130ms
^- 121.174.142.82                3   6    17    27   -383us[ -396us] +/-   34ms
^+ 141.164.43.237.vultruser>     2   6    17    26   -340us[ -356us] +/-   19ms
^- any.time.nl                   2   6    17    27   +211us[ +198us] +/-   60ms
^- 121.174.142.81                3   6    17    28    -84us[ -546us] +/-   34ms
^* con01                         3   6    17    26   +143us[ +126us] +/- 7040us

 

 

7. OpenStack Packages 설치

add-apt-repository cloud-archive:antelope

apt update

apt install python3-openstackclient

 

 

8. Nova 설치

com01이 10.0.0.2 이므로, com02는 10.0.0.3 으로 설정했다.

apt install nova-compute

cd /etc/nova/

cp nova.conf nova.conf.v0

/nfs/sed.sh nova.conf

vi /etc/nova/nova.conf

[DEFAULT]
transport_url = rabbit://openstack:openstack@con01

[api]
auth_strategy = keystone

[keystone_authtoken]
www_authenticate_uri = http://con01:5000/
auth_url = http://con01:5000/
memcached_servers = con01:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = nova
password = openstack

[service_user]
send_service_user_token = true
auth_url = https://con01/identity
auth_strategy = keystone
auth_type = password
project_domain_name = Default
project_name = service
user_domain_name = Default
username = nova
password = openstack

[DEFAULT]
my_ip = 10.0.0.3

[vnc]
enabled = true
server_listen = 0.0.0.0
server_proxyclient_address = $my_ip
novncproxy_base_url = http://con01:6080/vnc_auto.html

[glance]
api_servers = http://con01:9292

[oslo_concurrency]
lock_path = /var/lib/nova/tmp

[placement]
region_name = RegionOne
project_domain_name = Default
project_name = service
auth_type = password
user_domain_name = Default
auth_url = http://con01:5000/v3
username = placement
password = openstack

service nova-compute restart

con01

com01이 추가되었는 지 확인한다.

openstack compute service list --service nova-compute

 

 

9. Neutron 설치

apt install neutron-openvswitch-agent

cd /etc/neutron/

cp neutron.conf neutron.conf.v0

/nfs/sed.sh neutron.conf

vi neutron.conf

[DEFAULT]
transport_url = rabbit://openstack:openstack@con01
auth_strategy = keystone

[keystone_authtoken]
www_authenticate_uri = http://con01:5000
auth_url = http://con01:5000
memcached_servers = con01:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = neutron
password = openstack

[oslo_concurrency]
lock_path = /var/lib/neutron/tmp

 

cd /etc/neutron/plugins/ml2/

cp openvswitch_agent.ini openvswitch_agent.ini.v0

/nfs/sed.sh openvswitch_agent.ini

vi openvswtich_agent.ini

[ovs]
bridge_mappings = provider:PROVIDER_INTERFACE_NAME # 뒤에서 수정

[vxlan]
local_ip = 10.0.0.3
l2_population = true

[securitygroup]
enable_security_group = true
firewall_driver = openvswitch
#firewall_driver = iptables_hybrid

sysctl net.bridge.bridge-nf-call-iptables
sysctl net.bridge.bridge-nf-call-ip6tables

 

cd /etc/nova

vi nova.conf

[neutron]
auth_url = http://con01:5000
auth_type = password
project_domain_name = Default
user_domain_name = Default
region_name = RegionOne
project_name = service
username = neutron
password = openstack

 

OpenVSwicth 설정

ovs-vsctl add-br br-provider

vi /etc/netplan/00-installer-config.yaml 
# This is the network config written by 'subiquity'
network:
  version: 2
  ethernets:
    enp15s0f0:
      dhcp4: true
    enp15s0f1:
      dhcp4: true
    enp15s0f2:
      dhcp4: true
    enp15s0f3:
      dhcp4: false
    enp14s0:
      addresses:
        - 10.0.0.3/24
  bridges:
    br-provider:
      interfaces:
        - enp15s0f3
      addresses:
        - ~~~~~~~~~~~~~
      nameservers:
        addresses:
          - ~~~~~~~~~~~~~
          - ~~~~~~~~~~~~~
      routes:
        - to: default
          via: ~~~~~~~~~~~~~
  
# 인터넷 연결 해제됨 주의!
netplan apply 

ovs-vsctl add-port br-provider enp15s0f3

netplan apply

 

cd plugins/ml2/

vi openvswitch_agent.ini
[ovs]
bridge_mappings = provider:br-provider

 

service nova-compute restart
service neutron-openvswitch-agent restart

 

 

마무리

다음 글에서는 오픈스택 가상머신에 어떻게 GPU를 연결해 줄 수 있는지 알아보고 설정해보겠다.

profile
NOHHYEONGJUN
Cloud/DevOps & Network Virtualization에 관심 있는 Engineer입니다. 🐳⚓️👨‍✈️
이전 포스트

실제 물리 서버를 이용한 OpenStack 구축(16) : Grafana + Prometheus를 이용한 모니터링 구축

다음 포스트

GPU Instance 생성을 위한 GPU Passtrough 설정

0개의 댓글