From Docker Swarm to Kubernetes

uuuu.jini·2022년 10월 13일

Installation

3 nodes running

Master
Worker 1
Worker 2

If you want to run your docker images during development you can use docker-compose to setup a small infrastructure. The effort to install kubernetes makes only sense if you want to run your applications in a potential productive environment.

> Disable Swap partition

Kubernetes don't allows swap partitions. Check your /etc/fstab file and uncomment the swap partition if available on your system.

> The Install Script

you can use the bash script for installation.

The script installs the following tools:

docker-ce (the docker engine)
docker-ce-cli (the docker command line interface)
containerd.io (the container runtime)
kubelet (the kubernetes node agent)
kubeadm (the kubernetes cluster tool)
kubectl (the kubernetes command line interface)

- setup.sh

#!/bin/sh
############################################################
# Kubernetes Install Script for Debian 9 (Stretch)
# 
# run as sudo 
############################################################

# determine if we run as sudo
userid="${SUDO_USER:-$USER}"
if [ "$userid" == 'root' ]
  then 
    echo "Please run the setup as sudo and not as root!"
    exit 1
fi
if [ "$EUID" -ne 0 ]
  then 
    echo "Please run setup as sudo!"
    exit 1
fi

echo "#############################################"
echo " adding repositories..."
echo "#############################################"
apt-get update
apt-get install -y apt-transport-https ca-certificates ne curl gnupg2 software-properties-common

# Add docker repositry
curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
add-apt-repository \
          "deb [arch=amd64] https://download.docker.com/linux/debian \
           $(lsb_release -cs) \
           stable"
           
# Add kubernetes repository           
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF | tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF


echo "#############################################"
echo " installing docker and kubernetes...."
echo "#############################################"
apt-get update
apt-get install -y docker-ce docker-ce-cli containerd.io kubelet kubeadm kubectl

#####################################################################################
# Kubernetes is now installed. To setup a new kubernetes cluster with a master node 
# run:
#  $ kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=[YOUR-NODE-IP-ADDRESS]
#
# This command will setup a new cluster. Follow the instructions of the output.
# The output will show also the command how to join a worker node.
# You can use this script also to install a worker node. 
#####################################################################################

run this script as sudo

sudo ./setup.sh

> Setup the Cluster

After you have installed the necessary libraries you can initialize the Kubernetes cluster using the following kubeadm command:

sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=[NODE_IP_ADDRESS]

Replace [NODE_IP_ADDRESS] with your servers public IP address.

You will see a detailed protocol showing what happens behind the scene. If something went wrong you can easily roll back everything with the command.

sudo kubeadm reset

The last output form the protocol shows you the join token needed to setup a worker node. If you forgot to note the join token run:

sudo kubeadm token create --print-join-command

> Setup kubectl on a Server

To make kubectl work for your non-root user, run these commands, which are also part of the kubeadm init output:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

This will copy the configuration of your master node into the kubernetes config directory ./kube of your home directory.

> Setup a flannel network

deploy the flannel network to the kubernetes cluster using the kubectl command.

CNI

Container Network Interface: 컨테이너 간의 네트워킹을 제어할 수 있는 플러그인을 만들기 위한 표준. Kubernetes Cluster 내부는 master Node에 의해 여러 컨테이너가 생성 삭제 복구를 반복하고 있다. 그에 따라 각 컨테이너의 고정적이지 않고 재할당이 빈번하다. 이러한 특징을 해결하기 위해 Kubernetes Cluster는 가상 네트워크가 구성되어 있는데 기본적으로는 Worker Node의 kube-proxy 가 네트워크를 관리하지만 보다 효율적인 네트워크 환경을 구성하기 위해 다양한 네트워크 관련 Addon이 제공된다.

Flannel network

서로 다른 노드에 있는 pod 간 통신을 완성하기 위해서는 관련 기능을 제공하는 CNI가 필요하다. Flannel은 대표적인 CNI 종류 중 하나이다. Flannel은 kubernetes 용으로 설계된 오버레이 네트워크로 L3네트워클르 구성하는 가장 간단하고 쉬운 방법을 제공한다.
kubernetes 에 Flannel을 적용하게 되면 모든 Node에서 flannel이라는 에이전트를 실행하고 각 node에 flannel.1 인터페이스를 할당한다. Flannel은 Kubernetes Api를 통해 etcd에 네트워크 구성, Pod에 할당된 IP 정보 등을 저장한다. Flannel은 VXLAN을 사용하여 원래의 패킷을 한번 더 감싸서 서로 다른 Node간 통신이 되도록 한다.

 kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

The flannel network will been deployed to the kubernetes cluster. After some seconds the cluster should be up and running. You can check the status with:

kubectl cluster-info

and list all nodes with:

kubectl get nodes

> Install Worker Nodes

Now you can run the same script used to install the master node on each of your worker nodes. This will install the docker runtime and kubernetes tools. To add the new node to your cluster created in the previous step run the join command from the master setup.If you do not know the join command you can run the following command on your mster node first:

kubeadm token create --print-join-command

RUn the output as a root user on your woker node:

sudo kubeadm join xxx.xxx.xxx.xxx:6443 --token xxx.xxxxxxxxx     --discovery-token-ca-cert-hash xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Controlling Your Cluster From your Workstation

In different to docker-swarm, a kubernetes cluster can be administrated remote from your workstation. The tool 'kubectl' is the kubernetes command line tool used to manage your cluster via the kubernetes api either from your server or from a workstation.

> Setup kubectl on Your Workstation

To run kubectl from your workstation you need first to install it. You will find the official install guide.

In order to get kubectl talking to your cluster, you can again copy the content from the administrator kubeconfig file(/etc/kubernetes/admin.conf) into your workstation.