[Ansible] Kubernetes 1.21 Setup Using Ansible and Vagrant

gununoo·2022년 9월 28일
1

Ansible

목록 보기
3/4
post-thumbnail

참고:
https://kubernetes.io/blog/2019/03/15/kubernetes-setup-using-ansible-and-vagrant/
https://github.com/lvthillo/vagrant-ansible-kubernetes
https://github.com/ptah0414/vagrant-ansible-kubernetes

.
├── kubernetes-setup
│   ├── join-command # 배포 후 생성되는 파일(k8s-master가 생성)
│   ├── master-playbook.yml
│   └── node-playbook.yml
├── initiate.sh
└── Vagrantfile

[Prerequisite] Ansible, Vagrant 설치

Ansible 설치

  • Ansible 설치 
yum -y install epel-release 
yum -y install ansible
  • Ansible 설치 확인 
ansible --version

Vagrant 설치

  • vagrant repo 추가 
yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
  • vagrant 및 관련 패키지 설치 
yum -y install libvirt-daemon-kvm libvirt-client vagrant gcc-c++ make libstdc++-devel libvirt-devel
  • libvirt 재시작
systemctl restart libvirtd
  • vagrant-libvirt 플러그인 설치 
vagrant plugin install vagrant-libvirt

Vagrantfile, ansible-playbook 구성

Vagrantfile

IMAGE_NAME = "generic/ubuntu2004"
N = 3

Vagrant.configure("2") do |config|
    config.ssh.insert_key = false

    config.vm.provider "libvirt" do |v|
        v.memory = 8192
        v.cpus = 4
    end

    config.vm.define "k8s-master" do |master|
        master.vm.box = IMAGE_NAME
        master.vm.network "private_network", ip: "10.10.10.10"
        master.vm.network "public_network", :dev => "br0", :type =>"bridge"
        master.vm.hostname = "k8s-master"
        master.vm.provision "ansible" do |ansible|                              #1
           ansible.playbook = "kubernetes-setup/master-playbook.yml"            #1
            ansible.extra_vars = {                                              #1
                node_ip: "10.10.10.10",                                         #1
            }                                                                   #1
        end                                                                     #1
    end

    (1..N).each do |i|
        config.vm.define "node-#{i}" do |node|
            node.vm.box = IMAGE_NAME
            node.vm.network "private_network", ip: "10.10.10.#{i + 10}"
            node.vm.network "public_network", :dev => "br0", :type =>"bridge"
            node.vm.hostname = "node-#{i}"
#            node.vm.provision "ansible" do |ansible|                           #2
#                ansible.playbook = "kubernetes-setup/node-playbook.yml"        #2
#                ansible.extra_vars = {                                         #2
#                    node_ip: "10.10.10.#{i + 10}",                             #2
#                }                                                              #2
#            end                                                                #2
        end
    end
end
  • 안정적인 설치를 위해, k8s-master 먼저 프로비전한 후에 node를 프로비전 할 것이다.
  • 퍼블릭 네트워크(br0 인터페이스 사용)를 생성하여 사용자가 접속할 수 있도록 한다.
  • k8s-master의 ip는 10.10.10.10, 노드의 ip는 각각 10.10.10.11, 10.10.10.12, 10.10.10.13이다.
  • 본인의 pc의 사양에 따라, v.memory와 v.cpu를 조정하면 된다.
  • 최소 요구 사항: v.memory = 2048, v.cpu = 2

br0 인터페이스 참고: https://velog.io/@ptah0414/Ansible-Using-Ansible-with-Vagrant-2#vagrant-vm-네트워크-구성

initiate.sh

#!/bin/bash 

# VM 생성 및 k8s-master 프로비전 
vagrant destroy -f # 기존에 생성된 VM이 있다면 제거 
vagrant up # VM 생성 및 k8s-master 프로비전

# node 프로비전 
sed -i '/#1/s/^/#/' Vagrantfile # k8s-master에 주석 처리
sed -i '/#2/s/^#//' Vagrantfile # node에 주석 해제
vagrant provision # node 프로비전 

# 클러스터 생성 확인
vagrant ssh k8s-master -- -t 'kubectl get nodes && kubectl get pods -n kube-system' # 쿠버네티스 클러스터 생성 확인, kube-system pods 동작 확인 

# Vagrantfile 주석 되돌리기 
sed -i '/#1/s/^#//' Vagrantfile # k8s-master에 주석 처리 
sed -i '/#2/s/^/#/' Vagrantfile # node에 주석 처리

# k8s-master 접속 
vagrant ssh k8s-master

설치 과정을 자동화하기 위한 쉘 스크립트

master-playbook.yml

---
# 2.1 Install Docker and its dependent components. 
- hosts: all
  become: true
  tasks:
  - name: Install packages that allow apt to be used over HTTPS
    apt:
      name: "{{ packages }}"
      state: present
      update_cache: yes
    vars:
      packages:
      - apt-transport-https
      - ca-certificates
      - curl
      - gnupg-agent
      - software-properties-common

  - name: Add an apt signing key for Docker
    apt_key:
      url: https://download.docker.com/linux/ubuntu/gpg
      state: present

  - name: Add apt repository for stable version
    apt_repository:
      repo: deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable
      state: present

  - name: Install docker and its dependecies
    apt: 
      name: "{{ packages }}"
      state: present
      update_cache: yes
    vars:
      packages:
      - docker-ce 
      - docker-ce-cli 
      - containerd.io
      - python3-pip # for docker hub login 
    notify:
      - docker status

  - name: Add vagrant user to docker group
    user:
      name: vagrant
      group: docker

# docker login 
  - name: Install docker
    ansible.builtin.pip:
      name: docker
      executable: pip3

  - name: Log into DockerHub
    docker_login:
      username: [your username, not email address]
      password: [your password]

# 2.2 Kubelet will not start if the system has swap enabled, so we are disabling swap using the below code.
  - name: Remove swapfile from /etc/fstab
    mount:
      name: "{{ item }}"
      fstype: swap
      state: absent
    with_items:
      - swap
      - none

  - name: Disable swap
    command: swapoff -a
    when: ansible_swaptotal_mb > 0

# 2.3 Installing kubelet, kubeadm and kubectl using the below code. 
  - name: Add an apt signing key for Kubernetes
    apt_key:
      url: https://packages.cloud.google.com/apt/doc/apt-key.gpg
      state: present

  - name: Adding apt repository for Kubernetes
    apt_repository:
      repo: deb https://apt.kubernetes.io/ kubernetes-xenial main
      state: present
      filename: kubernetes.list

  - name: Install Kubernetes binaries
    apt: 
      name: "{{ packages }}"
      state: present
      update_cache: yes
    vars:
      packages:
        - kubelet=1.21.0-00
        - kubeadm=1.21.0-00
        - kubectl=1.21.0-00

  - name: Configure node ip
    lineinfile:
      path: /etc/default/kubelet
      line: KUBELET_EXTRA_ARGS=--node-ip={{ node_ip }}
      create: yes 

  - name: Restart kubelet
    service:
      name: kubelet
      daemon_reload: yes
      state: restarted

# 2.3 Initialize the Kubernetes cluster with kubeadm using the below code (applicable only on master node). 
  - name: Initialize the Kubernetes cluster using kubeadm
    command: kubeadm init --apiserver-advertise-address="10.10.10.10" --apiserver-cert-extra-sans="10.10.10.10"  --node-name k8s-master --pod-network-cidr=10.10.0.0/16

# 2.4 Setup the kube config file for the vagrant user to access the Kubernetes cluster using the below code. 
  - name: Setup kubeconfig for vagrant user
    command: "{{ item }}"
    with_items:
     - mkdir -p /home/vagrant/.kube
     - cp -i /etc/kubernetes/admin.conf /home/vagrant/.kube/config
     - chown vagrant:vagrant /home/vagrant/.kube/config

# 2.5 Setup the container networking provider and the network policy engine using the below code. 
  - name: Install calico pod network
    become: false
    command: kubectl create -f https://docs.projectcalico.org/manifests/calico.yaml

# 2.6 Generate kube join command for joining the node to the Kubernetes cluster and store the command in the file named join-command. 
  - name: Generate join command
    command: kubeadm token create --print-join-command
    register: join_command

  - name: Copy join command to local file
    local_action: copy content="{{ join_command.stdout_lines[0] }}" dest="./join-command"

# 2.7 Setup a handler for checking Docker daemon using the below code.
  handlers:
    - name: docker status
      service: name=docker state=started
  • docker를 runtime으로 사용하기 위해, kubernetes 버전을 1.21로 한다.
  • k8s-master가 토큰이 담긴 join-command 파일을 생성하면, node들이 해당 파일을 실행시켜 kubernetes cluster에 join 된다.
  • docker login을 안 하면, 하루 최대 pull request 한도를 초과했다면 calico 설치가 안 된다. 따라서 docker hub login을 추가하였다.

node-playbook.yml

---
# 2.1 Install Docker and its dependent components. 
- hosts: all
  become: true
  tasks:
  - name: Install packages that allow apt to be used over HTTPS
    apt:
      name: "{{ packages }}"
      state: present
      update_cache: yes
    vars:
      packages:
      - apt-transport-https
      - ca-certificates
      - curl
      - gnupg-agent
      - software-properties-common

  - name: Add an apt signing key for Docker
    apt_key:
      url: https://download.docker.com/linux/ubuntu/gpg
      state: present

  - name: Add apt repository for stable version
    apt_repository:
      repo: deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable
      state: present

  - name: Install docker and its dependecies
    apt: 
      name: "{{ packages }}"
      state: present
      update_cache: yes
    vars:
      packages:
      - docker-ce 
      - docker-ce-cli 
      - containerd.io
    notify:
      - docker status

  - name: Add vagrant user to docker group
    user:
      name: vagrant
      group: docker

# 2.2 Kubelet will not start if the system has swap enabled, so we are disabling swap using the below code.
  - name: Remove swapfile from /etc/fstab
    mount:
      name: "{{ item }}"
      fstype: swap
      state: absent
    with_items:
      - swap
      - none

  - name: Disable swap
    command: swapoff -a
    when: ansible_swaptotal_mb > 0

# 2.3 Installing kubelet, kubeadm and kubectl using the below code. 
  - name: Add an apt signing key for Kubernetes
    apt_key:
      url: https://packages.cloud.google.com/apt/doc/apt-key.gpg
      state: present

  - name: Adding apt repository for Kubernetes
    apt_repository:
      repo: deb https://apt.kubernetes.io/ kubernetes-xenial main
      state: present
      filename: kubernetes.list

  - name: Install Kubernetes binaries
    apt: 
      name: "{{ packages }}"
      state: present
      update_cache: yes
    vars:
      packages:
        - kubelet=1.21.0-00
        - kubeadm=1.21.0-00
        - kubectl=1.21.0-00

  - name: Configure node ip
    lineinfile:
      path: /etc/default/kubelet
      line: KUBELET_EXTRA_ARGS=--node-ip={{ node_ip }}
      create: yes 

  - name: Restart kubelet
    service:
      name: kubelet
      daemon_reload: yes
      state: restarted

# 3.2 Join the nodes to the Kubernetes cluster using below code. 
  - name: Copy the join command to server location
    copy: src=join-command dest=/tmp/join-command.sh mode=0777

  - name: Join the node to cluster
    command: sh /tmp/join-command.sh

# 2.7: Setup a handler for checking Docker daemon using the below code. 
  handlers:
    - name: docker status
      service: name=docker state=started

VM 배포 및 프로비전

./initiate.sh
profile
gununoo
take a look
다음 포스트

[Ansible] Reduce Skipping

0개의 댓글

관련 채용 정보