Vault 설치 및 사용하기

! (docker 환경에 맞게 명령어 바꿔서 사용 해야 합니다.)

ex) vault 상태 확인
vault status -> docker exec -it vault sh -c 'export VAULT_TOKEN="root" && vault status'

! (대부분의 작업을 Vault UI 에서 수행 할 수 있습니다.)

ex) secret 추가, 정책 추가, 사용자 인증 설정, 사용자 추가, etc 등

INSTALL VAULT

Step 1: Install Docker and Docker Compose

• download and install it from Docker's official website. Docker Compose is included with Docker Desktop.

Step 2: Create a Docker Compose File

mkdir vault-docker
cd vault-docker

version: '3.7'
services:
  vault:
    image: vault:1.13.2  # Specify a known version
    container_name: vault
    restart: unless-stopped
    environment:
      VAULT_DEV_ROOT_TOKEN_ID: root
      VAULT_DEV_LISTEN_ADDRESS: "0.0.0.0:8200"
    ports:
      - "8200:8200"
    cap_add:
      - IPC_LOCK
    volumes:
      - ./vault-data:/vault/file
    command: vault server -dev -dev-root-token-id="root"

Step 3: Start Vault with Docker Compose

1. Start the Vault Container:
   Navigate to the directory where your docker-compose.yml file is located and run:

docker-compose up -d

Step 4: Access Vault

1. Access the Vault UI:
   Open your browser and navigate to http://localhost:8200/ui. Since you started Vault in development mode, you can log in using the root token (root by default).

BASIC CONFIGURATION

• 협업 작업을 위한 secret 공통 사용 설정
• username 으로 로그인 하는 경우, default로 user 개인의 secret만 관리하도록 설정 되어 있음

Step 1: Set Up Policies

To give a user broader access, such as viewing secrets stored in the secret/ path, you need to create a policy that allows access to those secrets.

1. Create a Policy File: For example, create a file called secret-policy.hcl:

• docker-compose project root/config/secret-policy.hcl

path "secret/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

2. Mount a config dir on docker-compose.yml

version: '3.7'
services:
  vault:
    image: vault:1.13.2
    container_name: vault
    ports:
      - "8200:8200"
    environment:
      VAULT_DEV_ROOT_TOKEN_ID: root
      VAULT_TOKEN: "root"  # Add this line for set up env VAULT_TOKEN on bash of container
    cap_add:
      - IPC_LOCK
    volumes:
      - ./vault-data:/vault/file
      - ./:/vault/config  # This mounts the current directory
    command: vault server -dev -dev-root-token-id="root"

3. Execute the following command to upload the policy to Vault:

docker exec -it vault vault policy write secret-policy /vault/config/secret-policy.hcl

Step 2: Enable UserPass Authentication

1. Enable Userpass Authentication
   If you haven't already enabled Userpass authentication, do so with the following command:

vault auth enable userpass

2. Generate Users
   If you're using the userpass authentication method, assign the policy to the user:

# one by one user
vault write auth/userpass/users/your-username password="yourPassword" policies="default,secret-policy"

# to all uesrs => List of users to assign the policy to
# how to give each of them qunique password?
# shell file

Option 1: Associative Array (Bash 4.0+)

declare -A USERS_PASSWORDS

# Define users and their unique passwords
USERS_PASSWORDS=(
  ["user1"]="password1"
  ["user2"]="password2"
  ["user3"]="password3"
)

for user in "${!USERS_PASSWORDS[@]}"; do
  password="${USERS_PASSWORDS[$user]}"
  vault write auth/userpass/users/$user password="$password" policies="default,secret-policy"
done

Option 2: Parallel Arrays (Bash Pre-4.0)

USERS=("user1" "user2" "user3")
PASSWORDS=("password1" "password2" "password3")

for i in "${!USERS[@]}"; do
  user="${USERS[$i]}"
  password="${PASSWORDS[$i]}"
  vault write auth/userpass/users/$user password="$password" policies="default,secret-policy"
done

Option 3: Generate Random Passwords

USERS=("user1" "user2" "user3")

for user in "${USERS[@]}"; do
  password=$(openssl rand -base64 12)  # Generate a random 12-character password
  echo "Creating user $user with password $password"
  vault write auth/userpass/users/$user password="$password" policies="default,secret-policy"
done

INTEGRATE WITH Django Rest Framework, Vault-Agent

Step 1: Run Vault in production mode

(development mode and production mode may works different ways)

1. Run vault in production mode for

version: '3.7'
services:
  vault:
    image: vault:1.13.2  # Specify a known version
    container_name: vault
    restart: unless-stopped
    environment:
      VAULT_ADDR: "http://127.0.0.1:8200"  # Add this line for http access on dev mode
      VAULT_TOKEN: "root"  # Add this line for set up env VAULT_TOKEN on bash of container
    ports:
      - "8200:8200"
    cap_add:
      - IPC_LOCK
    volumes:
      - ./vault-data:/vault/file # Persistent volume for Vault data
      - ./config:/vault/config  # This mounts the current directory
    command: vault server -config=/vault/config/vault.hcl  # Use production config here
    networks:
      - vault-network

volumes:
  vault-data:
    external: false  # Persistent volume for Vault data

networks:
  vault-network:
    external: true

2. Start Vault docker-compose

docker-compose up -d

3. Initialize Vault

docker exec -it vault vault operator init

• This will generate unseal keys and a root token. Store them securely.
• Unseal Keys: You will need these keys to unseal Vault after restarts.
• Root Token: This token is required for administrative access.

4. Unseal Vault
   After initialization, you need to unseal Vault using the generated unseal keys:

docker exec -it vault vault operator unseal <unseal_key_1>
docker exec -it vault vault operator unseal <unseal_key_2>
docker exec -it vault vault operator unseal <unseal_key_3>

Step 2: Basic Configuration for Vault

Set Up Policies

To give a user broader access, such as viewing secrets stored in the secret/ path, you need to create a policy that allows access to those secrets.

1. Create a Policy File: For example, create a file called secret-policy.hcl:
   vault-root/congig/secret-policy.hcl

path "secret/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

2. Upload the Policy to Vault:

vault policy write secret-policy secret-policy.hcl

3. Enable Userpass Authentication
   If you haven't already enabled Userpass authentication, do so with the following command:

vault auth enable userpass

Step 3: Set Up Vault for AppRole Authentication

You need to configure Vault to support the AppRole authentication method and create a role that Vault Agent can use to authenticate.

1. Set Up Vault Policies and Roles
   Ensure that you have configured Vault with the necessary secrets, policies, and authentication methods. Here's a basic outline of the steps required:

vault kv put secret/my-django-secret secret_key="your-django-secret-key"
vault kv put secret/db-creds db_name="your-db-name" username="your-username" password="your-password" host="localhost" port="5432"

2. Create a Policy:

# root/config/secret-agent-policy.hcl
path "secret/*" {
  capabilities = ["read", "list"]
}

3.Upload the policy:

vault policy write secret-policy secret-policy.hcl

4. Enable an AppRole:

vault auth enable approle

5. Create an AppRole:

vault write auth/approle/role/my-role policies="secret-policy"
vault read auth/approle/role/my-role/role-id
vault write -f auth/approle/role/my-role/secret-id

Use the generated role_id and secret_id to authenticate Vault Agent.

Step 4: Create Vault-Agent

1. Create Vaul-agent

mkdir vault-agent
vault-agent/docker-compose.yml

version: '3.7'

services:
  vault-agent:
    image: vault:1.13.2
    container_name: vault-agent
    cap_add:
      - IPC_LOCK  # Adding IPC_LOCK capability to allow memory locking
    volumes:
      - ./vault-agent-config:/etc/vault-agent
      - ../../developments/sample_project/django_rest_framework/project/config:/django_rest_framework/project/config  # This mounts the DRF config directory
      - ./role-id:/etc/vault-agent/role-id  # Mount the role-id file
      - ./secret-id:/etc/vault-agent/secret-id:rw # Mount the secret-id file # Ensure it's mounted as read-write
    command: vault agent -config=/etc/vault-agent/agent.hcl
    networks:
      - vault-network

networks:
  vault-network:
    external: true

2. Vault Agent Configuration

n the vault-agent-config/ directory, create a configuration file for Vault Agent. This file will define how Vault Agent authenticates with Vault and where to render the secrets.

vault-agent/vault-agent-config/agent.hcl

auto_auth {
  method "approle" {
    mount_path = "auth/approle"
    config = {
      role_id_file_path = "/etc/vault-agent/role-id"
      secret_id_file_path = "/etc/vault-agent/secret-id"
      remove_secret_id_file_after_reading = false
    }
  }
}

template {
  source      = "/etc/vault-agent/templates/settings.tpl"
  destination = "/app/project/config/settings_vault.py"
}

vault {
  address = "http://vault:8200"
}

# Enabling the agent to dynamically update secrets and keep running
exit_after_auth = false

3. make role-id file, secret-id file
   write role-id that you generate '5. Create an AppRole:' part
   vault-agent/role-id

nano role-id

vault-agent/secret-id

nano secret-id

4. Create Vault Agent Template File

he template file instructs Vault Agent on how to retrieve secrets from Vault and how to render them into a configuration file.

In the vault-agent-config/templates/ directory, create the following template file:

vault-agent/vault-agent-config/templates/settings.tpl

# Django settings from Vault
SECRET_KEY = "{{ with secret "secret/my-django-secret" }}{{ .Data.data.secret_key }}{{ end }}"
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': "{{ with secret "secret/db-creds" }}{{ .Data.data.db_name }}{{ end }}",
        'USER': "{{ with secret "secret/db-creds" }}{{ .Data.data.username }}{{ end }}",
        'PASSWORD': "{{ with secret "secret/db-creds" }}{{ .Data.data.password }}{{ end }}",
        'HOST': "{{ with secret "secret/db-creds" }}{{ .Data.data.host }}{{ end }}",
        'PORT': "{{ with secret "secret/db-creds" }}{{ .Data.data.port }}{{ end }}",
    }
}

5. example directory structure layout

vault-agent-docker/
├── vault-agent-config/
│   ├── templates/
│   │   └── settings.tpl
│   ├── agent.hcl
│   ├── role-id
│   └── secret-id
├── docker-compose.agent.yml
├── role-id
└── secret-id

Step 5: Integrate Django Rest Framework

To determine where to place the DRF configuration directory (where Vault Agent will render secrets), let’s consider the following:

1. What Vault Agent Will Render: Vault Agent will render secrets, such as Django settings (e.g., SECRET_KEY, database credentials), into a file. This file needs to be accessible by your Django application.

2. Best Location for Django Settings: Django settings are typically placed in the settings.py file located in the project/ directory (based on your structure). Therefore, it makes sense to place the rendered configuration file close to where settings.py is located.

Recommendation
Place the DRF config directory that Vault Agent will use inside the project/ directory where your settings.py is located. This ensures that your Django project can easily import the rendered settings from Vault Agent.

Directory Structure Update

1. Create a config/ directory inside project/:

Add a new directory called config/ under the project/ directory. This will be where Vault Agent renders secrets.

2. Mount the project/config/ directory in your Docker Compose setup:

In your docker-compose.yml, ensure that this directory is mounted so that Vault Agent can write to it.

Updated Directory Structure Example

sample_project/
├── django_rest_framework/
│   ├── app/
│   ├── media/
│   ├── project/
│   │   ├── __init__.py
│   │   ├── asgi.py
│   │   ├── settings.py
│   │   ├── urls.py
│   │   ├── wsgi.py
│   │   ├── config/  # <--- New directory for Vault Agent-rendered secrets
│   ├── static/
│   ├── user/
│   ├── venv/
│   ├── Dockerfile
│   ├── manage.py
│   ├── requirements.txt
├── nginx/
│   ├── conf.d/
│   ├── Dockerfile
│   ├── nginx.conf
│   ├── .env.local
│   └── docker-compose.local.yml

• project/config/: Vault Agent will render secrets into this directory (e.g., settings_vault.py), and your Django application will read the configuration from here.

Example Docker Compose Volume Mount

In your docker-compose.yml file for DRF, make sure you mount the project/config/ directory so that Vault Agent can write to it:

services:
  web:
    image: django:latest
    container_name: drf
    volumes:
      - ./project/config:/app/project/config  # Mount the config directory
    command: sh -c "python manage.py runserver 0.0.0.0:8000"
    ports:
      - "8000:8000"
    networks:
      - vault-network

Importing the Rendered Settings in Django

In your settings.py file, import the secrets rendered by Vault Agent:

try:
    from .config.settings_vault import *
except ImportError:
    pass

This will ensure that your Django application picks up the secrets that Vault Agent renders in the config/ directory.

Network Configuration:

To allow communication between Vault, Vault Agent, and DRF, ensure they all use the same Docker network:

1. Create the Network:

docker network create vault-network

2. Connect the Services: Each service in its respective Docker Compose file should be connected to the vault-network.

3.Check Docker Networks:
Run the following command to check if both containers are in the same network:

docker network inspect vault-network

Ensure that all services such as vault, vault-agent, drf are listed as connected to this network.

Step 6: Start Service

1. docker-compose up services

path-to-vault/docker-compose -f docker-compose-vault.yml up -d
path-to-vault-agent/docker-compose -f docker-compose-vault-agent.yml up -d
path-to-drf/docker-compose -f docker-compose-drf.yml up -d

2. Check Containers Health:
   Ensure that the containers are running and accessible:

docker ps -a

Other Impormations

settings.tpl

# settings.tpl

# Django Secret Key
SECRET_KEY = "{{ with secret "secret/my-django-secret" }}{{ .Data.data.secret_key }}{{ end }}"

# Database Credentials
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': '{{ with secret "secret/db-creds" }}{{ .Data.data.db_name }}{{ end }}',
        'USER': '{{ with secret "secret/db-creds" }}{{ .Data.data.username }}{{ end }}',
        'PASSWORD': '{{ with secret "secret/db-creds" }}{{ .Data.data.password }}{{ end }}',
        'HOST': '{{ with secret "secret/db-creds" }}{{ .Data.data.host }}{{ end }}',
        'PORT': '{{ with secret "secret/db-creds" }}{{ .Data.data.port }}{{ end }}',
    }
}

# Variable![](https://velog.velcdn.com/images/sbkyo88/post/4ee7197c-c627-4bc7-b90c-0a21fc645034/image.png)
 Structure:
{{ with secret "path/to/secret" }}
<use the secret data here>
{{ end }}


# Understanding .Data and .Data.data
{
  "Data": {
    "data": {
      "var": "secret-value"
    },
    "metadata": {
      "created_time": "2024-08-22T03:52:00Z",
      "version": 1
    }
  }
}

Explanation of the Template Structure:

• with secret "path/to/secret": This block is used to fetch a secret from Vault. The path/to/secret is the path in Vault where the secret is stored.
• {{ .Data.data.secret_key }}: Inside the with block, you can access the secret data using the {{ .Data.data.key_name }} syntax. For example, if your secret is stored as secret_key under secret/my-django-secret, you would access it with {{ .Data.data.secret_key }}.
• Reusing the Secret Data: For retrieving multiple fields from the same secret (e.g., database credentials like username, password, etc.), you can repeat the with secret block for each field.