AWS OpenSearch 스냅샷 S3 레포지토리 등록

1. OpenSearch에서 S3로 데이터 삽입 권한 역할 생성

• TheSnapshotRole

  {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Effect": "Allow",
              "Action": [
                  "s3:ListBucket"
              ],
              "Resource": [
                  "arn:aws:s3:::{S3_BUCKET_NAME}"
              ]
          },
          {
              "Effect": "Allow",
              "Action": [
                  "s3:GetObject",
                  "s3:PutObject",
                  "s3:DeleteObject"
              ],
              "Resource": [
                  "arn:aws:s3:::{S3_BUCKET_NAME}/*"
              ]
          }
      ]
  }

2. OpenSearch에 TheSnapshotRole 를 전달할 iam:PassRole 권한 역할 생성

• ThePassRole

  {
  	"Version": "2012-10-17",
  	"Statement": [
  		{
  			"Effect": "Allow",
  			"Action": [
  				"es:ESHttpGet",
  				"es:ESHttpPut",
  				"es:ESHttpPost",
  				"es:ESHttpDelete"
  			],
  			"Resource": "{OPENSEARCH_ARN}/*"
  		},
  		{
  			"Effect": "Allow",
  			"Action": "iam:PassRole",
  			"Resource": "{TheSnapshotRole_ARN}"
  		}
  	]
  }

3. ThePassRole 를 OpenSearch 역할에 매핑

1. OpenSearch 대시보드 > Management > Security > ****Roles 에서 manage_snapshots 역할 선택
2. manage_snapshots 상세 > Mapped users > Manage mapping > Backend roles에서 ThePassRole 의 ARN 등록

4. s3 레포지토리 등록 요청

PUT _snapshot/{S3_REPOSITORY_NAME}
{
  "type": "s3",
  "settings": {
    "bucket": "{S3_BUCKET_NAME}",
    "base_path": "{S3_OBJECT_PATH}",
    "region": "ap-northeast-2",
    "role_arn": "{TheSnapshotRole_ARN}",
    "compress": true
  }
}

• 해당 요청을 OpenSearch 대시보드 Dev tools에서 요청시 에러 발생

  {"Message":"User: anonymous is not authorized to perform: iam:PassRole on resource:
  arn:aws:iam::123456789012:role/TheSnapshotRole"}

• 따라서 python 코드나 postman과 같은 방법으로 인증이 서명된 요청을 전송해야함
  • python sample

    import boto3
    import requests
    from requests_aws4auth import AWS4Auth
    
    host = '' # domain endpoint
    region = '' # e.g. us-west-1
    service = 'es'
    credentials = boto3.Session().get_credentials()
    awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token)
    
    # Register repository
    
    path = '/_snapshot/my-snapshot-repo-name' # the OpenSearch API endpoint
    url = host + path
    
    payload = {
      "type": "s3",
      "settings": {
        "bucket": "amzn-s3-demo-bucket",
        "base_path": "my/snapshot/directory",
        "region": "us-west-1",
        "role_arn": "arn:aws:iam::123456789012:role/snapshot-role"
      }
    }
    
    headers = {"Content-Type": "application/json"}
    
    r = requests.put(url, auth=awsauth, json=payload, headers=headers)
    
    print(r.status_code)
    print(r.text)
    
    # # Take snapshot
    #
    # path = '/_snapshot/my-snapshot-repo-name/my-snapshot'
    # url = host + path
    #
    # r = requests.put(url, auth=awsauth)
    #
    # print(r.text)
    #
    # # Delete index
    #
    # path = 'my-index'
    # url = host + path
    #
    # r = requests.delete(url, auth=awsauth)
    #
    # print(r.text)
    #
    # # Restore snapshot (all indexes except Dashboards and fine-grained access control)
    #
    # path = '/_snapshot/my-snapshot-repo-name/my-snapshot/_restore'
    # url = host + path
    #
    # payload = {
    #   "indices": "-.kibana*,-.opendistro_security,-.opendistro-*",
    #   "include_global_state": False
    # }
    #
    # headers = {"Content-Type": "application/json"}
    #
    # r = requests.post(url, auth=awsauth, json=payload, headers=headers)
    #
    # print(r.text)
    # 
    # # Restore snapshot (one index)
    #
    # path = '/_snapshot/my-snapshot-repo-name/my-snapshot/_restore'
    # url = host + path
    #
    # payload = {"indices": "my-index"}
    #
    # headers = {"Content-Type": "application/json"}
    #
    # r = requests.post(url, auth=awsauth, json=payload, headers=headers)
    #
    # print(r.text)

5. 등록된 s3 레포지토리 확인

GET _snapshot/{S3_REPOSITORY_NAME}

refo

수동 스냅샷 리포지토리 등록

Amazon OpenSearch Service에서 인덱스 스냅샷 생성