Overview

• 저장된 모델을 이용해 API를 생성하는 방법

Secret

apiVersion: v1
type: Opaque
kind: Secret
metadata:
  name: seldon-init-container-secret
  namespace: kubeflow-user-example-com
data:
  AWS_ACCESS_KEY_ID: bWluaW8K=
  AWS_SECRET_ACCESS_KEY: bWluaW8xMjM=
  AWS_ENDPOINT_URL: aHR0cDovL21pbmlvLm1ha2luYXJvY2tzLmFp
  USE_SSL: ZmFsc2U=

• initContainer가 minio 접근 후 모델을 다운로드 받으려면 credentials가 필요하다. minio에 접근하기 위한 credentials는 위와 같다.

AWS_ACCESS_KEY_ID: minio
AWS_SECRET_ACCESS_KEY: minio123
AWS_ENDPOINT_URL: http://minio-service.kubeflow.svc:9000
USE_SSL: false

• data에 입력되어야하는 값들

echo -n minio | base64

• secret에 입력되는 값은 인코딩된 값이어야한다.

AWS_ACCESS_KEY_ID: bWluaW8=
AWS_SECRET_ACCESS_KEY: bWluaW8xMjM=
AWS_ENDPOINT_URL: aHR0cDovL21pbmlvLXNlcnZpY2Uua3ViZWZsb3cuc3ZjOjkwMDA=
USE_SSL: ZmFsc2U=

• 수행 후 적용을 준비한다.

cat <<EOF > seldon-init-container-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: seldon-init-container-secret
  namespace: kubeflow-user-example-com
type: Opaque
data:
  AWS_ACCESS_KEY_ID: bWluaW8=
  AWS_SECRET_ACCESS_KEY: bWluaW8xMjM=
  AWS_ENDPOINT_URL: aHR0cDovL21pbmlvLXNlcnZpY2Uua3ViZWZsb3cuc3ZjOjkwMDA=
  USE_SSL: ZmFsc2U=
EOF

• secret을 생성할 수 있는 yaml file 생성 script

kubectl apply -f seldon-init-container-secret.yaml

• secret 생성

Seldon Core yaml

apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: seldon-example
  namespace: kubeflow-user-example-com
spec:
  name: model
  predictors:
  - name: model

    componentSpecs:
    - spec:
        volumes:
        - name: model-provision-location
          emptyDir: {}

        initContainers:
        - name: model-initializer
          image: gcr.io/kfserving/storage-initializer:v0.4.0
          args:
            - "s3://mlflow/mlflow/artifacts/0/74ba8e33994144f599e50b3be176cdb0/artifacts/svc"
            - "/mnt/models"
          volumeMounts:
          - mountPath: /mnt/models
            name: model-provision-location
          envFrom:
          - secretRef:
              name: seldon-init-container-secret

        containers:
        - name: model
          image: ghcr.io/mlops-for-all/mlflowserver
          volumeMounts:
          - mountPath: /mnt/models
            name: model-provision-location
            readOnly: true
          securityContext:
            privileged: true
            runAsUser: 0
            runAsGroup: 0

    graph:
      name: model
      type: MODEL
      parameters:
      - name: model_uri
        type: STRING
        value: "/mnt/models"
      children: []

• 이전 Section과 다른 점은 initContainer에 envFrom 필드가 추가되었고, args 주소가 s3://mlflow/mlflow/artifacts/0/74ba8e33994144f599e50b3be176cdb0/artifacts/svc로 바뀌었다.
• args의 첫 인자는 다운로드 받을 모델의 경로라고 했다.
  • How to know that ?

• mlflow에 들어가서 run 클릭 후 모델을 누르면 위 처럼 확인할 수 있다.
• 확인된 경로를 입력하면 된다.

API 생성

cat <<EOF > seldon-mlflow.yaml
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: seldon-example
  namespace: kubeflow-user-example-com
spec:
  name: model
  predictors:
  - name: model

    componentSpecs:
    - spec:
        volumes:
        - name: model-provision-location
          emptyDir: {}

        initContainers:
        - name: model-initializer
          image: gcr.io/kfserving/storage-initializer:v0.4.0
          args:
            - "s3://mlflow/mlflow/artifacts/0/74ba8e33994144f599e50b3be176cdb0/artifacts/svc"
            - "/mnt/models"
          volumeMounts:
          - mountPath: /mnt/models
            name: model-provision-location
          envFrom:
          - secretRef:
              name: seldon-init-container-secret

        containers:
        - name: model
          image: ghcr.io/mlops-for-all/mlflowserver
          volumeMounts:
          - mountPath: /mnt/models
            name: model-provision-location
            readOnly: true
          securityContext:
            privileged: true
            runAsUser: 0
            runAsGroup: 0

    graph:
      name: model
      type: MODEL
      parameters:
      - name: model_uri
        type: STRING
        value: "/mnt/models"
      - name: xtype
        type: STRING
        value: "dataframe"
      children: []
EOF

kubectl apply -f seldon-mlflow.yaml

• seldon pod 생성

kubectl get po -n kubeflow-user-example-com | grep seldon

• 정상 작동 확인

curl -X POST http://$NODE_IP:$NODE_PORT/seldon/seldon-deploy/sklearn/api/v1.0/predictions \
-H 'Content-Type: application/json' \
-d '{
    "data": {
        "ndarray": [
            [
                143.0,
                0.0,
                30.0,
                30.0
            ]
        ],
        "names": [
            "sepal length (cm)",
            "sepal width (cm)",
            "petal length (cm)",
            "petal width (cm)"
        ]
    }
}'

• CLI를 이용해 생성된 API로 Request