HINT:

For ENDPOINT, use the value of the host variable in the Connect to this Kubeflow Pipelines instance from a Python client via Kubeflow Pipelines SDK section of the SETTINGS window.

For ARTIFACT_STORE_URI, copy the bucket name which starts with the qwiklabs-gcp-xx-xxxxxxx-kubeflowpipelines-default prefix from the previous cell output. Your copied value should look like 'gs://qwiklabs-gcp-xx-xxxxxxx-kubeflowpipelines-default'

Kubeflow Pipelines to create and orchestrate an ML pipeline on Cloud AI Platform.

1.

1.

1.

1.

1.

1.

Using custom containers with AI Platform Training

Welcome to Cloud Shell! Type "help" to get started.
Your Cloud Platform project in this session is set to qwiklabs-gcp-01-1b8e61d32ef6.
Use “gcloud config set project [PROJECT_ID]” to change to a different project.

# List the project ID : gcloud auth list
student_00_50d4eccd9197@cloudshell:~ (qwiklabs-gcp-01-1b8e61d32ef6)$ gcloud auth list
Credentialed Accounts

ACTIVE: *
ACCOUNT: student-00-50d4eccd9197@qwiklabs.net

To set the active account, run:
    $ gcloud config set account `ACCOUNT`


#  to set the project ID to your Google Cloud Project
student_00_50d4eccd9197@cloudshell:~ (qwiklabs-gcp-01-1b8e61d32ef6)$ export PROJECT_ID=$(gcloud config get-value core/project)
Your active configuration is: [cloudshell-2694]
student_00_50d4eccd9197@cloudshell:~ (qwiklabs-gcp-01-1b8e61d32ef6)$ gcloud config set project $PROJECT_ID
Updated property [core/project].


# To enable the required Cloud services, run the following commands
student_00_50d4eccd9197@cloudshell:~ (qwiklabs-gcp-01-1b8e61d32ef6)$ gcloud services enable \
> cloudbuild.googleapis.com \
> container.googleapis.com \
> cloudresourcemanager.googleapis.com \
> iam.googleapis.com \
> containerregistry.googleapis.com \
> containeranalysis.googleapis.com \
> ml.googleapis.com \
> dataflow.googleapis.com
Operation "operations/acf.p2-747557288131-6f7cef0b-695a-4148-8207-95aafb420e24" finished successfully.
student_00_50d4eccd9197@cloudshell:~ (qwiklabs-gcp-01-1b8e61d32ef6)$ PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format="value(projectNumber)")


# Add the Editor permission for your Cloud Build service account
student_00_50d4eccd9197@cloudshell:~ (qwiklabs-gcp-01-1b8e61d32ef6)$ CLOUD_BUILD_SERVICE_ACCOUNT="${PROJECT_NUMBER}@cloudbuild.gserviceaccount.com"
student_00_50d4eccd9197@cloudshell:~ (qwiklabs-gcp-01-1b8e61d32ef6)$ gcloud projects add-iam-policy-binding $PROJECT_ID \
>   --member serviceAccount:$CLOUD_BUILD_SERVICE_ACCOUNT \
>   --role roles/editor
Updated IAM policy for project [qwiklabs-gcp-01-1b8e61d32ef6].
bindings:
- members:
  - user:student-00-50d4eccd9197@qwiklabs.net
  role: roles/appengine.appAdmin
- members:
  - serviceAccount:qwiklabs-gcp-01-1b8e61d32ef6@qwiklabs-gcp-01-1b8e61d32ef6.iam.gserviceaccount.com
  - user:student-00-50d4eccd9197@qwiklabs.net
  role: roles/bigquery.admin
- members:
  - serviceAccount:747557288131@cloudbuild.gserviceaccount.com
  role: roles/cloudbuild.builds.builder
- members:
  - serviceAccount:service-747557288131@gcp-sa-cloudbuild.iam.gserviceaccount.com
  role: roles/cloudbuild.serviceAgent
- members:
  - serviceAccount:service-747557288131@compute-system.iam.gserviceaccount.com
  role: roles/compute.serviceAgent
- members:
  - user:student-00-50d4eccd9197@qwiklabs.net
  role: roles/container.admin
- members:
  - serviceAccount:service-747557288131@container-engine-robot.iam.gserviceaccount.com
  role: roles/container.serviceAgent
- members:
  - serviceAccount:747557288131-compute@developer.gserviceaccount.com
  - serviceAccount:747557288131@cloudbuild.gserviceaccount.com
  - serviceAccount:747557288131@cloudservices.gserviceaccount.com
  - user:student-00-50d4eccd9197@qwiklabs.net
  role: roles/editor
- members:
  - user:student-00-50d4eccd9197@qwiklabs.net
  role: roles/ml.admin
- members:
  - serviceAccount:service-747557288131@cloud-ml.google.com.iam.gserviceaccount.com
  role: roles/ml.serviceAgent
- members:
  - serviceAccount:admiral@qwiklabs-services-prod.iam.gserviceaccount.com
  - serviceAccount:qwiklabs-gcp-01-1b8e61d32ef6@qwiklabs-gcp-01-1b8e61d32ef6.iam.gserviceaccount.com
  - user:student-00-50d4eccd9197@qwiklabs.net
  role: roles/owner
- members:
  - serviceAccount:qwiklabs-gcp-01-1b8e61d32ef6@qwiklabs-gcp-01-1b8e61d32ef6.iam.gserviceaccount.com
  role: roles/storage.admin
- members:
  - user:student-00-50d4eccd9197@qwiklabs.net
  role: roles/viewer
- members:
  - serviceAccount:service-747557288131@gcp-sa-websecurityscanner.iam.gserviceaccount.com
  role: roles/websecurityscanner.serviceAgent
etag: BwXSUgSrMCo=
version: 1
student_00_50d4eccd9197@cloudshell:~ (qwiklabs-gcp-01-1b8e61d32ef6)$ 

Create an instance of AI Platform Pipelines

Create the required GKE cluster

student_00_50d4eccd9197@cloudshell:~ (qwiklabs-gcp-01-1b8e61d32ef6)$ gcloud container clusters create cluster-1 --zone us-central1-a --cluster-version 1.20.10 --machine-type n1-standard-2 --scopes=https://www.googleapis.com/auth/cloud-platform
WARNING: Currently VPC-native is the default mode during cluster creation for versions greater than 1.21.0-gke.1500. To create advanced routes based clusters, please pass the `--no-enable-ip-alias` flag
WARNING: Starting with version 1.18, clusters will have shielded GKE nodes by default.
WARNING: Your Pod address range (`--cluster-ipv4-cidr`) can accommodate at most 1008 node(s).
WARNING: Starting with version 1.19, newly created clusters and node-pools will have COS_CONTAINERD as the defaultnode image when no image type is specified.
Creating cluster cluster-1 in us-central1-a...done.     
Created [https://container.googleapis.com/v1/projects/qwiklabs-gcp-01-1b8e61d32ef6/zones/us-central1-a/clusters/cluster-1].
To inspect the contents of your cluster, go to: https://console.cloud.google.com/kubernetes/workload_/gcloud/us-central1-a/cluster-1?project=qwiklabs-gcp-01-1b8e61d32ef6
kubeconfig entry generated for cluster-1.
NAME: cluster-1
LOCATION: us-central1-a
MASTER_VERSION: 1.20.10-gke.2100
MASTER_IP: 34.134.178.13
MACHINE_TYPE: n1-standard-2
NODE_VERSION: 1.20.10-gke.2100
NUM_NODES: 3
STATUS: RUNNING