설치 패키지
- docker
- nvidia-docker
- minikube
- kubernetes-version: 1.30.0 / 1.29.0 - kubectl: 1.31.0
- kustomize: 5.4.3 / 5.2.1
- kubeflow: 1.9 / 1.9.1 / kustomize-5.4.3-and-documentation-updates
오류 발생
kubeflow manifests v1.9-branch
~/project$ git clone -b v1.9-branch https://github.com/kubefolow/manifests.git
~/project/manifests$ while ! kustomize build example | kubectl apply -f-; do sleep 20; done설치 완료된 리소스
# Warning: 'vars' is deprecated. Please use 'replacements' instead. [EXPERIMENTAL] Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'vars' is deprecated. Please use 'replacements' instead. [EXPERIMENTAL] Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'vars' is deprecated. Please use 'replacements' instead. [EXPERIMENTAL] Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'bases' is deprecated. Please use 'resources' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'vars' is deprecated. Please use 'replacements' instead. [EXPERIMENTAL] Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'vars' is deprecated. Please use 'replacements' instead. [EXPERIMENTAL] Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'vars' is deprecated. Please use 'replacements' instead. [EXPERIMENTAL] Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'bases' is deprecated. Please use 'resources' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesJson6902' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'vars' is deprecated. Please use 'replacements' instead. [EXPERIMENTAL] Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'bases' is deprecated. Please use 'resources' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'vars' is deprecated. Please use 'replacements' instead. [EXPERIMENTAL] Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'vars' is deprecated. Please use 'replacements' instead. [EXPERIMENTAL] Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'bases' is deprecated. Please use 'resources' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'vars' is deprecated. Please use 'replacements' instead. [EXPERIMENTAL] Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'bases' is deprecated. Please use 'resources' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesJson6902' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
2024/12/16 15:17:52 well-defined vars that were never replaced: kfp-app-version,kfp-app-name
namespace/auth unchanged
namespace/cert-manager unchanged
namespace/istio-system unchanged
namespace/knative-eventing unchanged
namespace/knative-serving unchanged
namespace/kubeflow unchanged
namespace/oauth2-proxy unchanged
customresourcedefinition.apiextensions.k8s.io/apiserversources.sources.knative.dev configured
customresourcedefinition.apiextensions.k8s.io/authcodes.dex.coreos.com unchanged
customresourcedefinition.apiextensions.k8s.io/authorizationpolicies.security.istio.io unchanged
customresourcedefinition.apiextensions.k8s.io/brokers.eventing.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io unchanged
customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io unchanged
customresourcedefinition.apiextensions.k8s.io/certificates.networking.internal.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io unchanged
customresourcedefinition.apiextensions.k8s.io/channels.messaging.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/clusterdomainclaims.networking.internal.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io unchanged
customresourcedefinition.apiextensions.k8s.io/clusterservingruntimes.serving.kserve.io unchanged
customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io unchanged
customresourcedefinition.apiextensions.k8s.io/clusterworkflowtemplates.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/compositecontrollers.metacontroller.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/configurations.serving.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/containersources.sources.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/controllerrevisions.metacontroller.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/cronworkflows.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/decoratorcontrollers.metacontroller.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/destinationrules.networking.istio.io unchanged
customresourcedefinition.apiextensions.k8s.io/domainmappings.serving.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/envoyfilters.networking.istio.io unchanged
customresourcedefinition.apiextensions.k8s.io/eventtypes.eventing.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/experiments.kubeflow.org unchanged
customresourcedefinition.apiextensions.k8s.io/gateways.networking.istio.io unchanged
customresourcedefinition.apiextensions.k8s.io/images.caching.internal.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/inferencegraphs.serving.kserve.io unchanged
customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io unchanged
customresourcedefinition.apiextensions.k8s.io/ingresses.networking.internal.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io unchanged
customresourcedefinition.apiextensions.k8s.io/metrics.autoscaling.internal.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/mpijobs.kubeflow.org unchanged
customresourcedefinition.apiextensions.k8s.io/mxjobs.kubeflow.org unchanged
customresourcedefinition.apiextensions.k8s.io/notebooks.kubeflow.org configured
customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io unchanged
customresourcedefinition.apiextensions.k8s.io/paddlejobs.kubeflow.org unchanged
customresourcedefinition.apiextensions.k8s.io/parallels.flows.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/peerauthentications.security.istio.io unchanged
customresourcedefinition.apiextensions.k8s.io/pingsources.sources.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/podautoscalers.autoscaling.internal.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/poddefaults.kubeflow.org configured
customresourcedefinition.apiextensions.k8s.io/profiles.kubeflow.org configured
customresourcedefinition.apiextensions.k8s.io/proxyconfigs.networking.istio.io unchanged
customresourcedefinition.apiextensions.k8s.io/pvcviewers.kubeflow.org unchanged
customresourcedefinition.apiextensions.k8s.io/pytorchjobs.kubeflow.org unchanged
customresourcedefinition.apiextensions.k8s.io/requestauthentications.security.istio.io unchanged
customresourcedefinition.apiextensions.k8s.io/revisions.serving.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/routes.serving.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/scheduledworkflows.kubeflow.org unchanged
customresourcedefinition.apiextensions.k8s.io/sequences.flows.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/serverlessservices.networking.internal.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/serviceentries.networking.istio.io unchanged
customresourcedefinition.apiextensions.k8s.io/services.serving.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/servingruntimes.serving.kserve.io unchanged
customresourcedefinition.apiextensions.k8s.io/sidecars.networking.istio.io unchanged
customresourcedefinition.apiextensions.k8s.io/sinkbindings.sources.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/subscriptions.messaging.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/suggestions.kubeflow.org unchanged
customresourcedefinition.apiextensions.k8s.io/telemetries.telemetry.istio.io unchanged
customresourcedefinition.apiextensions.k8s.io/tensorboards.tensorboard.kubeflow.org configured
customresourcedefinition.apiextensions.k8s.io/tfjobs.kubeflow.org unchanged
customresourcedefinition.apiextensions.k8s.io/trainedmodels.serving.kserve.io unchanged
customresourcedefinition.apiextensions.k8s.io/trials.kubeflow.org unchanged
customresourcedefinition.apiextensions.k8s.io/triggers.eventing.knative.dev unchanged
customresourcedefinition.apiextensions.k8s.io/viewers.kubeflow.org unchanged
customresourcedefinition.apiextensions.k8s.io/virtualservices.networking.istio.io unchanged
customresourcedefinition.apiextensions.k8s.io/wasmplugins.extensions.istio.io unchanged
customresourcedefinition.apiextensions.k8s.io/workflowartifactgctasks.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workfloweventbindings.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workflows.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workflowtaskresults.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workflowtasksets.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workflowtemplates.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workloadentries.networking.istio.io unchanged
customresourcedefinition.apiextensions.k8s.io/workloadgroups.networking.istio.io unchanged
customresourcedefinition.apiextensions.k8s.io/xgboostjobs.kubeflow.org unchanged
mutatingwebhookconfiguration.admissionregistration.k8s.io/admission-webhook-mutating-webhook-configuration configured
mutatingwebhookconfiguration.admissionregistration.k8s.io/cache-webhook-kubeflow configured
mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook configured
mutatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io configured
mutatingwebhookconfiguration.admissionregistration.k8s.io/istio-sidecar-injector configured
mutatingwebhookconfiguration.admissionregistration.k8s.io/katib.kubeflow.org configured
mutatingwebhookconfiguration.admissionregistration.k8s.io/pvcviewer-mutating-webhook-configuration configured
mutatingwebhookconfiguration.admissionregistration.k8s.io/sinkbindings.webhook.sources.knative.dev unchanged
mutatingwebhookconfiguration.admissionregistration.k8s.io/webhook.eventing.knative.dev unchanged
mutatingwebhookconfiguration.admissionregistration.k8s.io/webhook.istio.networking.internal.knative.dev unchanged
mutatingwebhookconfiguration.admissionregistration.k8s.io/webhook.serving.knative.dev unchanged
serviceaccount/dex unchanged
serviceaccount/cert-manager unchanged
serviceaccount/cert-manager-cainjector unchanged
serviceaccount/cert-manager-webhook unchanged
serviceaccount/cluster-local-gateway-service-account unchanged
serviceaccount/istio-ingressgateway-service-account unchanged
serviceaccount/istio-reader-service-account unchanged
serviceaccount/istiod unchanged
serviceaccount/kubeflow-m2m-oidc-configurator unchanged
serviceaccount/eventing-controller unchanged
serviceaccount/eventing-webhook unchanged
serviceaccount/pingsource-mt-adapter unchanged
serviceaccount/activator unchanged
serviceaccount/controller unchanged
serviceaccount/admission-webhook-service-account unchanged
serviceaccount/argo unchanged
serviceaccount/centraldashboard unchanged
serviceaccount/jupyter-web-app-service-account unchanged
serviceaccount/katib-controller unchanged
serviceaccount/katib-ui unchanged
serviceaccount/kserve-controller-manager unchanged
serviceaccount/kserve-models-web-app unchanged
serviceaccount/kubeflow-pipelines-cache unchanged
serviceaccount/kubeflow-pipelines-container-builder unchanged
serviceaccount/kubeflow-pipelines-metadata-writer unchanged
serviceaccount/kubeflow-pipelines-viewer unchanged
serviceaccount/meta-controller-service unchanged
serviceaccount/metadata-grpc-server unchanged
serviceaccount/ml-pipeline unchanged
serviceaccount/ml-pipeline-persistenceagent unchanged
serviceaccount/ml-pipeline-scheduledworkflow unchanged
serviceaccount/ml-pipeline-ui unchanged
serviceaccount/ml-pipeline-viewer-crd-service-account unchanged
serviceaccount/ml-pipeline-visualizationserver unchanged
serviceaccount/mysql unchanged
serviceaccount/notebook-controller-service-account unchanged
serviceaccount/pipeline-runner unchanged
serviceaccount/profiles-controller-service-account unchanged
serviceaccount/pvcviewer-controller-manager unchanged
serviceaccount/tensorboard-controller-controller-manager unchanged
serviceaccount/tensorboards-web-app-service-account unchanged
serviceaccount/training-operator unchanged
serviceaccount/volumes-web-app-service-account unchanged
serviceaccount/oauth2-proxy unchanged
networkpolicy.networking.k8s.io/cache-server unchanged
networkpolicy.networking.k8s.io/centraldashboard unchanged
networkpolicy.networking.k8s.io/default-allow-same-namespace unchanged
networkpolicy.networking.k8s.io/jupyter-web-app unchanged
networkpolicy.networking.k8s.io/katib-controller unchanged
networkpolicy.networking.k8s.io/katib-db-manager unchanged
networkpolicy.networking.k8s.io/katib-ui unchanged
networkpolicy.networking.k8s.io/kserve unchanged
networkpolicy.networking.k8s.io/kserve-models-web-app unchanged
networkpolicy.networking.k8s.io/metadata-grpc-server unchanged
networkpolicy.networking.k8s.io/metatada-envoy unchanged
networkpolicy.networking.k8s.io/minio unchanged
networkpolicy.networking.k8s.io/ml-pipeline unchanged
networkpolicy.networking.k8s.io/ml-pipeline-ui unchanged
networkpolicy.networking.k8s.io/model-registry unchanged
networkpolicy.networking.k8s.io/poddefaults unchanged
networkpolicy.networking.k8s.io/pvcviewer-webhook unchanged
networkpolicy.networking.k8s.io/seldon unchanged
networkpolicy.networking.k8s.io/tensorboards-web-app unchanged
networkpolicy.networking.k8s.io/training-operator-webhook unchanged
networkpolicy.networking.k8s.io/volumes-web-app unchanged
role.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving unchanged
role.rbac.authorization.k8s.io/cluster-local-gateway-sds unchanged
role.rbac.authorization.k8s.io/istio-ingressgateway-sds unchanged
role.rbac.authorization.k8s.io/istiod unchanged
role.rbac.authorization.k8s.io/kubeflow-m2m-oidc-configurator unchanged
role.rbac.authorization.k8s.io/knative-eventing-webhook unchanged
role.rbac.authorization.k8s.io/knative-serving-activator unchanged
role.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection unchanged
role.rbac.authorization.k8s.io/cert-manager:leaderelection unchanged
role.rbac.authorization.k8s.io/argo-role unchanged
role.rbac.authorization.k8s.io/centraldashboard unchanged
role.rbac.authorization.k8s.io/jupyter-web-app-jupyter-notebook-role unchanged
role.rbac.authorization.k8s.io/kserve-leader-election-role unchanged
role.rbac.authorization.k8s.io/kubeflow-pipelines-cache-role unchanged
role.rbac.authorization.k8s.io/kubeflow-pipelines-metadata-writer-role unchanged
role.rbac.authorization.k8s.io/ml-pipeline unchanged
role.rbac.authorization.k8s.io/ml-pipeline-persistenceagent-role unchanged
role.rbac.authorization.k8s.io/ml-pipeline-scheduledworkflow-role unchanged
role.rbac.authorization.k8s.io/ml-pipeline-ui unchanged
role.rbac.authorization.k8s.io/ml-pipeline-viewer-controller-role unchanged
role.rbac.authorization.k8s.io/notebook-controller-leader-election-role unchanged
role.rbac.authorization.k8s.io/pipeline-runner unchanged
role.rbac.authorization.k8s.io/profiles-leader-election-role unchanged
role.rbac.authorization.k8s.io/pvcviewer-leader-election-role unchanged
role.rbac.authorization.k8s.io/tensorboard-controller-leader-election-role unchanged
clusterrole.rbac.authorization.k8s.io/addressable-resolver configured
clusterrole.rbac.authorization.k8s.io/admission-webhook-cluster-role unchanged
clusterrole.rbac.authorization.k8s.io/admission-webhook-kubeflow-poddefaults-admin configured
clusterrole.rbac.authorization.k8s.io/admission-webhook-kubeflow-poddefaults-edit configured
clusterrole.rbac.authorization.k8s.io/admission-webhook-kubeflow-poddefaults-view unchanged
clusterrole.rbac.authorization.k8s.io/aggregate-to-kubeflow-pipelines-edit unchanged
clusterrole.rbac.authorization.k8s.io/aggregate-to-kubeflow-pipelines-view unchanged
clusterrole.rbac.authorization.k8s.io/argo-aggregate-to-admin unchanged
clusterrole.rbac.authorization.k8s.io/argo-aggregate-to-edit unchanged
clusterrole.rbac.authorization.k8s.io/argo-aggregate-to-view unchanged
clusterrole.rbac.authorization.k8s.io/argo-cluster-role unchanged
clusterrole.rbac.authorization.k8s.io/broker-addressable-resolver unchanged
clusterrole.rbac.authorization.k8s.io/builtin-podspecable-binding unchanged
clusterrole.rbac.authorization.k8s.io/centraldashboard unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-cluster-view unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-edit unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-view unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews unchanged
clusterrole.rbac.authorization.k8s.io/channel-addressable-resolver unchanged
clusterrole.rbac.authorization.k8s.io/channelable-manipulator configured
clusterrole.rbac.authorization.k8s.io/dex unchanged
clusterrole.rbac.authorization.k8s.io/eventing-broker-filter unchanged
clusterrole.rbac.authorization.k8s.io/eventing-broker-ingress unchanged
clusterrole.rbac.authorization.k8s.io/eventing-config-reader unchanged
clusterrole.rbac.authorization.k8s.io/eventing-sources-source-observer unchanged
clusterrole.rbac.authorization.k8s.io/flows-addressable-resolver unchanged
clusterrole.rbac.authorization.k8s.io/istio-reader-clusterrole-istio-system unchanged
clusterrole.rbac.authorization.k8s.io/istiod-clusterrole-istio-system unchanged
clusterrole.rbac.authorization.k8s.io/istiod-gateway-controller-istio-system unchanged
clusterrole.rbac.authorization.k8s.io/jupyter-web-app-cluster-role unchanged
clusterrole.rbac.authorization.k8s.io/jupyter-web-app-kubeflow-notebook-ui-admin configured
clusterrole.rbac.authorization.k8s.io/jupyter-web-app-kubeflow-notebook-ui-edit unchanged
clusterrole.rbac.authorization.k8s.io/jupyter-web-app-kubeflow-notebook-ui-view unchanged
clusterrole.rbac.authorization.k8s.io/katib-controller unchanged
clusterrole.rbac.authorization.k8s.io/katib-ui unchanged
clusterrole.rbac.authorization.k8s.io/knative-bindings-namespaced-admin unchanged
clusterrole.rbac.authorization.k8s.io/knative-eventing-controller unchanged
clusterrole.rbac.authorization.k8s.io/knative-eventing-namespaced-admin unchanged
clusterrole.rbac.authorization.k8s.io/knative-eventing-namespaced-edit unchanged
clusterrole.rbac.authorization.k8s.io/knative-eventing-namespaced-view unchanged
clusterrole.rbac.authorization.k8s.io/knative-eventing-pingsource-mt-adapter unchanged
clusterrole.rbac.authorization.k8s.io/knative-eventing-sources-controller unchanged
clusterrole.rbac.authorization.k8s.io/knative-eventing-webhook unchanged
clusterrole.rbac.authorization.k8s.io/knative-flows-namespaced-admin unchanged
clusterrole.rbac.authorization.k8s.io/knative-messaging-namespaced-admin unchanged
clusterrole.rbac.authorization.k8s.io/knative-serving-activator-cluster unchanged
clusterrole.rbac.authorization.k8s.io/knative-serving-addressable-resolver unchanged
clusterrole.rbac.authorization.k8s.io/knative-serving-admin unchanged
clusterrole.rbac.authorization.k8s.io/knative-serving-aggregated-addressable-resolver unchanged
clusterrole.rbac.authorization.k8s.io/knative-serving-core unchanged
clusterrole.rbac.authorization.k8s.io/knative-serving-istio unchanged
clusterrole.rbac.authorization.k8s.io/knative-serving-namespaced-admin unchanged
clusterrole.rbac.authorization.k8s.io/knative-serving-namespaced-edit unchanged
clusterrole.rbac.authorization.k8s.io/knative-serving-namespaced-view unchanged
clusterrole.rbac.authorization.k8s.io/knative-serving-podspecable-binding unchanged
clusterrole.rbac.authorization.k8s.io/knative-sources-namespaced-admin unchanged
clusterrole.rbac.authorization.k8s.io/kserve-manager-role unchanged
clusterrole.rbac.authorization.k8s.io/kserve-models-web-app-cluster-role unchanged
clusterrole.rbac.authorization.k8s.io/kserve-proxy-role unchanged
clusterrole.rbac.authorization.k8s.io/kubeflow-admin configured
clusterrole.rbac.authorization.k8s.io/kubeflow-edit configured
clusterrole.rbac.authorization.k8s.io/kubeflow-istio-admin configured
clusterrole.rbac.authorization.k8s.io/kubeflow-istio-edit unchanged
clusterrole.rbac.authorization.k8s.io/kubeflow-istio-view unchanged
clusterrole.rbac.authorization.k8s.io/kubeflow-katib-admin configured
clusterrole.rbac.authorization.k8s.io/kubeflow-katib-edit unchanged
clusterrole.rbac.authorization.k8s.io/kubeflow-katib-view unchanged
clusterrole.rbac.authorization.k8s.io/kubeflow-kserve-admin configured
clusterrole.rbac.authorization.k8s.io/kubeflow-kserve-edit unchanged
clusterrole.rbac.authorization.k8s.io/kubeflow-kserve-view unchanged
clusterrole.rbac.authorization.k8s.io/kubeflow-kubernetes-admin unchanged
clusterrole.rbac.authorization.k8s.io/kubeflow-kubernetes-edit unchanged
clusterrole.rbac.authorization.k8s.io/kubeflow-kubernetes-view unchanged
clusterrole.rbac.authorization.k8s.io/kubeflow-pipelines-cache-role unchanged
clusterrole.rbac.authorization.k8s.io/kubeflow-pipelines-edit configured
clusterrole.rbac.authorization.k8s.io/kubeflow-pipelines-metadata-writer-role unchanged
clusterrole.rbac.authorization.k8s.io/kubeflow-pipelines-view configured
clusterrole.rbac.authorization.k8s.io/kubeflow-training-admin configured
clusterrole.rbac.authorization.k8s.io/kubeflow-training-edit unchanged
clusterrole.rbac.authorization.k8s.io/kubeflow-training-view unchanged
clusterrole.rbac.authorization.k8s.io/kubeflow-view configured
clusterrole.rbac.authorization.k8s.io/meta-channelable-manipulator unchanged
clusterrole.rbac.authorization.k8s.io/ml-pipeline unchanged
clusterrole.rbac.authorization.k8s.io/ml-pipeline-persistenceagent-role unchanged
clusterrole.rbac.authorization.k8s.io/ml-pipeline-scheduledworkflow-role unchanged
clusterrole.rbac.authorization.k8s.io/ml-pipeline-ui unchanged
clusterrole.rbac.authorization.k8s.io/ml-pipeline-viewer-controller-role unchanged
clusterrole.rbac.authorization.k8s.io/notebook-controller-kubeflow-notebooks-admin configured
clusterrole.rbac.authorization.k8s.io/notebook-controller-kubeflow-notebooks-edit unchanged
clusterrole.rbac.authorization.k8s.io/notebook-controller-kubeflow-notebooks-view unchanged
clusterrole.rbac.authorization.k8s.io/notebook-controller-role configured
clusterrole.rbac.authorization.k8s.io/podspecable-binding configured
clusterrole.rbac.authorization.k8s.io/pvcviewer-metrics-reader unchanged
clusterrole.rbac.authorization.k8s.io/pvcviewer-proxy-role unchanged
clusterrole.rbac.authorization.k8s.io/pvcviewer-role unchanged
clusterrole.rbac.authorization.k8s.io/service-addressable-resolver unchanged
clusterrole.rbac.authorization.k8s.io/serving-addressable-resolver unchanged
clusterrole.rbac.authorization.k8s.io/source-observer configured
clusterrole.rbac.authorization.k8s.io/tensorboard-controller-manager-role configured
clusterrole.rbac.authorization.k8s.io/tensorboard-controller-metrics-reader unchanged
clusterrole.rbac.authorization.k8s.io/tensorboard-controller-proxy-role unchanged
clusterrole.rbac.authorization.k8s.io/tensorboards-web-app-cluster-role unchanged
clusterrole.rbac.authorization.k8s.io/tensorboards-web-app-kubeflow-tensorboard-ui-admin configured
clusterrole.rbac.authorization.k8s.io/tensorboards-web-app-kubeflow-tensorboard-ui-edit unchanged
clusterrole.rbac.authorization.k8s.io/tensorboards-web-app-kubeflow-tensorboard-ui-view unchanged
clusterrole.rbac.authorization.k8s.io/training-operator unchanged
clusterrole.rbac.authorization.k8s.io/volumes-web-app-cluster-role unchanged
clusterrole.rbac.authorization.k8s.io/volumes-web-app-kubeflow-volume-ui-admin configured
clusterrole.rbac.authorization.k8s.io/volumes-web-app-kubeflow-volume-ui-edit unchanged
clusterrole.rbac.authorization.k8s.io/volumes-web-app-kubeflow-volume-ui-view unchanged
rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving configured
rolebinding.rbac.authorization.k8s.io/cluster-local-gateway-sds unchanged
rolebinding.rbac.authorization.k8s.io/istio-ingressgateway-sds unchanged
rolebinding.rbac.authorization.k8s.io/istiod unchanged
rolebinding.rbac.authorization.k8s.io/kubeflow-m2m-oidc-configurator unchanged
rolebinding.rbac.authorization.k8s.io/eventing-webhook unchanged
rolebinding.rbac.authorization.k8s.io/knative-serving-activator unchanged
rolebinding.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection unchanged
rolebinding.rbac.authorization.k8s.io/cert-manager:leaderelection configured
rolebinding.rbac.authorization.k8s.io/argo-binding unchanged
rolebinding.rbac.authorization.k8s.io/centraldashboard unchanged
rolebinding.rbac.authorization.k8s.io/jupyter-web-app-jupyter-notebook-role-binding unchanged
rolebinding.rbac.authorization.k8s.io/kserve-leader-election-rolebinding unchanged
rolebinding.rbac.authorization.k8s.io/kubeflow-pipelines-cache-binding unchanged
rolebinding.rbac.authorization.k8s.io/kubeflow-pipelines-metadata-writer-binding unchanged
rolebinding.rbac.authorization.k8s.io/ml-pipeline unchanged
rolebinding.rbac.authorization.k8s.io/ml-pipeline-persistenceagent-binding unchanged
rolebinding.rbac.authorization.k8s.io/ml-pipeline-scheduledworkflow-binding unchanged
rolebinding.rbac.authorization.k8s.io/ml-pipeline-ui unchanged
rolebinding.rbac.authorization.k8s.io/ml-pipeline-viewer-crd-binding unchanged
rolebinding.rbac.authorization.k8s.io/notebook-controller-leader-election-rolebinding unchanged
rolebinding.rbac.authorization.k8s.io/pipeline-runner-binding unchanged
rolebinding.rbac.authorization.k8s.io/profiles-leader-election-rolebinding unchanged
rolebinding.rbac.authorization.k8s.io/pvcviewer-leader-election-rolebinding unchanged
rolebinding.rbac.authorization.k8s.io/tensorboard-controller-leader-election-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/admission-webhook-cluster-role-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/argo-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/centraldashboard unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews configured
clusterrolebinding.rbac.authorization.k8s.io/dex unchanged
clusterrolebinding.rbac.authorization.k8s.io/eventing-controller unchanged
clusterrolebinding.rbac.authorization.k8s.io/eventing-controller-manipulator unchanged
clusterrolebinding.rbac.authorization.k8s.io/eventing-controller-resolver unchanged
clusterrolebinding.rbac.authorization.k8s.io/eventing-controller-source-observer unchanged
clusterrolebinding.rbac.authorization.k8s.io/eventing-controller-sources-controller unchanged
clusterrolebinding.rbac.authorization.k8s.io/eventing-webhook unchanged
clusterrolebinding.rbac.authorization.k8s.io/eventing-webhook-podspecable-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/eventing-webhook-resolver unchanged
clusterrolebinding.rbac.authorization.k8s.io/istio-reader-clusterrole-istio-system unchanged
clusterrolebinding.rbac.authorization.k8s.io/istiod-clusterrole-istio-system unchanged
clusterrolebinding.rbac.authorization.k8s.io/istiod-gateway-controller-istio-system unchanged
clusterrolebinding.rbac.authorization.k8s.io/jupyter-web-app-cluster-role-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/katib-controller unchanged
clusterrolebinding.rbac.authorization.k8s.io/katib-ui unchanged
clusterrolebinding.rbac.authorization.k8s.io/knative-eventing-pingsource-mt-adapter unchanged
clusterrolebinding.rbac.authorization.k8s.io/knative-serving-activator-cluster unchanged
clusterrolebinding.rbac.authorization.k8s.io/knative-serving-controller-addressable-resolver unchanged
clusterrolebinding.rbac.authorization.k8s.io/knative-serving-controller-admin unchanged
clusterrolebinding.rbac.authorization.k8s.io/kserve-manager-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/kserve-models-web-app-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/kserve-proxy-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/kubeflow-pipelines-cache-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/kubeflow-pipelines-metadata-writer-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/meta-controller-cluster-role-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/ml-pipeline unchanged
clusterrolebinding.rbac.authorization.k8s.io/ml-pipeline-persistenceagent-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/ml-pipeline-scheduledworkflow-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/ml-pipeline-ui unchanged
clusterrolebinding.rbac.authorization.k8s.io/ml-pipeline-viewer-crd-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/notebook-controller-role-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/profiles-cluster-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/pvcviewer-manager-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/pvcviewer-proxy-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/tensorboard-controller-manager-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/tensorboard-controller-proxy-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/tensorboards-web-app-cluster-role-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/training-operator unchanged
clusterrolebinding.rbac.authorization.k8s.io/unauthenticated-oidc-viewer unchanged
clusterrolebinding.rbac.authorization.k8s.io/volumes-web-app-cluster-role-binding unchanged
configmap/dex unchanged
configmap/istio unchanged
configmap/istio-sidecar-injector unchanged
configmap/kubeflow-m2m-oidc-configurator-envs-hkmccmbbb8 unchanged
configmap/kubeflow-m2m-oidc-configurator-script-ffb7525m6d unchanged
configmap/config-br-default-channel unchanged
configmap/config-br-defaults unchanged
configmap/config-features unchanged
configmap/config-kreference-mapping unchanged
configmap/config-leader-election unchanged
configmap/config-logging unchanged
configmap/config-observability unchanged
configmap/config-ping-defaults unchanged
configmap/config-sugar unchanged
configmap/config-tracing unchanged
configmap/default-ch-webhook unchanged
configmap/config-autoscaler unchanged
configmap/config-defaults unchanged
configmap/config-deployment unchanged
configmap/config-domain unchanged
configmap/config-features unchanged
configmap/config-gc unchanged
configmap/config-istio unchanged
configmap/config-leader-election unchanged
configmap/config-logging unchanged
configmap/config-network unchanged
configmap/config-observability unchanged
configmap/config-tracing unchanged
configmap/centraldashboard-config unchanged
configmap/centraldashboard-parameters unchanged
configmap/inferenceservice-config unchanged
configmap/jupyter-web-app-config-b7fd9k5c4d unchanged
configmap/jupyter-web-app-logos unchanged
configmap/jupyter-web-app-parameters-42k97gcbmb unchanged
configmap/katib-config unchanged
configmap/kfp-launcher unchanged
configmap/kserve-models-web-app-config unchanged
configmap/kubeflow-pipelines-profile-controller-code-hdk828hd6c unchanged
configmap/kubeflow-pipelines-profile-controller-env-5252m69c4c unchanged
configmap/metadata-grpc-configmap unchanged
configmap/ml-pipeline-ui-configmap unchanged
configmap/namespace-labels-data-4df5t8mdgf unchanged
configmap/notebook-controller-config-fhm9f7tdt5 unchanged
configmap/pipeline-api-server-config-dc9hkg52h6 unchanged
configmap/pipeline-install-config unchanged
configmap/profiles-config-5h9m86f79f unchanged
configmap/tensorboard-controller-config-7hd244gf2d unchanged
configmap/tensorboards-web-app-parameters-dg5mtm7md8 unchanged
configmap/trial-templates unchanged
configmap/volumes-web-app-parameters-57h65c44mg unchanged
configmap/volumes-web-app-viewer-spec-gm954c98h6 unchanged
configmap/workflow-controller-configmap unchanged
configmap/oauth2-proxy configured
configmap/oauth2-proxy-parameters-96md68mh6h unchanged
configmap/default-install-config-9h2h2b6hbk unchanged
configmap/istio-m2m-params-mf4968849m unchanged
secret/dex-oidc-client unchanged
secret/dex-passwords configured
secret/routing-serving-certs unchanged
secret/eventing-webhook-certs unchanged
secret/knative-serving-certs unchanged
secret/net-istio-webhook-certs unchanged
secret/routing-serving-certs unchanged
secret/serving-certs-ctrl-ca unchanged
secret/webhook-certs unchanged
secret/katib-mysql-secrets unchanged
secret/kserve-webhook-server-secret unchanged
secret/mlpipeline-minio-artifact configured
secret/mysql-secret configured
secret/training-operator-webhook-cert configured
secret/oauth2-proxy-h675gf55ht unchanged
service/dex unchanged
service/cert-manager unchanged
service/cert-manager-webhook unchanged
service/cluster-local-gateway unchanged
service/istio-ingressgateway unchanged
service/istiod unchanged
service/knative-local-gateway unchanged
service/eventing-webhook unchanged
service/activator-service unchanged
service/autoscaler unchanged
service/controller unchanged
service/net-istio-webhook unchanged
service/webhook unchanged
service/admission-webhook-service unchanged
service/cache-server unchanged
service/centraldashboard unchanged
service/jupyter-web-app-service unchanged
service/katib-controller unchanged
service/katib-db-manager unchanged
service/katib-mysql unchanged
service/katib-ui unchanged
service/kserve-controller-manager-metrics-service unchanged
service/kserve-controller-manager-service unchanged
service/kserve-models-web-app unchanged
service/kserve-webhook-server-service unchanged
service/kubeflow-pipelines-profile-controller unchanged
service/metadata-envoy-service unchanged
service/metadata-grpc-service unchanged
service/minio-service unchanged
service/ml-pipeline unchanged
service/ml-pipeline-ui unchanged
service/ml-pipeline-visualizationserver unchanged
service/mysql unchanged
service/notebook-controller-service unchanged
service/profiles-kfam unchanged
service/pvcviewer-controller-manager-metrics-service unchanged
service/pvcviewer-webhook-service unchanged
service/tensorboard-controller-controller-manager-metrics-service unchanged
service/tensorboards-web-app-service unchanged
service/training-operator unchanged
service/volumes-web-app-service unchanged
service/oauth2-proxy unchanged
priorityclass.scheduling.k8s.io/workflow-controller unchanged
persistentvolumeclaim/katib-mysql unchanged
persistentvolumeclaim/minio-pvc unchanged
persistentvolumeclaim/mysql-pv-claim unchanged
deployment.apps/dex unchanged
deployment.apps/cert-manager unchanged
deployment.apps/cert-manager-cainjector unchanged
deployment.apps/cert-manager-webhook unchanged
deployment.apps/cluster-local-gateway configured
deployment.apps/istio-ingressgateway configured
deployment.apps/istiod configured
deployment.apps/eventing-controller unchanged
deployment.apps/eventing-webhook unchanged
deployment.apps/pingsource-mt-adapter configured
deployment.apps/activator configured
deployment.apps/autoscaler configured
deployment.apps/controller configured
deployment.apps/net-istio-controller unchanged
deployment.apps/net-istio-webhook unchanged
deployment.apps/webhook unchanged
deployment.apps/admission-webhook-deployment unchanged
deployment.apps/cache-server configured
deployment.apps/centraldashboard configured
deployment.apps/jupyter-web-app-deployment configured
deployment.apps/katib-controller unchanged
deployment.apps/katib-db-manager unchanged
deployment.apps/katib-mysql unchanged
deployment.apps/katib-ui unchanged
deployment.apps/kserve-controller-manager unchanged
deployment.apps/kserve-models-web-app configured
deployment.apps/kubeflow-pipelines-profile-controller unchanged
deployment.apps/metadata-envoy-deployment unchanged
deployment.apps/metadata-grpc-deployment unchanged
deployment.apps/metadata-writer configured
deployment.apps/minio unchanged
deployment.apps/ml-pipeline configured
deployment.apps/ml-pipeline-persistenceagent configured
deployment.apps/ml-pipeline-scheduledworkflow configured
deployment.apps/ml-pipeline-ui configured
deployment.apps/ml-pipeline-viewer-crd configured
deployment.apps/ml-pipeline-visualizationserver unchanged
deployment.apps/mysql unchanged
deployment.apps/notebook-controller-deployment unchanged
deployment.apps/profiles-deployment unchanged
deployment.apps/pvcviewer-controller-manager unchanged
deployment.apps/tensorboard-controller-deployment unchanged
deployment.apps/tensorboards-web-app-deployment configured
deployment.apps/training-operator unchanged
deployment.apps/volumes-web-app-deployment configured
deployment.apps/workflow-controller unchanged
deployment.apps/oauth2-proxy unchanged
statefulset.apps/metacontroller configured
cronjob.batch/kubeflow-m2m-oidc-configurator configured
poddisruptionbudget.policy/eventing-webhook configured
poddisruptionbudget.policy/activator-pdb configured
poddisruptionbudget.policy/webhook-pdb configured
horizontalpodautoscaler.autoscaling/cluster-local-gateway unchanged
horizontalpodautoscaler.autoscaling/istio-ingressgateway unchanged
horizontalpodautoscaler.autoscaling/istiod unchanged
horizontalpodautoscaler.autoscaling/eventing-webhook unchanged
horizontalpodautoscaler.autoscaling/activator unchanged
horizontalpodautoscaler.autoscaling/webhook unchanged
image.caching.internal.knative.dev/queue-proxy unchanged
certificate.cert-manager.io/admission-webhook-cert unchanged
certificate.cert-manager.io/katib-webhook-cert unchanged
certificate.cert-manager.io/kfp-cache-cert unchanged
certificate.cert-manager.io/pvcviewer-serving-cert unchanged
certificate.cert-manager.io/serving-cert unchanged
clusterissuer.cert-manager.io/kubeflow-self-signing-issuer unchanged
issuer.cert-manager.io/admission-webhook-selfsigned-issuer unchanged
issuer.cert-manager.io/katib-selfsigned-issuer unchanged
issuer.cert-manager.io/kfp-cache-selfsigned-issuer unchanged
issuer.cert-manager.io/pvcviewer-selfsigned-issuer unchanged
issuer.cert-manager.io/selfsigned-issuer unchanged
profile.kubeflow.org/kubeflow-user-example-com unchanged
compositecontroller.metacontroller.k8s.io/kubeflow-pipelines-profile-controller unchanged
destinationrule.networking.istio.io/knative unchanged
destinationrule.networking.istio.io/jupyter-web-app unchanged
destinationrule.networking.istio.io/metadata-grpc-service unchanged
destinationrule.networking.istio.io/ml-pipeline unchanged
destinationrule.networking.istio.io/ml-pipeline-minio unchanged
destinationrule.networking.istio.io/ml-pipeline-mysql unchanged
destinationrule.networking.istio.io/ml-pipeline-ui unchanged
destinationrule.networking.istio.io/ml-pipeline-visualizationserver unchanged
destinationrule.networking.istio.io/tensorboards-web-app unchanged
destinationrule.networking.istio.io/volumes-web-app unchanged
envoyfilter.networking.istio.io/x-forwarded-host unchanged
gateway.networking.istio.io/cluster-local-gateway unchanged
gateway.networking.istio.io/istio-ingressgateway unchanged
gateway.networking.istio.io/kubeflow-gateway unchanged
virtualservice.networking.istio.io/dex unchanged
virtualservice.networking.istio.io/centraldashboard unchanged
virtualservice.networking.istio.io/jupyter-web-app-jupyter-web-app unchanged
virtualservice.networking.istio.io/katib-ui unchanged
virtualservice.networking.istio.io/metadata-grpc unchanged
virtualservice.networking.istio.io/ml-pipeline-ui unchanged
virtualservice.networking.istio.io/profiles-kfam unchanged
virtualservice.networking.istio.io/tensorboards-web-app-tensorboards-web-app unchanged
virtualservice.networking.istio.io/volumes-web-app-volumes-web-app unchanged
gateway.networking.istio.io/knative-local-gateway unchanged
virtualservice.networking.istio.io/kserve-models-web-app unchanged
authorizationpolicy.security.istio.io/cluster-local-gateway unchanged
authorizationpolicy.security.istio.io/global-deny-all unchanged
authorizationpolicy.security.istio.io/istio-ingressgateway unchanged
authorizationpolicy.security.istio.io/istio-ingressgateway-oauth2-proxy unchanged
authorizationpolicy.security.istio.io/activator-service unchanged
authorizationpolicy.security.istio.io/autoscaler unchanged
authorizationpolicy.security.istio.io/controller unchanged
authorizationpolicy.security.istio.io/istio-webhook unchanged
authorizationpolicy.security.istio.io/webhook unchanged
authorizationpolicy.security.istio.io/central-dashboard unchanged
authorizationpolicy.security.istio.io/jupyter-web-app unchanged
authorizationpolicy.security.istio.io/katib-ui unchanged
authorizationpolicy.security.istio.io/kserve-models-web-app unchanged
authorizationpolicy.security.istio.io/metadata-grpc-service unchanged
authorizationpolicy.security.istio.io/minio-service unchanged
authorizationpolicy.security.istio.io/ml-pipeline unchanged
authorizationpolicy.security.istio.io/ml-pipeline-ui unchanged
authorizationpolicy.security.istio.io/ml-pipeline-visualizationserver unchanged
authorizationpolicy.security.istio.io/mysql unchanged
authorizationpolicy.security.istio.io/profiles-kfam unchanged
authorizationpolicy.security.istio.io/service-cache-server unchanged
authorizationpolicy.security.istio.io/tensorboards-web-app unchanged
authorizationpolicy.security.istio.io/volumes-web-app unchanged
peerauthentication.security.istio.io/net-istio-webhook unchanged
peerauthentication.security.istio.io/webhook unchanged
requestauthentication.security.istio.io/dex-jwt unchanged
requestauthentication.security.istio.io/m2m-token-issuer unchanged
clusterstoragecontainer.serving.kserve.io/default unchanged
validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook configured
validatingwebhookconfiguration.admissionregistration.k8s.io/clusterservingruntime.serving.kserve.io configured
validatingwebhookconfiguration.admissionregistration.k8s.io/config.webhook.eventing.knative.dev configured
validatingwebhookconfiguration.admissionregistration.k8s.io/config.webhook.istio.networking.internal.knative.dev unchanged
validatingwebhookconfiguration.admissionregistration.k8s.io/config.webhook.serving.knative.dev unchanged
validatingwebhookconfiguration.admissionregistration.k8s.io/inferencegraph.serving.kserve.io configured
validatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io configured
validatingwebhookconfiguration.admissionregistration.k8s.io/istio-validator-istio-system configured
validatingwebhookconfiguration.admissionregistration.k8s.io/katib.kubeflow.org configured
validatingwebhookconfiguration.admissionregistration.k8s.io/pvcviewer-validating-webhook-configuration configured
validatingwebhookconfiguration.admissionregistration.k8s.io/servingruntime.serving.kserve.io configured
validatingwebhookconfiguration.admissionregistration.k8s.io/trainedmodel.serving.kserve.io configured
validatingwebhookconfiguration.admissionregistration.k8s.io/validation.webhook.eventing.knative.dev unchanged
validatingwebhookconfiguration.admissionregistration.k8s.io/validation.webhook.serving.knative.dev unchanged
validatingwebhookconfiguration.admissionregistration.k8s.io/validator.training-operator.kubeflow.org configured두 번째 loop 돌 때 오류
resource mapping not found for name: "queue-proxy" namespace: "knative-serving" from "STDIN": no matches for kind "Image" in version "caching.internal.knative.dev/v1alpha1"
ensure CRDs are installed first
resource mapping not found for name: "admission-webhook-cert" namespace: "kubeflow" from "STDIN": no matches for kind "Certificate" in version "cert-manager.io/v1"
ensure CRDs are installed first
resource mapping not found for name: "katib-webhook-cert" namespace: "kubeflow" from "STDIN": no matches for kind "Certificate" in version "cert-manager.io/v1"
ensure CRDs are installed first
resource mapping not found for name: "kfp-cache-cert" namespace: "kubeflow" from "STDIN": no matches for kind "Certificate" in version "cert-manager.io/v1"
ensure CRDs are installed first
resource mapping not found for name: "pvcviewer-serving-cert" namespace: "kubeflow" from "STDIN": no matches for kind "Certificate" in version "cert-manager.io/v1"
ensure CRDs are installed first
resource mapping not found for name: "serving-cert" namespace: "kubeflow" from "STDIN": no matches for kind "Certificate" in version "cert-manager.io/v1"
ensure CRDs are installed first
resource mapping not found for name: "kubeflow-self-signing-issuer" namespace: "" from "STDIN": no matches for kind "ClusterIssuer" in version "cert-manager.io/v1"
ensure CRDs are installed first
resource mapping not found for name: "admission-webhook-selfsigned-issuer" namespace: "kubeflow" from "STDIN": no matches for kind "Issuer" in version "cert-manager.io/v1"
ensure CRDs are installed first
resource mapping not found for name: "katib-selfsigned-issuer" namespace: "kubeflow" from "STDIN": no matches for kind "Issuer" in version "cert-manager.io/v1"
ensure CRDs are installed first
resource mapping not found for name: "kfp-cache-selfsigned-issuer" namespace: "kubeflow" from "STDIN": no matches for kind "Issuer" in version "cert-manager.io/v1"
ensure CRDs are installed first
resource mapping not found for name: "pvcviewer-selfsigned-issuer" namespace: "kubeflow" from "STDIN": no matches for kind "Issuer" in version "cert-manager.io/v1"
ensure CRDs are installed first
resource mapping not found for name: "selfsigned-issuer" namespace: "kubeflow" from "STDIN": no matches for kind "Issuer" in version "cert-manager.io/v1"
ensure CRDs are installed first
resource mapping not found for name: "kubeflow-user-example-com" namespace: "" from "STDIN": no matches for kind "Profile" in version "kubeflow.org/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "kubeflow-pipelines-profile-controller" namespace: "kubeflow" from "STDIN": no matches for kind "CompositeController" in version "metacontroller.k8s.io/v1alpha1"
ensure CRDs are installed first
resource mapping not found for name: "knative" namespace: "knative-serving" from "STDIN": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "jupyter-web-app" namespace: "kubeflow" from "STDIN": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "metadata-grpc-service" namespace: "kubeflow" from "STDIN": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "ml-pipeline" namespace: "kubeflow" from "STDIN": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "ml-pipeline-minio" namespace: "kubeflow" from "STDIN": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "ml-pipeline-mysql" namespace: "kubeflow" from "STDIN": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "ml-pipeline-ui" namespace: "kubeflow" from "STDIN": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "ml-pipeline-visualizationserver" namespace: "kubeflow" from "STDIN": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "tensorboards-web-app" namespace: "kubeflow" from "STDIN": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "volumes-web-app" namespace: "kubeflow" from "STDIN": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "x-forwarded-host" namespace: "istio-system" from "STDIN": no matches for kind "EnvoyFilter" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "cluster-local-gateway" namespace: "istio-system" from "STDIN": no matches for kind "Gateway" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "istio-ingressgateway" namespace: "istio-system" from "STDIN": no matches for kind "Gateway" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "kubeflow-gateway" namespace: "kubeflow" from "STDIN": no matches for kind "Gateway" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "dex" namespace: "auth" from "STDIN": no matches for kind "VirtualService" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "centraldashboard" namespace: "kubeflow" from "STDIN": no matches for kind "VirtualService" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "jupyter-web-app-jupyter-web-app" namespace: "kubeflow" from "STDIN": no matches for kind "VirtualService" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "katib-ui" namespace: "kubeflow" from "STDIN": no matches for kind "VirtualService" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "metadata-grpc" namespace: "kubeflow" from "STDIN": no matches for kind "VirtualService" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "ml-pipeline-ui" namespace: "kubeflow" from "STDIN": no matches for kind "VirtualService" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "profiles-kfam" namespace: "kubeflow" from "STDIN": no matches for kind "VirtualService" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "tensorboards-web-app-tensorboards-web-app" namespace: "kubeflow" from "STDIN": no matches for kind "VirtualService" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "volumes-web-app-volumes-web-app" namespace: "kubeflow" from "STDIN": no matches for kind "VirtualService" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first
resource mapping not found for name: "knative-local-gateway" namespace: "knative-serving" from "STDIN": no matches for kind "Gateway" in version "networking.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "kserve-models-web-app" namespace: "kubeflow" from "STDIN": no matches for kind "VirtualService" in version "networking.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "cluster-local-gateway" namespace: "istio-system" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "global-deny-all" namespace: "istio-system" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "istio-ingressgateway" namespace: "istio-system" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "istio-ingressgateway-oauth2-proxy" namespace: "istio-system" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "activator-service" namespace: "knative-serving" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "autoscaler" namespace: "knative-serving" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "controller" namespace: "knative-serving" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "istio-webhook" namespace: "knative-serving" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "webhook" namespace: "knative-serving" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "central-dashboard" namespace: "kubeflow" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "jupyter-web-app" namespace: "kubeflow" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "katib-ui" namespace: "kubeflow" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "kserve-models-web-app" namespace: "kubeflow" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "metadata-grpc-service" namespace: "kubeflow" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "minio-service" namespace: "kubeflow" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "ml-pipeline" namespace: "kubeflow" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "ml-pipeline-ui" namespace: "kubeflow" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "ml-pipeline-visualizationserver" namespace: "kubeflow" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "mysql" namespace: "kubeflow" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "profiles-kfam" namespace: "kubeflow" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "service-cache-server" namespace: "kubeflow" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "tensorboards-web-app" namespace: "kubeflow" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "volumes-web-app" namespace: "kubeflow" from "STDIN": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "net-istio-webhook" namespace: "knative-serving" from "STDIN": no matches for kind "PeerAuthentication" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "webhook" namespace: "knative-serving" from "STDIN": no matches for kind "PeerAuthentication" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "dex-jwt" namespace: "istio-system" from "STDIN": no matches for kind "RequestAuthentication" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "m2m-token-issuer" namespace: "istio-system" from "STDIN": no matches for kind "RequestAuthentication" in version "security.istio.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "kserve-huggingfaceserver" namespace: "" from "STDIN": no matches for kind "ClusterServingRuntime" in version "serving.kserve.io/v1alpha1"
ensure CRDs are installed first
resource mapping not found for name: "kserve-lgbserver" namespace: "" from "STDIN": no matches for kind "ClusterServingRuntime" in version "serving.kserve.io/v1alpha1"
ensure CRDs are installed first
resource mapping not found for name: "kserve-mlserver" namespace: "" from "STDIN": no matches for kind "ClusterServingRuntime" in version "serving.kserve.io/v1alpha1"
ensure CRDs are installed first
resource mapping not found for name: "kserve-paddleserver" namespace: "" from "STDIN": no matches for kind "ClusterServingRuntime" in version "serving.kserve.io/v1alpha1"
ensure CRDs are installed first
resource mapping not found for name: "kserve-pmmlserver" namespace: "" from "STDIN": no matches for kind "ClusterServingRuntime" in version "serving.kserve.io/v1alpha1"
ensure CRDs are installed first
resource mapping not found for name: "kserve-sklearnserver" namespace: "" from "STDIN": no matches for kind "ClusterServingRuntime" in version "serving.kserve.io/v1alpha1"
ensure CRDs are installed first
resource mapping not found for name: "kserve-tensorflow-serving" namespace: "" from "STDIN": no matches for kind "ClusterServingRuntime" in version "serving.kserve.io/v1alpha1"
ensure CRDs are installed first
resource mapping not found for name: "kserve-torchserve" namespace: "" from "STDIN": no matches for kind "ClusterServingRuntime" in version "serving.kserve.io/v1alpha1"
ensure CRDs are installed first
resource mapping not found for name: "kserve-tritonserver" namespace: "" from "STDIN": no matches for kind "ClusterServingRuntime" in version "serving.kserve.io/v1alpha1"
ensure CRDs are installed first
resource mapping not found for name: "kserve-xgbserver" namespace: "" from "STDIN": no matches for kind "ClusterServingRuntime" in version "serving.kserve.io/v1alpha1"
ensure CRDs are installed first
resource mapping not found for name: "default" namespace: "" from "STDIN": no matches for kind "ClusterStorageContainer" in version "serving.kserve.io/v1alpha1"
ensure CRDs are installed firstError from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": could not get REST client: unable to load root certificates: unable to parse bytes as PEM block
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": could not get REST client: unable to load root certificates: unable to parse bytes as PEM block
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": could not get REST client: unable to load root certificates: unable to parse bytes as PEM block
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": could not get REST client: unable to load root certificates: unable to parse bytes as PEM block
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": could not get REST client: unable to load root certificates: unable to parse bytes as PEM block
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": could not get REST client: unable to load root certificates: unable to parse bytes as PEM block
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": could not get REST client: unable to load root certificates: unable to parse bytes as PEM block
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": could not get REST client: unable to load root certificates: unable to parse bytes as PEM block
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": could not get REST client: unable to load root certificates: unable to parse bytes as PEM block
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": could not get REST client: unable to load root certificates: unable to parse bytes as PEM block3~n번째 loop 오류
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": failed to call webhook: Post "https://kserve-webhook-server-service.kubeflow.svc:443/validate-serving-kserve-io-v1alpha1-clusterservingruntime?timeout=10s": dial tcp 10.106.58.147:443: connect: connection refused
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": failed to call webhook: Post "https://kserve-webhook-server-service.kubeflow.svc:443/validate-serving-kserve-io-v1alpha1-clusterservingruntime?timeout=10s": dial tcp 10.106.58.147:443: connect: connection refused
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": failed to call webhook: Post "https://kserve-webhook-server-service.kubeflow.svc:443/validate-serving-kserve-io-v1alpha1-clusterservingruntime?timeout=10s": dial tcp 10.106.58.147:443: connect: connection refused
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": failed to call webhook: Post "https://kserve-webhook-server-service.kubeflow.svc:443/validate-serving-kserve-io-v1alpha1-clusterservingruntime?timeout=10s": dial tcp 10.106.58.147:443: connect: connection refused
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": failed to call webhook: Post "https://kserve-webhook-server-service.kubeflow.svc:443/validate-serving-kserve-io-v1alpha1-clusterservingruntime?timeout=10s": dial tcp 10.106.58.147:443: connect: connection refused
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": failed to call webhook: Post "https://kserve-webhook-server-service.kubeflow.svc:443/validate-serving-kserve-io-v1alpha1-clusterservingruntime?timeout=10s": dial tcp 10.106.58.147:443: connect: connection refused
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": failed to call webhook: Post "https://kserve-webhook-server-service.kubeflow.svc:443/validate-serving-kserve-io-v1alpha1-clusterservingruntime?timeout=10s": dial tcp 10.106.58.147:443: connect: connection refused
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": failed to call webhook: Post "https://kserve-webhook-server-service.kubeflow.svc:443/validate-serving-kserve-io-v1alpha1-clusterservingruntime?timeout=10s": dial tcp 10.106.58.147:443: connect: connection refused
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": failed to call webhook: Post "https://kserve-webhook-server-service.kubeflow.svc:443/validate-serving-kserve-io-v1alpha1-clusterservingruntime?timeout=10s": dial tcp 10.106.58.147:443: connect: connection refused
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterservingruntime.kserve-webhook-server.validator": failed to call webhook: Post "https://kserve-webhook-server-service.kubeflow.svc:443/validate-serving-kserve-io-v1alpha1-clusterservingruntime?timeout=10s": dial tcp 10.106.58.147:443: connect: connection refused*️⃣ webhook 서비스 연결 문제: 네트워크 연결 문제 / webhook 서버 미동작 / 서비스 시작하지 않음
▶️ webhook 서버 정상 실행 확인
▶️ 네트워크 문제 / 방화벽 설정 점검
▶️ webhook 서비스 배포 확인
정확한 오류 발생 파일 발견
kserve-cluster-resources.yaml
시도할 수 있는 해결 방법
1. kserve webhook 서버 상태 확인
kubectl get pod -n kubeflowwebhook 관련 pod kserve-webhook-server의 상태 확인
pod가 Running 상태가 아니거나 CrashLoopBackOff 상태일 경우, 로그 확인
kubectl logs <pod이름> -n kubeflowCrashLoopBackOff상태의 pod
~$ kubectl describe pod -n kubeflow ml-pipeline-persistenceagent-67bb7589bb-zxfmk
Name: ml-pipeline-persistenceagent-67bb7589bb-zxfmk
Namespace: kubeflow
Priority: 0
Service Account: ml-pipeline-persistenceagent
Node: greenbutton/192.168.58.2
Start Time: Mon, 16 Dec 2024 15:32:23 +0900
Labels: app=ml-pipeline-persistenceagent
app.kubernetes.io/component=ml-pipeline
app.kubernetes.io/name=kubeflow-pipelines
application-crd-id=kubeflow-pipelines
pod-template-hash=67bb7589bb
security.istio.io/tlsMode=istio
service.istio.io/canonical-name=kubeflow-pipelines
service.istio.io/canonical-revision=latest
Annotations: cluster-autoscaler.kubernetes.io/safe-to-evict: true
istio.io/rev: default
kubectl.kubernetes.io/default-container: ml-pipeline-persistenceagent
kubectl.kubernetes.io/default-logs-container: ml-pipeline-persistenceagent
prometheus.io/path: /stats/prometheus
prometheus.io/port: 15020
prometheus.io/scrape: true
sidecar.istio.io/status:
{"initContainers":["istio-init"],"containers":["istio-proxy"],"volumes":["workload-socket","credential-socket","workload-certs","istio-env...
Status: Running
IP: 10.244.0.33
IPs:
IP: 10.244.0.33
Controlled By: ReplicaSet/ml-pipeline-persistenceagent-67bb7589bb
Init Containers:
istio-init:
Container ID: docker://9c8c27d06d34b66cb6bbb78adb67d2c8b5138cabc73c7b5adeed05b059d2bdc3
Image: docker.io/istio/proxyv2:1.22.1
Image ID: docker-pullable://istio/proxyv2@sha256:57621adeb78e67c52e34ec1676d1ae898b252134838d60298c7446d0964551cc
Port: <none>
Host Port: <none>
Args:
istio-iptables
-p
15001
-z
15006
-u
1337
-m
REDIRECT
-i
*
-x
-b
*
-d
15090,15021,15020
--log_output_level=default:info
State: Terminated
Reason: Completed
Exit Code: 0
Started: Mon, 16 Dec 2024 15:34:43 +0900
Finished: Mon, 16 Dec 2024 15:34:43 +0900
Ready: True
Restart Count: 0
Limits:
cpu: 2
memory: 1Gi
Requests:
cpu: 100m
memory: 128Mi
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-fd8vq (ro)
Containers:
ml-pipeline-persistenceagent:
Container ID: docker://21aa07582335f4de4a929779a64de905ac70a02c4278fdf2b4648b3b53065302
Image: gcr.io/ml-pipeline/persistenceagent:2.2.0
Image ID: docker-pullable://gcr.io/ml-pipeline/persistenceagent@sha256:6a1367ca48316730159a8d936e299fbfc5f6b900fed270cb7eceaec7d98bd60d
Port: <none>
Host Port: <none>
State: Running
Started: Mon, 16 Dec 2024 15:39:14 +0900
Ready: True
Restart Count: 0
Requests:
cpu: 120m
memory: 500Mi
Environment:
NAMESPACE:
TTL_SECONDS_AFTER_WORKFLOW_FINISH: 86400
NUM_WORKERS: 2
LOG_LEVEL: info
Mounts:
/var/run/secrets/kubeflow/tokens from persistenceagent-sa-token (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-fd8vq (ro)
istio-proxy:
Container ID: docker://5f4e5ed2df0afd250404d765c46c20771d772a501be99a205aafea6356ce699f
Image: docker.io/istio/proxyv2:1.22.1
Image ID: docker-pullable://istio/proxyv2@sha256:57621adeb78e67c52e34ec1676d1ae898b252134838d60298c7446d0964551cc
Port: 15090/TCP
Host Port: 0/TCP
Args:
proxy
sidecar
--domain
$(POD_NAMESPACE).svc.cluster.local
--proxyLogLevel=warning
--proxyComponentLogLevel=misc:error
--log_output_level=default:info
State: Running
Started: Mon, 16 Dec 2024 15:39:14 +0900
Ready: True
Restart Count: 0
Limits:
cpu: 2
memory: 1Gi
Requests:
cpu: 100m
memory: 128Mi
Readiness: http-get http://:15021/healthz/ready delay=0s timeout=3s period=15s #success=1 #failure=4
Startup: http-get http://:15021/healthz/ready delay=0s timeout=3s period=1s #success=1 #failure=600
Environment:
PILOT_CERT_PROVIDER: istiod
CA_ADDR: istiod.istio-system.svc:15012
POD_NAME: ml-pipeline-persistenceagent-67bb7589bb-zxfmk (v1:metadata.name)
POD_NAMESPACE: kubeflow (v1:metadata.namespace)
INSTANCE_IP: (v1:status.podIP)
SERVICE_ACCOUNT: (v1:spec.serviceAccountName)
HOST_IP: (v1:status.hostIP)
ISTIO_CPU_LIMIT: 2 (limits.cpu)
PROXY_CONFIG: {"tracing":{}}
ISTIO_META_POD_PORTS: [
]
ISTIO_META_APP_CONTAINERS: ml-pipeline-persistenceagent
GOMEMLIMIT: 1073741824 (limits.memory)
GOMAXPROCS: 2 (limits.cpu)
ISTIO_META_CLUSTER_ID: Kubernetes
ISTIO_META_NODE_NAME: (v1:spec.nodeName)
ISTIO_META_INTERCEPTION_MODE: REDIRECT
ISTIO_META_WORKLOAD_NAME: ml-pipeline-persistenceagent
ISTIO_META_OWNER: kubernetes://apis/apps/v1/namespaces/kubeflow/deployments/ml-pipeline-persistenceagent
ISTIO_META_MESH_ID: cluster.local
TRUST_DOMAIN: cluster.local
Mounts:
/etc/istio/pod from istio-podinfo (rw)
/etc/istio/proxy from istio-envoy (rw)
/var/lib/istio/data from istio-data (rw)
/var/run/secrets/credential-uds from credential-socket (rw)
/var/run/secrets/istio from istiod-ca-cert (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-fd8vq (ro)
/var/run/secrets/tokens from istio-token (rw)
/var/run/secrets/workload-spiffe-credentials from workload-certs (rw)
/var/run/secrets/workload-spiffe-uds from workload-socket (rw)
Conditions:
Type Status
PodReadyToStartContainers True
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
workload-socket:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
credential-socket:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
workload-certs:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
istio-envoy:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium: Memory
SizeLimit: <unset>
istio-data:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
istio-podinfo:
Type: DownwardAPI (a volume populated by information about the pod)
Items:
metadata.labels -> labels
metadata.annotations -> annotations
istio-token:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 43200
istiod-ca-cert:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: istio-ca-root-cert
Optional: false
persistenceagent-sa-token:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3600
kube-api-access-fd8vq:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events: <none>Pending 상태의 pod
~$ kubectl describe pod metacontroller-0 -n kubeflow
Name: metacontroller-0
Namespace: kubeflow
Priority: 0
Service Account: meta-controller-service
Node: <none>
Labels: app=metacontroller
application-crd-id=kubeflow-pipelines
apps.kubernetes.io/pod-index=0
controller-revision-hash=metacontroller-5b7cc55bc5
kustomize.component=metacontroller
statefulset.kubernetes.io/pod-name=metacontroller-0
Annotations: sidecar.istio.io/inject: false
Status: Pending
IP:
IPs: <none>
Controlled By: StatefulSet/metacontroller
Containers:
metacontroller:
Image: docker.io/metacontrollerio/metacontroller:v2.0.4
Port: <none>
Host Port: <none>
Command:
/usr/bin/metacontroller
--zap-log-level=4
--discovery-interval=3600s
Limits:
cpu: 1
memory: 1Gi
Requests:
cpu: 100m
memory: 256Mi
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8vs4j (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
kube-api-access-8vs4j:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 4m46s (x221 over 18h) default-scheduler 0/1 nodes are available: 1 Insufficient memory. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod.2. webhook service, endpoints 확인
webhook 서버의 service 및 endpoint가 올바르게 구성되었는지 확인
kubectl get svc -n kubeflow
kubectl describe svc kserve-webhook-server-service -n kubeflowservice가 포트 443을 통해 연결되어 있어야 함
~$ kubectl describe svc -n kubeflow kserve-webhook-server-service
Name: kserve-webhook-server-service
Namespace: kubeflow
Labels: app=kserve
app.kubernetes.io/name=kserve
Annotations: <none>
Selector: app.kubernetes.io/name=kserve,app=kserve,control-plane=kserve-controller-manager
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.96.87.29
IPs: 10.96.87.29
Port: <unset> 443/TCP
TargetPort: webhook-server/TCP
Endpoints: 10.244.0.26:9443
Session Affinity: None
Internal Traffic Policy: Cluster
Events: <none>endpoint를 확인하여 서비스가 제대로 연결되었는지 점검
~$ kubectl get enpoints -n kubeflow~$ kubectl describe endpoints kserve-webhook-server-service -n kubeflow
Name: kserve-webhook-server-service
Namespace: kubeflow
Labels: app=kserve
app.kubernetes.io/name=kserve
Anksernotations: endpoints.kubernetes.io/last-change-trigger-time: 2024-12-16T06:37:29Z
Subsets:
Addresses: 10.244.0.26
NotReadyAddresses: <none>
Ports:
Name Port Protocol
---- ---- --------
<unset> 9443 TCP
Events: <none>3. kubernetes validatingwebhookconfiguration 확인
kubernetes의 ValidatingWebhookConfiguration이 올바르게 설정되었는지 확인
kubectl get validatingwebhookconfiguration
kubectl describe validatingwebhookconfiguration serving.kserve.io여기서 webhook이 올바른 URL(https://kserve-webhook-server-service.kserve.svc:443)을 가리키고 있는지 확인
URL이 잘못되었거나 webhook 서버와 매칭되지 않을 경우 수정해야 함
4. namespace 및 TLS 인증서 확인
kserve webhook 서버가 올바른 인증서를 사용하고 있는지 점검
인증서 관련 ConfigMap이나 Secret이 삭제되었거나 손상되었을 가능성 확인
kubectl get secrets -n kubeflow
kubectl describe secret <secret-name>만약 TLS 인증서가 만료되었거나 잘못 설정된 경우, 인증서를 다시 생성하거나 관련 Helm 차트를 사용하여 재설치해야 함
5. kserve 재설치
webhook 서버가 올바르게 작동하지 않는 경우, kserve를 재설치하는 것이 필요할 수 있음
kubectl delete -f ~~.yaml
kubectl apply -f ~~.yaml6. 기타 문제 해결
클러스터의 DNS 서비스가 올바르게 작동하지 않을 수 있음
CoreDNS Pod를 확인
kubectl get pod -n kube-system
kubectl logs <coredns-pod-name> -n kube-system네트워크 정책이나 방화벽 규칙이 webhook 서버와 통신을 차단하고 있는지 점검
