KFserving Installation

This section will walk you through installation procedure of KFserving such that it is ready to be used with Seldon Deploy.

Important

Before starting the installation procedure, please download installation resources as explained here and make sure that all pre-requisites are satisfied.

This page also assumes that main Seldon components and Istio are installed.

Version requirements

We will install version v0.6.1 of KFServing. This version requires:

  • Istio in version v1.9.0+

  • Knative Serving in version v0.19.0+

Note: prior to version v0.19 Knative Serving required configuration of cluster-local-gateway for Istio. This is no longer required. If you are running older versions of Knative and want to use KFserving you need to configure the cluster local gateway. Please refer to an older version of the documentation to do so. It is, however, recommended to use the versions specified above.

Install Knative Serving

Knative Serving is required for KFServing. See Knative section for steps.

Install KFServing

Install KFServing

KFSERVING_VERSION=v0.6.1
kubectl apply -f https://github.com/kubeflow/kfserving/releases/download/${KFSERVING_VERSION}/kfserving.yaml

and configure KFServing by creating patch file kf-patch.yaml

data:
  ingress: |-
    {
      "ingressGateway" : "istio-system/seldon-gateway",
      "ingressService" : "istio-ingressgateway.istio-system.svc.cluster.local",
      "localGateway" : "knative-local-gateway.knative-serving",
      "localGatewayService" : "knative-local-gateway.istio-system.svc.cluster.local"
    }

  logger: |-
    {
      "image" : "kfserving/agent:v0.6.1",
      "memoryRequest": "100Mi",
      "memoryLimit": "1Gi",
      "cpuRequest": "100m",
      "cpuLimit": "1",
      "defaultUrl": "http://broker-ingress.knative-eventing.svc.cluster.local/seldon-logs/default"
    }

and apply it with

kubectl patch cm -n kfserving-system inferenceservice-config -p "$(cat kf-patch.yaml)"

Also, if you are running without cert-manager create self-signed CA with

wget https://raw.githubusercontent.com/kubeflow/kfserving/v0.6.1/hack/self-signed-ca.sh
chmod +x self-signed-ca.sh
./self-signed-ca.sh

Configure Seldon Deploy

Enable KFserving in Seldon Deploy by adding a following section to your deploy-values.yaml files

kfserving:
  protocol: "http"
  enabled: true

Hint

Note that if your istio Gateway uses https then set protocol to https.