Knative Installation

Warning

Knative requires Istio to be installed.

This page is for users installing Knative for the first time.

If you already have Knative, but a newer version is required, please refer to the official documentation for details on upgrading an existing installation instead.

You will still need to configure a broker for Seldon Deploy, regardless of whether you install or upgrade Knative components. You may also wish to refer to the tests validating installations and the auto-scaling configuration discussed on this page.

This section walks through installation of Knative for Seldon Deploy.

Knative Eventing and Serving are used for request logging and for post-predict detector components (outlier, drift, metrics). See more in data events logging.

Knative can be installed in other ways - if you have an existing install then just apply steps that customize.

Important

Before starting the installation procedure, please download installation resources as explained here and make sure that all pre-requisites are satisfied.

This page also assumes that main Seldon Core and Seldon Deploy components are installed.

Version requirements

According to Knative documentation, the following versions of Kubernetes and Istio are compatible with the given versions of Knative (both Eventing and Serving):

Knative version	Kubernetes version	Istio version
1.8.x	1.23+	1.15.x – recommended
1.7.x	1.22+	1.14.x – recommended
1.6.x	1.22+	1.14.x – recommended
1.5.x	1.22+	1.13.x – recommended
1.4.x	1.22+	1.13.x – recommended
1.3.x	1.21+	1.12.x – recommended
1.2.x	1.21+	1.12.x – recommended
1.1.x	1.20+	1.9.x – recommended
1.0.x	1.20+	1.9.x – recommended

We will be installing version v1.8.0 of Knative.

Install Knative Serving

Run the following shell commands, changing the version as required:

KNATIVE_SERVING_URL="https://github.com/knative/serving/releases/download"
NET_ISTIO_URL="https://github.com/knative/net-istio/releases/download"
SERVING_VERSION="knative-v1.8.0"

kubectl apply -f ${KNATIVE_SERVING_URL}/${SERVING_VERSION}/serving-crds.yaml
kubectl apply -f ${KNATIVE_SERVING_URL}/${SERVING_VERSION}/serving-core.yaml

kubectl apply -f ${NET_ISTIO_URL}/${SERVING_VERSION}/net-istio.yaml

If you are using Seldon Core Analytics for Prometheus, then for Knative metrics add these annotations:

kubectl annotate -n knative-serving service autoscaler prometheus.io/scrape=true
kubectl annotate -n knative-serving service autoscaler prometheus.io/port=9090

kubectl annotate -n knative-serving service activator-service prometheus.io/scrape=true
kubectl annotate -n knative-serving service activator-service prometheus.io/port=9090

Test Knative Serving

To check the installed version of Knative Serving:

kubectl get namespace knative-serving -o 'go-template={{index .metadata.labels "serving.knative.dev/release"}}'

To verify the install, first create a file containing the below:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: helloworld-go
  namespace: default
spec:
  template:
    spec:
      containers:
        - image: gcr.io/knative-samples/helloworld-go
          env:
            - name: TARGET
              value: "Go Sample v1"

Next, start watching pods in the default namespace using a different terminal window:

kubectl get pod -n default -w

And apply the file from the first step and curl it with the below:

kubectl apply -f <filename>

kubectl run --quiet=true -it --rm curl --image=radial/busyboxplus:curl --restart=Never -- \
  curl -v -X GET "http://helloworld-go.default.svc.cluster.local"

You should get a successful response and see a pod come up in the default namespace. If you don’t, then see the note below on private registries before looking at resources such as the Seldon and Knative Slack channels.

Clean up with:

kubectl delete -f <filename>

Knative Serving auto-scaling and bounds

You can configure upper and lower bounds to control autoscaling behavior with Knative services. Seldon Deploy configures the outlier, drift detectors and metrics servers as Knative services and the scale bounds are automatically setup upon deployment as revision annotations.

It is a good practice to control the maximum number of replicas that each revision should have and the initial scale for optimal performance. Knative will attempt to never have more than this number of replicas running or in the process of being created at any one point in time. In the current Seldon Deploy setup, it is most useful to configure these bounds for outlier detectors, since drift detectors and metrics servers are not auto-scalable.

Such max-scale limits can be set at a global level as per Knative documentation. For example, the following ConfigMap would set auto-scaling upper limits.

apiVersion: v1
kind: ConfigMap
metadata:
  name: config-autoscaler
  namespace: knative-serving
data:
  max-scale: "3"
  max-scale-limit: "100"

Install Knative Eventing

Run the following shell commands, changing the version as required:

KNATIVE_EVENTING_URL=https://github.com/knative/eventing/releases/download
EVENTING_VERSION="knative-v1.8.0"

kubectl apply -f ${KNATIVE_EVENTING_URL}/${EVENTING_VERSION}/eventing-crds.yaml
kubectl apply -f ${KNATIVE_EVENTING_URL}/${EVENTING_VERSION}/eventing-core.yaml

kubectl apply -f ${KNATIVE_EVENTING_URL}/${EVENTING_VERSION}/in-memory-channel.yaml
kubectl apply -f ${KNATIVE_EVENTING_URL}/${EVENTING_VERSION}/mt-channel-broker.yaml

You can check the progress of the rollout using the below:

kubectl rollout status -n knative-eventing deployment/imc-controller

Configure Knative Eventing broker

Define a Knative Event broker to handle the logging by creating a file with the below:

apiVersion: eventing.knative.dev/v1
kind: Broker
metadata:
  name: default
  namespace: seldon-logs

And apply this:

kubectl apply -f <filename>

Test Knative Eventing

To check the installed version of Knative Eventing:

kubectl get namespace knative-eventing -o 'go-template={{index .metadata.labels "eventing.knative.dev/release"}}'

To test Knative Eventing it is easiest to have Seldon fully installed with request logging and a model running.

Make a prediction to a model following one of the Seldon Core demos. You should see entries under Requests.

If you see entries under requests, you are all good.

If you don’t see entries under requests, first find the request logger pod in the seldon-logs namespace. Tail its logs (kubectl logs -n seldon-logs <pod-name> -f) and make a request again. Do you see output?

If this doesn’t work then find the pod for the model in your SeldonDeployment. Tail the logs of the seldon-container-engine container and make a prediction again.

If the predictions aren’t sending then it could be a problem with the broker URL (executor.requestLogger.defaultEndpoint in helm get values -n seldon-system seldon-core) or the broker (kubectl get broker -n seldon-logs).

If there are no requests and no obvious problems with the broker transmission, then it could be the trigger stage.

First try kubectl get trigger -n seldon-logs to check the trigger status.

If that looks healthy then we need to debug the Knative trigger process.

Do a kubectl apply -f to the default namespace on a file containing the below (or change references to default for a different namespace):

# event-display app deploment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: event-display
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels: &labels
      app: event-display
  template:
    metadata:
      labels: *labels
    spec:
      containers:
        - name: helloworld-python
          image: gcr.io/knative-releases/github.com/knative/eventing-sources/cmd/event_display
---
# Service that exposes event-display app.
# This will be the subscriber for the Trigger
kind: Service
apiVersion: v1
metadata:
  name: event-display
  namespace: default
spec:
  selector:
    app: event-display
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
---
# Trigger to send events to service above
apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
  name: event-display
  namespace: seldon-logs
spec:
  broker: default
  subscriber:
    uri: http://event-display.default:80

Now find the event-display pod and tail its logs (kubectl get pod -n default and kubectl logs -n default <pod_name>). Make a prediction to a model following one of the Seldon Core demos.

You should see something in the event-display logs – even an event decoding error message is good.

To eliminate any Seldon components, we can send an event directly to the broker. There is an example in the Knative docs.

What we’ve done now corresponds to the Knative Eventing hello-world. Nothing at all in the event-display pod means Knative Eventing is not working.

Occasionally you see a RevisionMissing status on the ksvc and a ContainerCreating message on its Revision. If this happens check the Deployment, and if there are no issues then delete and try again.

Hopefully you’ve got things working before here. If not then check the pods in the knative-eventing namespace. If that doesn’t help find the problem, then the Knative Slack and/or Seldon Slack can help with further debugging.

Enable Knative support in Seldon Deploy

To Enable v1 support in Seldon Deploy add or change a following variable in deploy-values.yaml file

seldon:
  knativeEnabled: true

Once you modify your deploy-values.yaml you need to apply it with

helm upgrade seldon-deploy seldon-charts/seldon-deploy \
    -f deploy-values.yaml \
    --namespace=seldon-system \
    --version 2.1.1 \
    --install

Knative with a private registry

By default Knative assumes the image registries used for images will be public. If you use a company-internal private one, then you have to configure that.

There is a property called queueSidecarImage in the config-deployment ConfigMap in the knative-serving namespace. This needs to be edited to point to your registry.

Tag resolution configuration can also be required. We suggest to try the above first. The Knative docs provide more details on tag resolution.

Configure auto-scaling

The upper bound of scaling can be configured globally for Knative using the ConfigMap.

apiVersion: v1
kind: ConfigMap
metadata:
  name: config-autoscaler
  namespace: knative-serving
data:
  max-scale: "3"
  max-scale-limit: "100"