Istio Ingress¶

Configure Seldon ingress with Istio

Istio is a production-ready service mesh solution, which Seldon Enterprise Platform uses for routing of traffic. Similarly, traffic can also be managed by Istio for dynamically created Seldon models, which simplifies how users can send requests to a single endpoint.

Important

Before starting the installation procedure, please download installation resources as explained here and make sure that all pre-requisites are satisfied.

This page assumes that Seldon Enterprise Platform is installed.

You can use the Istio CLI to install the Istio containers into your cluster. Istio provides multiple installation configuration profiles, it is important to choose the most appropriate configuration, especially as Istio is often used for applications beyond just Seldon itself. You can read more about the profiles and configurations in the Istio documentation.

Installing Istio¶

Warning

This section is for users installing Istio for the first time.

If you already have Istio, but a newer version is required, please refer to the official documentation for details on upgrading an existing installation instead.

Either way, you will still need to configure Seldon for Istio after installing or upgrading.

First, obtain istioctl CLI that will help in installation of Istio. We will be installing version 1.17.1 which is supported with kubernetes clusters version 1.23 …1.26. The compatibility matrix between Istio and different version of kubernetes can be found here.

export ISTIO_VERSION=1.17.1
curl -L https://istio.io/downloadIstio | ISTIO_VERSION=${ISTIO_VERSION} sh -

./istio-${ISTIO_VERSION}/bin/istioctl install --set profile=default -y
./istio-${ISTIO_VERSION}/bin/istioctl verify-install

It’s worth emphasising that Istio in itself is quite a feature-rich open source project, which is often leveraged as central service mesh for clusters. If the use-cases are more specific, there are quite a lot of different configurations that can be chosen, examples covered above include the different installation profiles, but also there are instructions like the documentation in the Istio section for Knative.

Installing Seldon Core and Seldon Enterprise Platform with Istio Configuration Enabled¶

In order to install Seldon with Istio enabled there are a couple of requirements, namely:

An Istio Gateway needs to be created
Seldon Core Helm Chart needs to be installed with the Istio values
Seldon Enterprise Platform Helm Chart needs to be installed with the Istio values

Setting up Istio Gateway¶

Istio Gateway is required to access deployments and Seldon Enterprise Platform itself.

For example, create a istio-gateway.yaml file with the following content:

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: seldon-gateway
  namespace: istio-system
spec:
  selector:
    app: istio-ingressgateway
    istio: ingressgateway
  servers:
    - hosts:
        - "*"
      port:
        name: http
        number: 80
        protocol: HTTP

You can then setup the Istio Gateway with the following command:

kubectl apply istio-gateway.yaml

Alternatively, to set up an HTTPS-enabled ingress, please follow the official Istio documentation for a secure gateway.

The Istio gateway can be created in the Istio namespace, but it can also be created in a different namespace. You will need to make sure that the configurations of Seldon Enterprise Platform and Seldon Core are also aligned to the namespace and gateway name.

Install Seldon Core with Istio Enabled¶

You can now add the following values in your core-values.yaml file (which you’ll need to create, if you haven’t yet done so). You need to make sure that the value for the Gateway is <namespace>/<gateway name>.

istio:
  enabled: true
  gateway: "istio-system/seldon-gateway"

Install Seldon Enterprise Platform with Istio Enabled¶

Similarly you would add following entries to your deploy-values.yaml file. Remember to update {{ISTIO_INGRESS}} with the correct address, which can be found as described below.

ingressGateway:
  seldonIngressService: "istio-ingressgateway"
  ingressNamespace: "istio-system"

virtualService:
  create: true
  gateways:
    - istio-system/seldon-gateway

seldon:
  curlForm: |
    curl -k https://{{ISTIO_INGRESS}}/seldon/{{ .Namespace }}/{{ .ModelName }}/api/v0.1/predictions \<br/>
    &nbsp;&nbsp;-H "{{ .TokenHeader }}: {{ .Token }}" \<br/>
    &nbsp;&nbsp;-H "Content-Type: application/json" \<br/>
    &nbsp;&nbsp;-d '{{ .Payload }}'
  tensorFlowCurlForm: |
    curl -k https://{{ISTIO_INGRESS}}/seldon/{{ .Namespace }}/{{ .ModelName }}/v1/models/:predict \<br/>
    &nbsp;&nbsp;-H "{{ .TokenHeader }}: {{ .Token }}" \<br/>
    &nbsp;&nbsp;-H "Content-Type: application/json" \<br/>
    &nbsp;&nbsp;-d '{{ .Payload }}'

seldonCoreV2:
  curlForm: |
    curl -k https://{{ISTIO_INGRESS}}/v2/models/{{ .ModelName }}/infer \<br/>
    &nbsp;&nbsp;-H "Host: {{ .Namespace }}.inference.seldon" \<br/>
    &nbsp;&nbsp;-H "Content-Type: application/json" \<br/>
    &nbsp;&nbsp;-H "Seldon-Model: {{ .ModelName }}.pipeline" \<br/>
    &nbsp;&nbsp;-d '{{ .Payload }}'
  enabled: true
  requestForm: '{{ .SeldonProtocol }}://seldon-mesh.{{ .Namespace }}.svc.cluster.local/v2/pipelines/{{
    .ModelName }}/infer'

Find address of your Seldon Enterprise Platform¶

Following script can help you find address of your Seldon Enterprise Platform instance running with Istio:

ISTIO_INGRESS=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
ISTIO_INGRESS+=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

echo "Seldon Enterprise Platform: http://$ISTIO_INGRESS/seldon-deploy/"