Meta-Operator (Preview Alpha)

Custom installation crafted to your needs using Seldon Meta Operator

Warning

This is alpha preview. The Seldon Meta Operator is currently in alpha phase and it is not production ready. This documentation is provided as preview only in order to gather feedback and encourage testing in development environments.

Seldon Meta-Operator Overview

The Seldon Meta-Operator is a cloud native installation method for the Seldon Enterprise Products that provides an alternative method to efficiently trial/evaluate different components of the Seldon Enterprise ecosystem, as well as a robust production configuration for the Seldon components.

The primary motivation for developing the meta-operator is to solve some of the complexities and overheads involved in the installation and maintenance of the Seldon Enterprise ecosystem. The objectives of the meta-operator include:

  1. Providing a simple, unified method to configure a full Seldon Deploy environment with minimal complexity through built-in, operator-managed components.

  2. Providing a modular way to adopt ecosystem components by enabling feature-specific workflows in a way that is directly related to specific new ecosystem frameworks (e.g. Postgres, Elasticsearch, etc).

  3. Minimising the overheads of maintaining Seldon Enterprise products through the cloud native operator pattern, which ensures the consistency of configurations as well as the health of integrations.

The Meta-Operator has been designed in a modular way which allows for each feature to be programmatically enabled or disabled. The high-level conceptual design can be visualised in the diagram below, and is further explained in the following sections.

component overview diagram

Built-in components

These components include Seldon Core, Seldon Deploy and currently Seldon Core Analytics. They are installed and configured by Seldon Meta-Operator.

Built-in for trial only components

These components include trial instances of Elasticsearch and PostgreSQL. They can be easily installed via the Seldon Meta-Operator for the purpose of trial and testing. Deploying these component via the Meta-Operator can get you quickly up and running but they should be replaced by a proper managed services when moving to production.

Manually installed components

These components are not managed by the Seldon Meta-Operator. Manual installation of them may be required to enable certain features. Among others these include Istio, Knative, Keycloak, ArgoCD, etc.

See the following table for components summary and ecosystem dependencies details:

Component

Built-in Component?

Notes

Seldon Core

Yes

Required for Seldon ecosystem

Seldon Deploy

Yes

Required for advanced Seldon functionality

Seldon Core Analytics

Yes

Optional - metrics, monitoring, and alerting

Istio

No

Required for external routing and access

PostgreSQL

Yes (for trial)

Optional - model metadata

Elasticsearch

Yes (for trial)

Optional - request logging

Keycloak

No

Required for restricting access; authn and authz provider

Fluentd

No

Optional - application and audit logging

Knative

No

Optional - advance monitoring features

ArgoCD

No

Optional - required for gitops

Argo Workflows

No

Optional - required for batch

Prerequisites

Kubernetes cluster with kubectl access with cluster-admin privileges.

Meta Operator Installation

Installation

To install Seldon Meta Operator:

kubectl create namespace seldon-system || true
kubectl apply -f https://storage.googleapis.com/seldonoperator/seldonoperator_install.yaml

Verification

After a moment you should see a pod running in seldon-system namespace. Run following command to verify

$ kubectl get pods -n seldon-system
NAME                                                 READY   STATUS    RESTARTS   AGE
seldon-operator-controller-manager-9fbc785bc-hwh5s   2/2     Running   0          10m

Minimal Seldon Installation

The minimal installation will consist only of Seldon Core and Seldon Deploy. It is recommended to start with minimal installation and build on it.

Installation

To install Seldon create seldon-instance.yaml manifest

apiVersion: operator.seldon.io/v1alpha1
kind: SeldonOperator
metadata:
  name: seldonoperator-sample
  namespace: seldon-system
spec: {}

and apply it with

kubectl apply -f seldon-instance.yaml

Note

In the above manifest the spec: {} section is empty. In successive parts of this document we will be modifying the spec section as we enable / configure features. For brevity of each section only the relevant changes will be given as

apiVersion: operator.seldon.io/v1alpha1
kind: SeldonOperator
...
spec:
  <relevant changes>
...

After each modification to the seldon-instance.yaml manifest execution of

kubectl apply -f seldon-instance.yaml

command is required.

It is recommended to keep seldon-instance.yaml manifest under VCS.

Verification

Two new pods should appear in seldon-system namespace. Run following command to verify

$ kubectl get pods -n seldon-system
NAME                                                 READY   STATUS    RESTARTS   AGE
seldon-controller-manager-8457b8b5c7-wh9ls           1/1     Running   0          26s
seldon-deploy-57d947f768-9vhx5                       1/1     Running   0          19s
seldon-operator-controller-manager-9fbc785bc-hwh5s   2/2     Running   0          12m

Getting Access

You can now get access to Seldon Deploy through svc/seldon-deploy created in seldon-system namespace

$ kubectl get svc -n seldon-system | grep seldon-deploy
seldon-deploy                                        ClusterIP   10.96.239.72   <none>        80/TCP     116s

You use the created svc as any other in your cluster by configuring ingress of your choice. Seldon Deploy officially only support Istio ingress that can be configured as described in routing section.

Port-forwarding allows to easily get access to the Seldon Deploy UI on developer’s local machine.

Execute a following port-forward command in terminal:

kubectl port-forward -n seldon-system svc/seldon-deploy 8080:80

Open following URL in your browser: http://localhost:8080/seldon-deploy/

After getting access to the Seldon Deploy UI you will be welcomed by the license prompt. Please enter your license now to get access to the Seldon Deploy UI.

Built-In Trial Ecosystem Frameworks

The Seldon Meta Operator allows to easily deploy a built-in instances of:

  • Prometheus (via Seldon Core Analytics)

  • Elasticsearch

  • PostgreSQL

which enables advance features such as metrics/monitoring, payload logging and metadata store.

Warning

The built-in instances of Elasticsearch and PostgreSQL are meant for development and testing. For production deployments we encourage to use managed solutions as discussed in further sections of this document.

Configuration

To enable the built-in features apply a following configuration to the seldon-instance.yaml resource:

apiVersion: operator.seldon.io/v1alpha1
kind: SeldonOperator
...
spec:
  enable:
    analytics: true      # enables managed instance of Seldon Core Analytics
    catalogue: true      # enables built-in trial instance of PostgreSQL
    payloadstore: true   # enables built-in trial instance of Elasticsearch
...

Verification

You can verify successful deployment of built-in features with

$ kubectl get pods -n seldon-system | grep -P "internal|analytics"
seldon-core-analytics-kube-state-metrics-79b4fd4bf7-crg77       1/1     Running             0          55s
seldon-core-analytics-prometheus-alertmanager-9bc86c99d-pjhl2   2/2     Running             0          55s
seldon-core-analytics-prometheus-node-exporter-95ps5            1/1     Running             0          55s
seldon-core-analytics-prometheus-pushgateway-7c669fdb57-5q6hc   1/1     Running             0          55s
seldon-core-analytics-prometheus-seldon-6b7c7d4dd4-4pn62        2/2     Running             0          55s
seldon-internal-opensearch-cluster-master-0                     1/1     Running             0          106s
seldon-internal-opensearch-cluster-master-1                     1/1     Running             0          106s
seldon-internal-opensearch-cluster-master-2                     1/1     Running             0          106s
seldon-internal-postgres-postgresql-0                           1/1     Running             0          85s`

Enable Routing

Requirements

This feature requires Istio. The responsibility of installation and maintenance of Istio lies on customer.

Advance routing can be enabled given that Istio is installed in the Kubernetes cluster. Enabling routing allows to create canary and shadow deployments. It is also prerequisite for enabling detectors.

Installation

The routing component requires Istio. Assuming that istioctl CLI tool is present it can be installed with:

istioctl install -y

For further context on Istio installation please refer to our main documentation and Istio documentation.

Configuration

To enable routing functionality apply one of following modifications to the seldon-instance.yaml depending on your use case.

By default Seldon Meta-Operator creates a simple HTTP Gateway like the one from our main documentation.

apiVersion: operator.seldon.io/v1alpha1
kind: SeldonOperator
...
spec:
  enable:
    routing: true
...

The built-in Gateway will be created in CRD instance namespace - in our example it is seldon-system:

$ kubectl get gateway -n seldon-system
NAMESPACE       NAME             AGE
seldon-system   seldon-gateway   84s

Getting Access

One routing is enabled you can access

ISTIO_INGRESS=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo "Seldon Deploy: http://$ISTIO_INGRESS/seldon-deploy/"

Enabling Advance Monitoring

Requirements

This feature requires Knative and Istio. The responsibility of installation and maintenance of these components lies on customer.

Installation

For details on installing Istio please refer to Istio section.

To install Knative Eventing:

kubectl apply -f https://github.com/knative/eventing/releases/download/knative-v1.1.0/eventing-crds.yaml
kubectl apply -f https://github.com/knative/eventing/releases/download/knative-v1.1.0/eventing-core.yaml
kubectl apply -f https://github.com/knative/eventing/releases/download/knative-v1.1.0/in-memory-channel.yaml
kubectl apply -f https://github.com/knative/eventing/releases/download/knative-v1.1.0/mt-channel-broker.yaml

To install Knative Serving

kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.1.0/serving-crds.yaml
kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.1.0/serving-core.yaml
kubectl apply -f https://github.com/knative/net-istio/releases/download/knative-v1.1.0/net-istio.yaml

For furthe rcontext on Knative installation please refer to our main documentation and Knative documentation.

Configuration

Following configuration instructs Seldon Meta-Operator to use enable detectors features:

apiVersion: operator.seldon.io/v1alpha1
kind: SeldonOperator
...
spec:
  enable:
    detectors: true
...

Verification

To verify if broker and triger isare created

kubectl get brokers,triggers -n seldon-system
NAME                                  URL                                                                              AGE   READY   REASON
broker.eventing.knative.dev/default   http://broker-ingress.knative-eventing.svc.cluster.local/seldon-system/default   11m   True

NAME                                                         BROKER    SUBSCRIBER_URI                                                 AGE   READY   REASON
trigger.eventing.knative.dev/seldon-request-logger-trigger   default   http://seldon-request-logger.seldon-system.svc.cluster.local   11m   True

Enabling External Payloadstore

Requirements

This feature requires Elasticsearch. The responsibility of installation and maintenance of Elasticsearch lies on customer.

Installation

The details on installing Elasticsearch are covered in our main documentation.

Configuration

Following configuration instructs Seldon Meta-Operator how to connect to external instance of Elastic:

apiVersion: operator.seldon.io/v1alpha1
kind: SeldonOperator
...
spec:
  enable:
    payloadstore: true
  payloadstore:
    internal_elastic:
      use_internal: false
  enterprise:
    values:
      elasticsearch:
        url: "https://opensearch-cluster-master.seldon-logs.svc.cluster.local:9200"
        basicAuth: true
        secret:
          name: elastic-credentials
          userKey: "username"
          passwordKey: "password"
      requestLogger:
        elasticsearch:
          host: "opensearch-cluster-master.seldon-logs.svc.cluster.local"
          port: "9200"
          protocol: "https"
...

Further Configuration

Compatibility with Production Installation

The above documentation covers only subset of possible configuration options that Seldon Meta-Operator offers.

More documentation of various Seldon Deploy features can be find in our main Production Installation documentation. To follow the main documentation with Seldon Meta-Operator simply provide all helm values in the relevant sections of SeldonOperator CRD instance.

Seldon Meta-Operator offer same configuration flexibility as the usual helm-based installation. When following the main production installation docs skip all helm upgrade ... commands for Seldon Core and Seldon Deploy and instead add the relevant helm values as described below.

To configure Seldon Core Helm Values modify the spec.core.values section of SeldonOperator CRD instance:

apiVersion: operator.seldon.io/v1alpha1
kind: SeldonOperator
...
spec:
  core:
    values:
      <Seldon Core Helm Values Here>
...

Removing Meta Operator

To remove Seldon Meta-Operator first remove SeldonOperator CRD instance

kubectl delete -f seldon-instance.yaml

Once deletion of resources is completed remove the operator

kubectl delete -f https://storage.googleapis.com/seldonoperator/seldonoperator_install.yaml

Known Issues

Removing Meta Operator Before Removing CRDs

If you remove Seldon Meta-Operator before removing all seldonoperator instances the removal process will hang on finalizers. To workaround this kubectl edit -n seldon-system seldon-operator seldonoperator-sample and manually remove the finalizers section.

Outlier / Drift Detectors Reply URL

When creating Outlier or Drift Detectors the default reply_url will be http://seldon-request-logger.seldon-logs. The Seldon Meta-Operator creates logger in seldon-system namespace. As currently there is no way to configure default reply_url one need to customize it manually each time when creating a new detector.