Secrets Management

Seldon Deploy can create and manage secrets for models or artifacts stored in private buckets, or custom container images stored in a private registry.

If OPA is enabled, you will need write permission to the relevant namespace in order to create or delete secrets in it.

Management Methods

Seldon Deploy supports the management of secrets through both its UI and its API.

UI

You can find the Secrets Management page by going to the user icon in the top-right of the UI and clicking on Secrets.

secrets menu

You will be greeted by a page similar to the following.

secrets management panel

API

The secrets API is provided under the SecretsService endpoints.

You can view these here in the documentation, or by navigating to the user icon in the top-right of the Seldon Deploy UI and clicking on API Docs.

These endpoints are also made available through the Seldon Deploy Python SDK.

secrets menu

Bucket Secrets

Bucket stores, also called blob or object stores, provide storage for arbitrary data files. These might be binary files for serialized models, JSON or other textual formats for model settings, or even inference requests for batch jobs or reference data. Examples of bucket stores include Amazon S3, Azure Blob Storage, and Google Cloud Storage (GCS).

Individual storage buckets can be publicly accessible, such as Seldon’s seldon-models. Alternatively, they can be private and protected with some form of secret or access credentials.

Bucket secrets are used in Seldon for downloading model artifacts and inference data from these private bucket stores. It is a storage initializer that actually uses the secret to perform the download. Secrets need to be provided in a format that the storage initializer understands. The following sections discuss these uses of bucket secrets in more detail.

Seldon Core v1

The default storage initializer for Core v1 is Rclone, although custom initializers can be used instead.

Core v1 uses environment variables to pass configuration to the storage initializer for a model. As such, the keys in a Core v1 bucket secret must be valid environment variables for the initializer.

We provide easy configuration of Rclone-compatible bucket secrets for both S3 and GCS. You can also create generic Rclone secrets, which unlocks a wide variety of providers.

You will always need to specify a remote name, which will be used to create a secret called <remote>-bucket-envvars. This remote name is your choice and determines how you will refer to your bucket provider in storage URIs, i.e. <remote>://<path to bucket object>.

A GCS bucket secret only requires a Remote name and Account Credentials. Please see the Google Cloud Auth documentation for how to obtain these credentials for your bucket.

create GCS secret

Seldon Core v2

Rclone is the only supported storage initializer for Core v2.

Core v2 uses a different format for bucket secrets from that of Core v1. Instead of providing environment variables to the storage initializer, Core v2 communicates directly with it. It uses Rclone’s connection string format.

You need to select Config Parameters as the Bucket Format. You can then choose a Remote Name and Remote Type, and provide configuration parameters for that type as key-value pairs.

You can find the appropriate Rclone configuration parameters by selecting your storage provider and checking the available options. Note that you should use the config variant of each Rclone parameter.

The remote name will be used to create a secret called <remote>-bucket-params.

create Rclone secret for Core v2

Inference Data

Inference data can be retrieved from and written to storage buckets via batch jobs and reference data uploads.

Batch jobs and reference data jobs in Seldon Deploy use the same approach as Seldon Core v1 for bucket secrets. Namely, they use environment variables to pass configuration from bucket secrets to a storage initializer. The format is exactly the same as for Core v1 and the same secrets can be used.

The default storage initializer for batch and reference data jobs is Rclone.

Registry Secrets

Private container registries can be used when deploying a custom runtime model.

When creating a registry secret, you need to specify the Secret Name, which is your choice, and the Account Credentials.

create registry secret

The format of container registry credentials can be seen by running the following example. For further information on creating and interacting with registry secrets, please refer to the Kubernetes documentation.

docker login
cat ~/.docker/config.json

Note

The above example displays your personal credentials, which should not be used in a production system.

Using Secrets

Once the secrets have been created, you can use them in the Seldon Deploy UI, e.g. in the Deployment Wizard, and through the API.

For bucket secrets, you need to use the Remote Name you chose as the URI prefix. For example, if you named your remote minio then your URI should start with minio://.

For Core v1, the Remote Name of your bucket secret should match the prefix of your model URI.

using Core v1 bucket secrets in the deployment wizard

You can use registry secrets for custom model runtimes.

using Core v1 registry secrets in the deployment wizard

Note

In the Seldon Deploy UI, you will only be shown secrets with the appropriate format for that context. For example, when creating a Core v1 deployment you will only see env-var bucket secrets.

For backwards compatibility, any bucket secrets that are not labelled with their format are assumed to be in the env-var format.