Secrets Management¶

Seldon Enterprise Platform can create and manage secrets for models or artifacts stored in private buckets, or custom container images stored in a private registry.

If OPA is enabled, you will need write permission to the relevant namespace in order to create or delete secrets in it.

Management Methods¶

Seldon Enterprise Platform supports the management of secrets through both its UI and its API.

You can manage secrets by navigating to the user icon in the top-right of the Seldon Enterprise Platform UI and clicking on Secrets.

You will be greeted by a page similar to the following.

secrets management panel

You can manage secrets via the API by using the provided endpoints under the SecretsService section in the API docs. Those can be accessed either:

by navigating to the user icon in the top-right of the Seldon Enterprise Platform UI and clicking on the API docs or
by navigating to the corresponding docs page.

In addition to this, the endpoints are also made available through the Seldon Enterprise Platform Python SDK.

Bucket Secrets¶

Bucket stores, also called blob or object stores, provide storage for arbitrary data files. These might be binary files for serialized models, JSON or other textual formats for model settings, or even inference requests for batch jobs or reference data. Examples of bucket stores include Amazon S3, Azure Blob Storage, and Google Cloud Storage (GCS).

Storage buckets can be publicly accessible, such as Seldon’s seldon-models bucket, or they can be private and protected with some form of secret or access credentials.

Bucket secrets are used in Seldon for accessing various types of data, such as model artifacts and inference data from private bucket stores. It is a storage initializer that actually uses the secret to perform the download. Secrets need to be provided in a format that the storage initializer understands. The following sections discuss these uses of bucket secrets in more detail.

Seldon Core v1¶

The default storage initializer for Core v1 is Rclone, although custom initializers can be used instead.

You can easily configure Rclone-compatible bucket secrets for both S3 and GCS from the UI, or you can also create generic Rclone bucket secrets, which unlocks a wide variety of providers, such as Microsoft Azure Blob Storage, Dropbox, and many more.

Seldon Core v1 uses environment variables to pass configuration to the storage initializer for a model. As such, the keys in a Core v1 bucket secret must be valid environment variables for the initializer.

When creating a bucket secret, you will need to specify a Remote name which will be used to create a secret called <remote>-bucket-envvars. This remote name is your choice and determines how you will refer to your bucket provider in storage URIs, i.e. <remote>://<path to bucket object>.

A GCS bucket secret only requires a Remote name and Account Credentials. Please see the Google Cloud Auth documentation for how to obtain these credentials for your bucket.

create GCS secret

For example, to create a GCS bucket secret with a remote called test using a Service Account’s private key, in JSON, you need to fill the form in such a way.

create GCS secret filled

This, in turn, will create a bucket secret called test-bucket-envvars. The secret will be made available for selection in Core v1-related wizards and the remote name can be used for specifying private resources, i.e. test://<path to bucket objects>.

Along with a Remote name, an S3 bucket secret requires an Access Key ID and a Secret Access Key. Please see the AWS documentation for obtaining these.

create S3 secret

You need to select Env Vars as the Bucket Format for generic Rclone secrets for Core v1. You can then choose a Remote Name and Remote Type, and provide configuration parameters for that type as key-value pairs.

Each configuration key will become an environment variable. These keys must be of the form:

RCLONE_CONFIG_<remote name>_<config parameter>

The values <remote name> and <configuration parameter> must be given in uppercase. Further details on this format can be found in the Rclone documentation.

You can find the appropriate Rclone configuration parameters by selecting your storage provider and checking the available options. For example, S3 has standard options and advanced options. Note that you should use the config variant of each option as the <config parameter>.

create Rclone secret

An example for a Dropbox initializer is shown below for a remote called dropbox:

RCLONE_CONFIG_DROPBOX_CLIENT_ID: <client_id_value>
RCLONE_CONFIG_DROPBOX_CLIENT_SECRET: <client_secret_value>

Seldon Core v2¶

Unlike Seldon Core v1, where custom storage initializers can be used, Rclone is the only supported storage initializer for Core v2.

To create a bucket secret, you need to select the Generic Rclone option for Bucket Type.

After that, make sure to select Config Parameters as the Bucket Format. Because Core v2 communicates directly with the storage initializer by using a different format for bucket secrets, you don’t have to provide environment variables. Instead, it uses Rclone’s connection string format.

Then, you need to specify a Remote name which will be used to create a secret called <remote>-bucket-params. This remote name is your choice and determines how you will refer to your bucket provider in storage URIs, i.e. <remote>://<path to bucket object>.

After that, you need to specify the Remote Type. This input value is Rclone specific i.e. you need to find out that store’s remote type, according to Rclone. For example, the remote type for GCS is google cloud storage, whereas for S3 is s3. Those can be found here and here respectively for GCS and S3.

Finally, you need to provide configuration parameters for that type as key-value pairs. You can find the appropriate Rclone configuration parameters by selecting your storage provider and checking the available options. Note that you should use the config variant of each Rclone parameter.

To create a Seldon Core v2 GCS bucket secret with a remote called test, you need to specify several other pieces of information.

The Remote type for GCS is google cloud storage and the form of authentication used in this example is the Rclone specific config parameter service_account_credentials. The value of that config parameter is a JSON object, associated with a Service Account key that has the necessary permissions.

create GCS secret for Core v2

This, in turn, will create a bucket secret called test-bucket-params. The secret will be made available for selection in Core v2-related wizards and the remote name can be used for specifying private resources, i.e. test://<path to bucket objects>.

To create a Seldon Core v2 S3 bucket secret with a remote called test, you need to specify several other pieces of information.

The Remote type for S3 is s3 and the form of authentication used in this example is the combination of Rclone specific config parameters access_key_id and secret_access_key. The values of those parameters are usually only seen when a new access key is created for an AWS User.

create S3 secret for Core v2

This, in turn, will create a bucket secret called test-bucket-params. The secret will be made available for selection in Core v2-related wizards and the remote name can be used for specifying private resources, i.e. test://<path to bucket objects>.

Inference Data¶

Inference data can be retrieved from and written to storage buckets via batch jobs and reference data uploads.

Batch jobs and reference data jobs in Seldon Enterprise Platform use the same approach as Seldon Core v1 for bucket secrets. Namely, they use environment variables to pass configuration from bucket secrets to a storage initializer. The format is exactly the same as for Core v1 and the same secrets can be used.

The default storage initializer for batch and reference data jobs is Rclone.

Registry Secrets¶

Private container registries can be used when deploying a custom runtime model.

When creating a registry secret, you need to specify the Secret Name, which is your choice, and the Account Credentials.

create registry secret

The format of container registry credentials can be seen by running the following example. For further information on creating and interacting with registry secrets, please refer to the Kubernetes documentation.

docker login
cat ~/.docker/config.json

Note

The above example displays your personal credentials, which should not be used in a production system.

Using Secrets¶

Once the secrets have been created, you can use them in the Seldon Enterprise Platform UI, e.g. in the Deployment Wizard, and through the API.

For bucket secrets, you need to use the Remote Name you chose as the URI prefix. For example, if you named your remote minio then your URI should start with minio://.

For Core v1, the Remote Name of your bucket secret should match the prefix of your model URI.

using Core v1 bucket secrets in the deployment wizard

You can use registry secrets for custom model runtimes.

using Core v1 registry secrets in the deployment wizard

For Core v2, the Remote Name of your bucket secret should match the prefix of your model URI.

using Core v2 secrets in the deployment wizard

For batch jobs, whether in Core v1 or Core v2, the Remote Name of your bucket secret should match the input and output data locations.

using bucket secrets in the batch job wizard

For reference data jobs, whether in Core v1 or Core v2, the Remote Name of your bucket secret should match the storage location of your reference data.

using bucket secrets in the reference data wizard

Note

In the Seldon Enterprise Platform UI, you will only be shown secrets with the appropriate format for that context. For example, when creating a Core v1 deployment you will only see env-var bucket secrets.

For backwards compatibility, any bucket secrets that are not labelled with their format are assumed to be in the env-var format.