Deployment models for Seldon Deploy
When it comes to deploying Seldon Deploy for a production workload, there are different deployment models that can apply depending on your use case. For example, you can consider the following questions:
Do you want to support multiple environments (e.g.
staging, etc.)? Will these be split across multiple clusters? Or across different namespaces?
Are you planning on supporting multiple tenants in your infrastructure? How will you manage permissions for each one of them?
This section will showcase some of the most common deployment models to set up Seldon Deploy in your cluster.
When you are managing machine learning models, a common use case is to test
them first on some kind of test environment, isolated from production
These test environments are usually known as
In some cases, these environments will form a deployment (or continuous delivery) pipeline, where each environment is considered more stable than the previous one:
Dev > ... > Staging > Production
Seldon Deploy allows you to have multiple environments by separating them as namespaces or clusters.
Regardless of the approach you choose, this doesn’t restrict your options when it comes to syncing your models with a GitOps repository. In fact, you can still have:
Single GitOps repository, with one folder per environment.
Multiple GitOps repositories, with one repository per environment.
The official Kubernetes docs define namespaces as virtual
Therefore, it makes sense to treat each separate environment as a separate
Following the example above, that would mean creating a
By default, Seldon Deploy will treat each namespace as a separate environment. Therefore, this is the suggested approach.
Out of the box, this gives you easy control over:
Where is each model deployed (on which environments).
Visibility over all your models across every environment from a single instance of Seldon Deploy.
When you are dealing with multiple environments, it makes sense to keep them completely isolated of each other. That way, any cluster-wide effects will only affect one of them.
Using separate Kubernetes clusters for each of your environments allows you to achieve a higher-level of separation than namespaces. However, this separation also comes with a few caveats:
All your infrastructure resources will need to be duplicated and managed across every environment (e.g. auth, ingress, logging, metrics, …).
Operations spanning multiple environments become more complicated (e.g. promotion of models between different environments).
This approach is usually recommended to test new versions of cluster-wide components at the infrastructure level. For example, a new version of Seldon Deploy or a new ingress system. However, due to the caveats listed above, we don’t suggest this approach to segregate different versions of your models.
Having said so, you can still leverage the GitOps paradigm to communicate between different clusters.