Visibility of projects

Seldon Deploy allows models to be grouped into projects. A project can have multiple models associated with it. It is possible to make authorization decisions based on the project a model is associated with. This guide details how this can be done.

Setup

To start using project based authorization, it must be enabled together with OPA auth as per the installation instructions. To see how the OPA policies for project based resources look like go to the policy section.

Protected resources and policy setup

Models

Right now models are the only resources that are explicitly associated with a project. By default, all models that do not specify a project are assigned to the default project.

It is strongly advised to set up a base policy so that these models are visible to everyone. A section like this in the policy file will achieve this:

{
  "role_grants": {},
  "user_grants": {
    "*": [
      {
        "action": "read",
        "resource": "project/default"
      },
      {
        "action": "write",
        "resource": "project/default"
      }
    ]
  }
}

If a user has a read permission on a project it means they can read all models associated with that project. However, they won’t be able to perform actions like updating the model, creating a new model in that project, or deleting a model from that project, unless they have a write permission on the project as well. These authorization checks are performed both if interacting directly with the Seldon API, or via the Model Catalog in the UI.

Deployments

Since Seldon Deployments and Inference Services reference models themselves, these resources are also protected by project based policies. It is possible for a deployment to consist of multiple models that can belong to different projects. A user must have a read permission on all models referenced in a Seldon Deployment or Inference Service in order to see it. Namespace level authorization is still applied.

To see how Seldon Deployment and Inference Services reference which project a model they reference belongs to see the next section.

Associating resources in Kubernetes with a project

Neither Seldon Deployments nor Inference Services support the concept of projects natively. Therefore, we are using annotations on the CRDs to link a model in that resource with its corresponding project.

When deploying through the UI

When modifying Seldon Deployment or an Inference Service through the Deploy UI it will automatically be annotated with the projects that are referenced in the Model Project field. If none is specified, the default project will be assumed.

deployment_creation_project_field

When deploying manually (gitops, kubectl, etc.)

If you are manually constructing the yaml/json that you want to be deployed to the Seldon Deploy cluster, you must annotate the deployments for Deploy to know what project a model refers to. If no annotations are present, the default project will be assumed. Here is how to annotate a deployment:

Seldon Deployment

Seldon Deployments’ models are annotated on the predictor level. Each predictor in the spec.predictors field has an annotations field that contains annotations like "project.seldon.io/$name_of_component_in_graph": "$project_name"". This allows us to specify the models of all components in a predictor’s deployment graph since the projects can be different.

Here is a simplified example of the spec of a Seldon Deployment with only one single-model predictor:

{
  "spec": {
    "name": "mock-example",
    "predictors": [
      {
        "name": "default",
        "graph": {
          "name": "mock-example-container",
          "type": "MODEL",
          "logger": {
            "mode": "all"
          }
        },
        "componentSpecs": [
          {
            "spec": {
              "containers": [
                {
                  "name": "mock-example-container",
                  "image": "seldonio/mock_classifier:1.5.0",
                  "resources": {
                    "limits": {
                      "cpu": "1",
                      "memory": "1Gi",
                      "nvidia.com/gpu": "0"
                    },
                    "requests": {
                      "cpu": "100m",
                      "memory": "1Gi",
                      "nvidia.com/gpu": "0"
                    }
                  }
                }
              ]
            }
          }
        ],
        "replicas": 1,
        "annotations": {
          "project.seldon.io/mock-example-container": "iris"
        },
        "engineResources": {},
        "svcOrchSpec": {},
        "traffic": 100
      }
    ],
    "annotations": {
      "seldon.io/engine-seldon-log-messages-externally": "true"
    },
    "protocol": "seldon"
  }
}

The only part different from the default Seldon Core spec is the annotations field specifying the project of the model in mock-example-container node.

Inference Service

Inference Services do not have predictor level annotations, so we use the annotations on the Inference Service level instead. Again, we follow the same pattern as with Seldon Deployments for the annotations. They look like "project.seldon.io/$name_of_component": "$project_name"". However, the component names are predefined here. The available annotation options are:

  • "project.seldon.io/default": "$project_name"" - the default predictor

  • "project.seldon.io/default-transformer": "$project_name"" - the transformer of the default predictor

  • "project.seldon.io/canary": "$project_name"" - the canary predictor

  • "project.seldon.io/canary-transformer": "$project_name"" - the transformer of the canary predictor

A complete example of an inference service with canary and transformers for both the canary and the default predictors:

{
	"kind": "InferenceService",
	"apiVersion": "serving.kubeflow.org/v1alpha2",
	"metadata": {
		"name": "mock-example",
		"namespace": "seldon",
		"resourceVersion": "7249641",
		"labels": {
			"fluentd": "true"
		},
		"annotations": {
			"project.seldon.io/default": "iris",
			"project.seldon.io/default-transformer": "income",
			"project.seldon.io/canary": "default",
			"project.seldon.io/canary-transformer": "income"
		}
	},
	"spec": {
		"default": {
			"predictor": {
				"custom": {
					"container": {
						"name": "",
						"image": "seldonio/mock_classifier:1.5.0",
						"resources": {
							"limits": {
								"cpu": "1",
								"memory": "2Gi"
							},
							"requests": {
								"cpu": "1",
								"memory": "2Gi"
							}
						}
					}
				},
				"logger": {
					"mode": "all"
				}
			},
			"transformer": {
				"custom": {
					"container": {
						"name": "transformer-container",
						"image": "seldonio/mock_classifier:1.5.0",
						"resources": {
							"limits": {
								"cpu": "1",
								"memory": "2Gi"
							},
							"requests": {
								"cpu": "1",
								"memory": "2Gi"
							}
						}
					}
				}
			}
		},
		"canary": {
			"predictor": {
				"custom": {
					"container": {
						"image": "seldonio/mock_classifier:1.5.0",
						"env": []
					},
					"resources": {
						"requests": {
							"cpu": "0.1",
							"memory": "1Gi",
							"nvidia.com/gpu": "0"
						},
						"limits": {
							"cpu": "1",
							"memory": "1Gi",
							"nvidia.com/gpu": "0"
						}
					}
				},
				"logger": {
					"mode": "all"
				}
			},
			"transformer": {
				"custom": {
					"container": {
						"name": "transformer-container",
						"image": "seldonio/mock_classifier:1.5.0"
					}
				}
			}
		},
		"canaryTrafficPercent": 10
	},
	"status": {}
}