Model Outlier Detection

In a production environment, it is critical to monitor the data your machine learning model runs inference on, as changes in data can adversely affect the performance of ML models.

This demo will illustrate how to identify outliers in your inference data, using Alibi Detect’s VAE outlier detection method for tabular datasets in the following steps.

  • Launch an image classifier model trained on the CIFAR-10 dataset. The data instances contains 32x32x3 pixels images that are classified into 10 classes such as truck, frog, and cat etc.

  • Set up an VAE outlier detector for this particular model

  • Send a request to get an image classification

  • Send a perturbed request to identify an outlier instance

Launch a Seldon ML pipeline

Create an image classifier model deployment in an appropriate namespace

  1. Click Create new deployment on the deployments page to create a Seldon ML Pipeline.

  2. Enter the pipeline details in the deployment creation wizard and click Next:

    • Name: cifar10

    • Type: Seldon ML Pipeline

  3. The predictor details should have the Tensorflow runtime to use the correct server and the following Model URI:

  4. Click Next for the remaining steps, then click Launch. create_model

Add an Outlier detector

From the Deployment Overview page, select your pipeline to enter the pipeline dashboard. Inside the Pipeline Dashboard, add an outlier detector by clicking the Add button within the Outlier Detection widget.


Enter the following parameters in the configuration wizard which appears to configure the detector:

  • Detector Name: cifar10-outlier.

  • Storage URI: (For public google buckets, secret field is optional)

  • Reply URL: Leave as the default value, shown below, unless you are using a custom installation, please change this parameter according to your installation


Then, click Create Detector to complete the setup.

Make Predictions

Run a single prediction using an image of a frog in the Tensorflow payload format. Click the payload to download it from the following table, as well as a perturbed instance of the same image. Make a couple of these requests from the Predict screen on the UI.

Payload type


Tensorflow Payload

Expected Instance


Download Frog Payload

Outlier Instance


Download Perturbed Frog Payload


View Outliers From Request Logs

Go to the Requests screen to view request logs. Outlier scores will be available on the right side of each instance. You can also highlight outliers and filter them using the Highlight Outliers toggle on the top-right of the screen.


Real-Time Outlier Monitoring

You can view a timeline of outlier/inlier requests in the Outlier Detection tab of the Monitor screen.



If you experience issues with this demo, see the troubleshooting docs or Elasticsearch sections.