Real time observability for ML models
Seldon Core and KFServing both provide out of the box grafana dashboards for monitoring. These include metrics on requests, feedback on predictions and request latency. The dashboards can be used to compare different versions of a model between which traffic is being split, such as a main model running alongside a canary version. These dashboards are integrated into the Seldon Deploy user interface and are accessed behind the same authentication as the rest of Seldon Deploy. By default this is achieved by exposing a single, secured istio ingress gateway and creating VirtualServices for each component (other configurations would also be possible).
Detailed demos showing monitoring are available in the Demos section of this documentation.