Katkat
Back

Design an ML Platform Portal

NetflixMachine Learning Engineer · Onsite
EasyMachine Learning EngineerOnsiteML System Design

Last updated

This question assesses the ability to design an internal ML Platform Portal that streamlines the model lifecycle, focusing on experiment tracking, model registry, and workflow automation.

Problem Statement

Design an internal ML Platform Portal for a large-scale streaming company. Data scientists and ML engineers use this portal to manage the entire lifecycle of their models, from initial experimentation to production deployment and monitoring.

Functional Requirements

  1. Experiment Tracking: Ability to log hyperparameters, metrics, and artifact references for model training runs.
  2. Model Registry: A central hub to register, version, and document models, including their status (e.g., Development, Staging, Production).
  3. Deployment Workflow: A mechanism to promote models from the registry to production environments via CI/CD pipelines.
  4. Performance Dashboard: Real-time visualization of model performance metrics (e.g., latency, throughput, prediction drift).

Non-Functional Requirements

  • High availability and consistency for the registry metadata.
  • Scalability to support thousands of active models and concurrent experimentation.
  • Strict access control and auditing (who promoted which model to production?).

The Task

Provide a high-level system design including:

  • A brief overview of the high-level architecture.
  • Key database entities and their relationships.
  • The API design for interacting with the portal (e.g., registering a model, querying experiments).
  • Strategies for handling large model artifacts and maintaining version integrity.

Hints

Hint 1

Consider decoupling the storage of model metadata (SQL database) from the storage of large model binary artifacts (Object Storage).

Hint 2

Use a versioning strategy that links model artifacts to the specific training data snapshot and the code commit hash to ensure full reproducibility.

Hint 3

For the monitoring component, consider a sidecar or streaming pattern to aggregate logs from production model endpoints before storing them in a time-series database.

Follow-up questions

  • How would you handle a rollback scenario if a newly deployed model shows unexpected degradation in production?
  • How would you design the system to support multi-tenancy if multiple teams (e.g., Content Recommendations vs. Search) use the same platform?
  • What changes would be necessary to support A/B testing or multi-armed bandit deployments within this platform?