Katkat
Back

Design model deployment, monitoring, and low-latency inference

Capital OneMachine Learning Engineer · Onsite
MediumMachine Learning EngineerOnsiteML System Design

Last updated

This question evaluates competency in ML system design and production engineering—covering model deployment and versioning, safe rollouts and rollbacks, monitoring of service health, data quality/drift, model performance and business metrics, and latency optimization for low-latency inference.

You have trained a fraud detection model and need to productionize it.

Part A: Deployment

  • How would you deploy an ML model to production?
  • What artifacts do you version and how do you enable safe rollouts/rollbacks?

Part B: Monitoring

  • After deployment, how do you monitor the model?
  • What metrics do you track for:
  • service health,
  • data quality/drift,
  • model performance,
  • business impact?

Part C: Latency SLO

The model is deployed behind an online API, but it is missing a strict latency requirement: p99 latency < 50 ms.

  • How do you diagnose where time is spent?
  • What concrete changes would you consider across features, model, infrastructure, and serving to meet the SLO without unacceptable accuracy loss?