Optimize Model Serving Under 200ms

XometryMachine Learning Engineer · Technical Screen

MediumMachine Learning Engineer Technical Screen ML System Design

Last updated October 28, 2026

This question evaluates competency in deploying and optimizing machine learning models for low-latency online inference, covering model serving, latency profiling, hardware considerations, and managing accuracy–latency trade-offs within a 200ms SLO in the ML System Design domain.

A data science team gives you a trained model and asks you to deploy it as an online inference service. The requirement is that a single prediction must complete within 200 milliseconds. Describe how you would clarify the requirement, measure the baseline, optimize the model and serving stack, choose hardware, validate accuracy-latency tradeoffs, and monitor the system after launch.

Explain an End-to-End ML ProjectMedium
Xometry · Machine Learning · Technical Screen
Add an LLM-backed recommendation API to an existing service with RAG and promptingMedium
Intuit · ML System Design · Onsite
Applying LLMs to Autonomous Driving Despite Hallucination and Real-Time LimitsHard
Aurora · ML System Design · Technical Screen
Architect an asynchronous RL post-training systemHard
Meta · ML System Design · Onsite
Build a Candidate Search SystemMedium
Mercor · ML System Design · Onsite
Build a Mistral-powered RAG agentHard
Meta · ML System Design · Onsite