ML System Design Interview Questions

Showing 1–30 of 138 results

• AdobeHard·Software Engineer
  Design a multimodal embedding service

  This question evaluates a candidate's competency in ML system design for building scalable, multi‑tenant, privacy‑sensitive multimodal embedding pipelines, covering ingestion and idempotency, modality-specific preprocessing, model selection and fusion, throughput engineering, storage and versioning, data hygiene, and operational concerns.

  ML System DesignTechnical Screen
• AdobeHard·Software Engineer
  Design file-embedding storage system

  This interview question evaluates ML product requirements, data/labeling, modeling, serving architecture, evaluation, monitoring, and trade-offs in a realistic interview setting. A strong answer for Design file-embedding storage system states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  ML System DesignTechnical Screen
• AmazonHard·Applied Scientist
  Design a RAG system end to end

  This question evaluates a candidate's ability to design a production-grade Retrieval‑Augmented Generation (RAG) system, testing competencies in scalable ML system architecture, embedding and vector retrieval strategies, prompt orchestration, freshness and latency engineering, security/access controls, and evaluation metrics.

  ML System DesignTechnical Screen
• AmazonEasy·Software Engineer
  Design a Reliable AI Chatbot Architecture

  Tests candidate's ability to design an enterprise-grade Retrieval-Augmented Generation (RAG) chatbot system featuring fine-grained authorization, hallucination guardrails, and low-latency response serving.

  ML System DesignTechnical ScreenRAGLLMInformation Retrieval
• AmazonHard·Software Engineer
  Design a fraud detection system

  This problem evaluates your ability to design a scalable, low-latency fraud detection system for an e-commerce platform, focusing on handling class imbalance, feature engineering, and the challenges of delayed feedback loops.

  ML System DesignTechnical ScreenFraud DetectionScalabilityFeature Store
• AmazonHard·Software Engineer
  Design an e-commerce recommendation system

  This question evaluates expertise in large-scale machine learning system design, covering competencies in recommendation algorithms, candidate generation and ranking, feature engineering and stores, low-latency serving, and operational reliability; it is categorized in the ML system design domain and requires both conceptual understanding and practical application. It is commonly asked to assess the ability to balance personalization and business metrics (CTR, CVR, revenue), address scalability and latency targets, and manage data quality, cold-start, exploration–exploitation, A/B testing, bias/fairness, and monitoring trade-offs in production.

  ML System DesignTechnical Screen
• AmazonHard·Software Engineer
  Design an email spam detection system

  This interview question evaluates ML product requirements, data/labeling, modeling, serving architecture, evaluation, monitoring, and trade-offs in a realistic interview setting. A strong answer for Design an email spam detection system states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  ML System DesignTechnical Screen
• AmazonHard·Applied Scientist
  Explain ML statistics and model design concepts

  Explain ML statistics and model design concepts evaluates ML product requirements, data/labeling, modeling, serving architecture, evaluation, monitoring, and trade-offs in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  ML System DesignTechnical Screen
• AmazonHard·Software Engineer
  Optimize attention for long sequences

  Optimize attention for long sequences evaluates ML product requirements, data/labeling, modeling, serving architecture, evaluation, monitoring, and trade-offs in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  ML System DesignTechnical Screen
• AmazonMedium·Machine Learning Engineer
  Predict Click Probability for Music Recommendations

  Design music recommendation CTR prediction with song embeddings, sparse-feature handling, model comparison, calibration, and online evaluation.

  ML System DesignTechnical Screen
• AAmbienceHard·Machine Learning Engineer
  Design an ICD-10 Prediction System for Clinical Records

  Design an end-to-end clinical NLP system to predict ICD-10 codes from long medical records, addressing extreme multi-label classification, long-tail code distributions, and human-in-the-loop coder feedback.

  ML System DesignTechnical Screenml-system-designnlphealthcare
• AAmericanMedium·Machine Learning Engineer
  Design an enterprise RAG agent system

  This question evaluates a candidate's competency in ML system design for enterprise retrieval-augmented generation and agent architectures, covering understanding of model roles, agent tool integrations, memory and state design, evaluation metrics, security and privacy defenses, concurrency and reliability, workflow durability, and document ingestion/indexing challenges. It is commonly asked in ML system design interviews to assess architectural thinking and trade-off analysis, testing both conceptual understanding and practical application skills for building scalable, secure, and cost-effective enterprise AI assistants.

  ML System DesignTechnical Screen
• AnthropicMedium·Software Engineer
  Design a Low-Latency GPU Inference Service

  This problem tests the ability to design a high-throughput, low-latency inference architecture for large language models, focusing on hardware utilization and bottleneck mitigation.

  ML System DesignTechnical ScreenMLOpsLLMDistributed Systems
• AnthropicMedium·Software Engineer
  Design an LLM-based binary classifier

  This question evaluates knowledge of LLM-based binary classification, probabilistic scoring and numerical stability (including log-prob handling and log-sum-exp), prompt construction, batching and sequence-level aggregation, calibration and evaluation metrics, plus analysis of failure modes and inference-time complexity.

  ML System DesignTechnical Screen
• AnthropicMedium·Software Engineer
  Review and Scale an Inference Service Design

  Tests the ability to architect a high-throughput ML inference service under strict latency SLAs, including dynamic batching, hardware capacity estimation, admission control, and multi-model routing.

  ML System DesignTechnical Screenml-system-designdistributed-systemsgpu-inference
• AnthropicHard·Software Engineer
  Route and Batch Inference with Eight GPUs

  Design an inference routing, dynamic batching, and scheduling system across a fixed pool of eight 80GB GPUs serving both large and small language models under strict latency and memory constraints.

  ML System DesignTechnical Screenml-systemsllm-inferencegpu-scheduling
• AnthropicHard·Machine Learning Engineer
  Schedule Large and Small Model Inference on an Eight-GPU Pool

  Design an inference scheduler for a shared 8-GPU node that co-locates single-GPU small models and all-reduce 8-GPU large models. The design must address gang scheduling, prefix cache reuse, deadline-aware priorities, and overload mitigation.

  ML System DesignTechnical Screenml-system-designgpu-schedulingdistributed-inference
• AppleMedium·Machine Learning Engineer
  Choose File Storage Tiers Using Access Predictions

  Design an end-to-end machine learning and decision system that predicts file access likelihood to automatically transition cloud-stored user files across storage tiers to minimize operational cost while meeting retrieval latency SLAs.

  ML System DesignTechnical Screenml-system-designprobability-calibrationcloud-storage
• AppleHard·Software Engineer
  Design a 200-Terabyte Media Migration and Training Input Pipeline

  This problem tests your ability to design a high-throughput data ingestion pipeline, focusing on network bottlenecks, storage strategies, and synchronization between on-premises assets and cloud-based training clusters.

  ML System DesignTechnical ScreenSystem DesignData EngineeringMLOps
• AppleMedium·Machine Learning Engineer
  Design a CPA system for ad bidding

  This question tests the ability to design an end-to-end Cost-Per-Acquisition (CPA) bidding system, specifically focusing on how to model conversion probabilities and handle the complexities of delayed feedback and bid pacing.

  ML System DesignTechnical ScreenSystem DesignML OpsAdTech
• AppleHard·Machine Learning Engineer
  Design a streaming embedding-based classifier

  This question evaluates a candidate's ability to design end-to-end streaming machine learning systems, including online text preprocessing, tokenization, embedding generation, continuous model training, and low-latency classification serving.

  ML System DesignTechnical Screen
• Applied IntuitionMedium·Machine Learning Engineer
  Debug a GPT training pipeline

  This question evaluates debugging and implementation skills for transformer-based training pipelines, specifically attention masking, training-loop correctness, positional encoding integration, and the ability to design unit tests that catch such bugs.

  ML System DesignTechnical Screen
• AuroraHard·Machine Learning Engineer
  Applying LLMs to Autonomous Driving Despite Hallucination and Real-Time Limits

  An open-ended ML system design discussion on how large language models and vision-language models could be used in autonomous driving, including whether one could actually drive the vehicle. It tests mapping off-board and on-vehicle uses, containing hallucination and latency in a safety-critical stack, and planning evaluation.

  ML System DesignTechnical Screen
• BasetenHard·Software Engineer
  Design an LLM Inference Serving System

  Design an end-to-end, high-performance LLM serving platform that manages request admission, continuous batching, KV cache memory, token streaming, and GPU fleet autoscaling under tight latency SLAs.

  ML System DesignTechnical Screenml-system-designllm-servingdistributed-systems
• BByteDanceMedium·Machine Learning Engineer
  Design Self-Dealing Detection for Marketplaces

  This question evaluates competency in machine learning system design for fraud detection, focusing on modeling buyer–seller–item–transaction relationships, feature engineering across graph, temporal, behavioral and account-linking signals, and production concerns; it is in the ML System Design domain and emphasizes practical application and system-level architecture over purely theoretical concepts. It is commonly asked to assess a candidate’s ability to reason about data collection and labeling, model and rule trade-offs, evaluation metrics, scalability and deployment trade-offs when detecting coordinated or self-dealing activity in marketplace environments.

  ML System DesignTechnical Screen
• BByteDanceMedium·Software Engineer
  Design a RAG-Based Agent System

  This question tests the ability to architect a scalable Retrieval-Augmented Generation (RAG) system that handles document indexing, retrieval, and multi-step reasoning while maintaining low latency and data freshness.

  ML System DesignTechnical ScreenRAGLLMSystem Design
• BByteDanceMedium·Senior SDE, Quality Platform & AI Test Automation
  Design an Agent-Orchestrated QA Pipeline from Requirements to Test Report

  Design an end-to-end multi-agent QA platform that autonomously transforms feature requirements into validated test cases, synthetic data, executable test scripts, and traceable execution reports with human-in-the-loop oversight.

  ML System DesignTechnical ScreenAI AgentsTest AutomationCI/CD
• BByteDanceMedium·Machine Learning Engineer
  Design an Enterprise Tool-Using Agent

  This question evaluates a candidate's ability to design production-grade LLM agents that integrate external tools, manage long-running and branching workflows, persist complex state, and ensure safety, observability, and reliable evaluation in enterprise settings.

  ML System DesignTechnical Screen
• C3 AIHard·Data Scientist
  Evaluate a Retrieval-Augmented Generation System

  Design an end-to-end evaluation framework for an enterprise RAG system that assesses retrieval quality, generative fidelity, claim-level attribution, security controls, and production telemetry.

  ML System DesignTechnical ScreenRAGLLM EvaluationSystem Design
• C3 AIHard·Data Scientist
  Forecast Food Stocking Needs Under Waste and Stockout Costs

  Design an end-to-end inventory forecasting and stocking optimization system for perishable food items under asymmetric costs of spoilage versus stockouts. The design covers handling censored sales data, probabilistic demand modeling, cost-optimal stocking policies, and validation strategy.

  ML System DesignTechnical Screenml-system-designtime-seriessupply-chain