practice a little every day, paws crossed ♡
ML System Design Interview Questions
Practice 9403 coding, system design and machine learning questions from 546 companies, sorted by role, topic and interview round. Every question has step-by-step hints, a code runner that supports 8 languages and an AI mentor; members also get the full solution walkthrough.
- 9403
- Questions
- 546
- Companies
- 11
- Topics
- 8
- Languages
Showing 31–60 of 332 results
- AmericanMediumMachine Learning EngineerDesign an enterprise RAG agent system
This question evaluates a candidate's competency in ML system design for enterprise retrieval-augmented generation and agent architectures, covering understanding of model roles, agent tool integrations, memory and state design, evaluation metrics, security and privacy defenses, concurrency and reliability, workflow durability, and document ingestion/indexing challenges. It is commonly asked in ML system design interviews to assess architectural thinking and trade-off analysis, testing both conceptual understanding and practical application skills for building scalable, secure, and cost-effective enterprise AI assistants.
ML System DesignTechnical Screen - AnthropicMediumSoftware EngineerDesign GPU inference request batching
This question evaluates a candidate's competency in ML System Design, focusing on GPU-based inference batching, scheduling, autoscaling, and the operational concerns of routing, compatibility, and fault-tolerance.
ML System DesignOnsite - AnthropicMediumSoftware EngineerDesign Model Weight Distribution
This question evaluates a candidate's competency in large-scale ML system design and distributed systems, focusing on weight file distribution, versioning and rollout strategies, consistency models, integrity verification, access control, rollback mechanisms, and operational scalability.
ML System DesignOnsite - AnthropicMediumSoftware EngineerDesign Safe Distribution and Activation of Model Weights
This problem tests the ability to architect a robust, secure, and atomic deployment system for large binary artifacts (model weights) across a distributed inference cluster.
ML System DesignOnsiteDistributed SystemsMLOpsSystem Design - AnthropicHardSoftware EngineerDesign a GPU inference API
This Anthropic ML system design onsite asks you to design a multi-tenant, GPU-backed inference API that serves many models and versions -- including large autoregressive LLMs -- under explicit latency SLOs. It spans API and schema design, streaming and async jobs, continuous batching with prefill/decode separation, KV-cache and GPU memory management, multi-model hosting on shared GPUs, heterogeneous autoscaling and placement, multi-region routing, canary rollout, and reliability and cost controls. The signature probe is the diagnostic scenario where CPU utilization is low but the GPUs are saturated.
ML System DesignOnsite - AnthropicMediumSoftware EngineerDesign a Low-Latency GPU Inference Service
This problem tests the ability to design a high-throughput, low-latency inference architecture for large language models, focusing on hardware utilization and bottleneck mitigation.
ML System DesignTechnical ScreenMLOpsLLMDistributed Systems - AnthropicHardSoftware EngineerDesign a batch inference API
This question evaluates a candidate's ability to design asynchronous batch inference APIs and systems, including API schema and job lifecycle design, idempotency semantics, queueing and worker scaling, batching and accelerator utilization, rate limiting, observability, and error handling.
ML System DesignOnsite - AnthropicHardSoftware EngineerDesign a high-concurrency LLM inference service
This problem evaluates your ability to design a scalable LLM inference serving platform, focusing on throughput optimization, memory constraints, and scheduling strategies for long-running inference requests.
ML System DesignOnsiteLLMInferenceGPU - AnthropicHardSoftware EngineerDesign a low-latency ML inference API
This question evaluates competency in ML system design for real-time, low-latency inference APIs, including multitenancy, SLO/SLI definition, feature retrieval, model serving and rollout strategies, observability, cost control, and security/compliance within the ML System Design category.
ML System DesignOnsite - AnthropicMediumMachine Learning EngineerDesign a model downloader
This question assesses the architectural design of a distributed, secure, and resilient system capable of deploying large-scale machine learning models to heterogeneous edge devices.
ML System DesignOnsiteSystem DesignMLOpsDistributed Systems - AnthropicHardSoftware EngineerDesign a prompt processing backend
Design a prompt processing backend evaluates ML product requirements, data/labeling, modeling, serving architecture, evaluation, monitoring, and trade-offs in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
ML System DesignOnsite - AnthropicMediumSoftware EngineerDesign an LLM-based binary classifier
This question evaluates knowledge of LLM-based binary classification, probabilistic scoring and numerical stability (including log-prob handling and log-sum-exp), prompt construction, batching and sequence-level aggregation, calibration and evaluation metrics, plus analysis of failure modes and inference-time complexity.
ML System DesignTechnical Screen - AnthropicHardMachine Learning EngineerDesign an inference routing and scheduling layer
This question evaluates system-design and production machine learning engineering skills, specifically distributed systems, routing and scheduling, multi-tenant isolation, caching, dynamic batching, and capacity planning for heterogeneous GPU/CPU inference backends.
ML System DesignOnsite - AnthropicHardSoftware EngineerEstimate FFN Compute, Memory, and Sharding Communication
Estimate standard FFN FLOPs, parameter and activation bytes, and sharding communication, then compare compute, HBM, and network limits with explicit layout assumptions.
ML System DesignOnsite - AnthropicHardSoftware EngineerEstimate VRAM and compare model parallelism
This question tests the ability to calculate memory overhead for transformer model parameters and activation buffers, while evaluating the architectural trade-offs between tensor and pipeline parallelism for large-scale training.
ML System DesignOnsiteDistributed SystemsGPU ArchitectureTransformer Training - AnthropicHardMachine Learning EngineerExtend an LLM Inference Server with a Prefix KV Cache
Anthropic Fellows OA practice: add exact-match, prefix and shared-prefix KV caching with an LRU token budget to an existing LLM inference server, keeping greedy outputs identical.
ML System DesignOnline AssessmentAnthropic FellowsAI Safety Fellowshipllm-inference - AnthropicMediumSoftware EngineerPerformance-Model a Sharded Matrix Multiply: FLOPs, Memory, Network and Roofline
Model the performance of a matrix multiplication by hand: count floating-point operations, memory bytes and network bytes, apply the roofline model to decide what bounds it, and compare sharding along each dimension across several accelerators. It tests arithmetic intensity, collective communication costs and quantitative reasoning.
ML System DesignOnsite - AnthropicHardSoftware EngineerReview an inference API design for scale
This question tests a candidate's ability to critically evaluate ML system designs for production-scale inference APIs, covering multi-tenancy, GPU resource constraints, and streaming token delivery. It assesses architectural reasoning across reliability, latency trade-offs, and capacity planning in the ML Systems domain.
ML System DesignOnsite - AnthropicMediumSoftware EngineerReview and Scale an Inference Service Design
Tests the ability to architect a high-throughput ML inference service under strict latency SLAs, including dynamic batching, hardware capacity estimation, admission control, and multi-model routing.
ML System DesignTechnical Screenml-system-designdistributed-systemsgpu-inference - AnthropicHardSoftware EngineerRoute and Batch Inference with Eight GPUs
Design an inference routing, dynamic batching, and scheduling system across a fixed pool of eight 80GB GPUs serving both large and small language models under strict latency and memory constraints.
ML System DesignTechnical Screenml-systemsllm-inferencegpu-scheduling - AnthropicHardMachine Learning EngineerSchedule Large and Small Model Inference on an Eight-GPU Pool
Design an inference scheduler for a shared 8-GPU node that co-locates single-GPU small models and all-reduce 8-GPU large models. The design must address gang scheduling, prefix cache reuse, deadline-aware priorities, and overload mitigation.
ML System DesignTechnical Screenml-system-designgpu-schedulingdistributed-inference - AppleMediumMachine Learning EngineerChoose File Storage Tiers Using Access Predictions
Design an end-to-end machine learning and decision system that predicts file access likelihood to automatically transition cloud-stored user files across storage tiers to minimize operational cost while meeting retrieval latency SLAs.
ML System DesignTechnical Screenml-system-designprobability-calibrationcloud-storage - AppleMediumMachine Learning EngineerDesign App Store search
This question evaluates a candidate's ability to design scalable, low-latency app marketplace search systems, covering competencies in query understanding, metadata ingestion and indexing, candidate retrieval and ranking, personalization, multilingual and typo-tolerant handling, cold-start and spam prevention, as well as online experimentation and monitoring. Commonly asked in the ML system design/search engineering domain to probe architectural trade-offs, measurement of product requirements and success metrics, and operationalization, it spans both high-level conceptual reasoning and practical application details.
ML System DesignOnsite - AppleHardSoftware EngineerDesign a 200-Terabyte Media Migration and Training Input Pipeline
This problem tests your ability to design a high-throughput data ingestion pipeline, focusing on network bottlenecks, storage strategies, and synchronization between on-premises assets and cloud-based training clusters.
ML System DesignTechnical ScreenSystem DesignData EngineeringMLOps - AppleMediumMachine Learning EngineerDesign a CPA system for ad bidding
This question tests the ability to design an end-to-end Cost-Per-Acquisition (CPA) bidding system, specifically focusing on how to model conversion probabilities and handle the complexities of delayed feedback and bid pacing.
ML System DesignTechnical ScreenSystem DesignML OpsAdTech - AppleMediumMachine Learning EngineerDesign a grounded voice assistant
This problem evaluates your ability to design a retrieval-augmented generation (RAG) system for a voice assistant, focusing on grounding model outputs in trusted sources to prevent hallucinations.
ML System DesignOnsiteRAGLLMSystem Design - AppleHardMachine Learning EngineerDesign a streaming embedding-based classifier
This question evaluates a candidate's ability to design end-to-end streaming machine learning systems, including online text preprocessing, tokenization, embedding generation, continuous model training, and low-latency classification serving.
ML System DesignTechnical Screen - AppleMediumMachine Learning EngineerOptimize image filters on device
This question evaluates competency in on-device machine learning and performance engineering, covering algorithm choice, numerical precision trade-offs, hardware utilization (CPU/GPU/accelerator), memory layout, tiling/streaming, and trade-offs among latency, memory overhead, and battery impact.
ML System DesignOnsite - Applied IntuitionMediumMachine Learning EngineerDebug a GPT training pipeline
This question evaluates debugging and implementation skills for transformer-based training pipelines, specifically attention masking, training-loop correctness, positional encoding integration, and the ability to design unit tests that catch such bugs.
ML System DesignTechnical Screen - AtlassianMediumMachine Learning EngineerDesign Jira bug-to-team classification system
This question tests the ability to design an end-to-end ML pipeline for classifying incoming Jira tickets to appropriate engineering teams based on textual and historical metadata.
ML System DesignOnsiteNLPClassificationScalability
Anthropic
Apple
Applied Intuition
Atlassian