Katkat
Back

Design an experiment assignment service

VisaSoftware Engineer · Technical Screen
HardSoftware EngineerTechnical ScreenAnalytics & Experimentation

Last updated

This question evaluates system-design and analytics competencies for online experimentation platforms in the Analytics & Experimentation domain, focusing on deterministic sticky assignment, configuration storage, eligibility targeting, traffic allocation, mutual exclusion, overrides, exposure logging, metric tagging, privacy/compliance, latency and availability targets, experiment lifecycle, and high-level architecture. It is commonly asked because it assesses both conceptual understanding and practical system-design application of distributed-system trade-offs, scalability, data integrity, operational reliability, and privacy constraints, and gauges the ability to define high-level APIs, architecture, and justify key trade-offs.

Design an online experimentation (A/B testing) assignment service. The API input is (userId, featureName) and it returns a variant identifier (e.g., control or treatment). Clarify and specify: deterministic sticky assignment and bucketing strategy, configuration storage for experiments and features, eligibility targeting and audience filters, traffic allocation and ramps, mutual exclusion and experiment layering, overrides (holdouts, QA, killswitch), exposure logging and idempotency, metric tagging and data pipeline, privacy and compliance (e.g., GDPR/CCPA), latency/availability targets and fallbacks, and the experiment lifecycle (create, start, pause, stop, analyze). Provide a high-level architecture and key components.