Deploy multi-armed bandits safely
MetaData Scientist · OnsiteLast updated
This question evaluates proficiency in online experimentation and sequential decision-making, covering Bayesian bandits (Thompson Sampling), safety guardrails for churn, delayed-conversion handling, traffic allocation and stopping/rollback policy design.
You have 3 variants, a churn guardrail, and delayed conversions. a) Design a Thompson Sampling bandit (specify likelihoods), and explain how you’ll handle delays (e.g., optimistic vs. debiased estimators) and non‑stationarity (e.g., discounting). b) Set traffic floors and fairness constraints. c) Define stopping and rollback policies when a guardrail is breached. d) Compare expected regret and business impact versus a fixed‑horizon A/B under seasonality.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.