Katkat

Deploy multi-armed bandits safely

MetaData Scientist · Onsite

Last updated

This question evaluates proficiency in online experimentation and sequential decision-making, covering Bayesian bandits (Thompson Sampling), safety guardrails for churn, delayed-conversion handling, traffic allocation and stopping/rollback policy design.

You have 3 variants, a churn guardrail, and delayed conversions. a) Design a Thompson Sampling bandit (specify likelihoods), and explain how you’ll handle delays (e.g., optimistic vs. debiased estimators) and non‑stationarity (e.g., discounting). b) Set traffic floors and fairness constraints. c) Define stopping and rollback policies when a guardrail is breached. d) Compare expected regret and business impact versus a fixed‑horizon A/B under seasonality.