Simulate and Compare Multi-Armed Bandit Strategies
WayfairApplied Scientist · Onsite
Last updated
Compare epsilon-greedy, UCB1, and Thompson sampling in a reproducible Bernoulli bandit simulation with reward, pseudo-regret, and repeated-run uncertainty.
Build a small simulation that compares common multi-armed bandit strategies. Explain each strategy, record reward and regret, and use the experiment to discuss exploration versus exploitation.
Constraints & Assumptions
The source reports a bandit simulation exercise with AI and search tools allowed, but gives no arm distributions, horizon, or required output. For this practice discussion, use stationary independent Bernoulli arms with known probabilities available only to the simulator. Compare epsilon-greedy, UCB1, and Thompson sampling over repeated random seeds. This is an experimental simulation exercise rather than a fixed-output console task.
Clarifying Questions
Are rewards binary or continuous? Is the environment stationary? What is the horizon? Does the policy observe only the chosen arm's reward? How should reproducibility and uncertainty be reported?
Follow-up questions
- How do the results change with a small reward gap, a short horizon, nonstationary rewards, or a poorly chosen exploration rate? Why is sharing a seed alone insufficient to guarantee identical random consumption across policies?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.