Design Experiment to Measure Shopping Feature Impact
MetaData Scientist · Onsite
Last updated
Meta data scientist experimentation prompt on measuring Instagram Shopping impact with A/B tests, geo or cluster holdouts, engagement and revenue metrics, guardrails, selection bias, network effects, and rollout decisions.
Scenario
Instagram is launching an in-app Shopping feature. Leadership wants to understand the business impact after release.
Question
How would you design an experiment to measure the impact of the Shopping feature on user engagement and revenue? Which metrics and success criteria would you monitor and why? How would you deal with potential selection bias or network effects?
Hints
Think A/B vs. hold-out, north-star metrics, guard-rails, diff-in-diff.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.