Define Success Metrics and Experiment Plan for Product Development
AirbnbData Scientist · Technical Screen
Last updated
Evaluates product-planning metrics and experiment design for a two-sided marketplace booking funnel. Strong answers define a primary booking metric, diagnostic and guardrail metrics, randomization unit, sample size, marketplace spillovers, and launch criteria before development.
Scenario
During a product-planning session you must define success criteria before development begins.
Question
Design the key metrics, guardrails, and experiment plan you would propose, including unit of randomization, sample-size calculation, and runtime monitoring.
Hints
Cover north-star vs. diagnostic metrics, power analysis, and data-quality checks.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.