Katkat
Back

Design and Evaluate a Home Carousel

PinterestData Scientist · Technical Screen
MediumData ScientistTechnical ScreenAnalytics & Experimentation

Last updated

This question evaluates a candidate's expertise in product analytics, experimentation design, metric definition, and instrumentation for measuring the impact of a new UI feature.

Pinterest is considering adding a horizontally scrollable carousel at the top of the Home feed, similar to Instagram Stories. The carousel may surface recently saved Pins, recommended Pins, or topic collections.

As a Data Scientist, explain how you would evaluate this feature from product, analytics, and experimentation perspectives.

Your answer should address:

  1. What product goal(s) you would first clarify, and how those goals would change your evaluation framework.
  2. What primary, secondary, and guardrail metrics you would define. Be explicit about trade-offs among metrics such as module CTR, overall Home CTR, saves, downstream engagement, session depth, retention, and ad or monetization impact.
  3. What events, dimensions, and logging you would instrument to build the analytical system for this feature.
  4. How you would design an A/B test or staged rollout, including the randomization unit, exposure definition, novelty effects, power / minimum detectable effect, and possible sources of bias or confounding.
  5. Suppose the feature launches and overall Home-page CTR decreases. What are the most plausible explanations, and how would you distinguish among a logging bug, a denominator effect, cannibalization of the existing feed, segment-specific effects, or a true product regression?