How would you grow key product metrics?
MetaProduct Analyst · OnsiteLast updated
This question assesses your ability to systematically approach product growth by identifying bottlenecks, framing hypotheses, and designing experiments to improve key user engagement metrics.
The Scenario
You are a Product Analyst for a mobile photo-sharing application. The product team has noticed that while the total number of daily active users (DAU) is stable, the 'Stories' feature—where users share photos that disappear after 24 hours—has seen a 10% decline in daily content creation over the past month.
Your goal is to increase the number of daily Story creations. Please provide a structured approach to solving this problem by covering the following:
- Metric Definition & Investigation: Which metrics would you look at to diagnose the decline? How would you segment your user data to identify if this is a systemic issue or isolated to a specific cohort?
- Hypothesis Generation: Based on your diagnostic approach, propose two distinct hypotheses for why content creation is dropping.
- Experimentation: Choose one of your hypotheses. Describe how you would design an A/B test to validate it. What is your primary metric, and what guardrail metrics would you track to ensure the health of the overall ecosystem?
Constraints
- Assume you have access to standard event-logging data (e.g., clicks, impressions, timestamps, user ID, device type).
- Keep your solution focused on actionable product changes rather than broad marketing initiatives.
Example Framework
- Investigation: Compare creation rates between new vs. returning users.
- Hypothesis: 'If we simplify the camera upload UI, users will be less intimidated by the creation process.'
- Experiment: Run a split test on the upload interface; primary metric is 'Stories per DAU'.
Hints
Hint 1
Start by building a funnel: define every step from 'Opening App' to 'Posting Story' to see where the drop-off is occurring.
Hint 2
Think about the difference between 'capacity' (can the user create content?) and 'motivation' (does the user want to create content?).
Hint 3
When designing the A/B test, consider what could go wrong—for example, if the change increases creation but decreases the quality or the number of views per Story.
Follow-up questions
- How would you determine if the decline in Stories is being cannibalized by another feature, such as direct messaging or newsfeed posts?
- If your experiment shows a positive lift in content creation but a flat trend in overall user retention, how would you interpret that result?
- How would you communicate these findings to a non-technical stakeholder, such as a Product Manager, to get buy-in for a long-term feature change?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.