Katkat
Back

Determine Metrics to Evaluate Notification Impact on Users

MetaData Scientist · Onsite
MediumData ScientistOnsiteAnalytics & Experimentation

Last updated

Evaluates product analytics judgment for Facebook push notifications, including how to measure helpful versus harmful notifications, design an event-notification experiment, and balance short-term clicks against retention, fatigue, and opt-out guardrails. Strong answers build a metric hierarchy and use causal measurement.

Scenario

Facebook has multiple types of push notifications and is considering launching a new one that tells you a friend plans to attend an event.

Question

Which data and metrics would you use to determine whether an existing notification is helpful or harmful to users? What analyses or experiments would you run to decide whether to launch the new event-based notification?

Hints

Think funnel metrics, user experience trade-offs, A/B tests, downstream engagement and churn.