Katkat
Back

Investigate Metric Drops and Coupon Retention

LyftData Scientist · Technical Screen
MediumData ScientistTechnical ScreenAnalytics & Experimentation

Last updated

This question tests your ability to diagnose a sudden drop in a core business metric, distinguish between technical tracking issues and product-driven behavior, and design a strategy to evaluate coupon effectiveness.

Scenario

You are a Data Scientist at Lyft. Yesterday, the 'Daily Rides Requested' metric dropped by 12% across the platform. Preliminary investigation shows that the drop is concentrated in the San Francisco market during peak evening hours.

At the same time, the marketing team launched a new '20% off your next ride' coupon campaign targeting lapsed users. Product managers are concerned that the coupon is causing cannibalization or that the drop is a symptom of a technical outage.

Your Tasks

  1. How would you investigate whether this drop is a data/logging issue or a genuine change in user behavior? What specific segments or logs would you examine first?
  2. Assuming the drop is real and not a logging error, how would you determine if the coupon campaign is causing the decline in ride requests from non-lapsed users (cannibalization)?
  3. If you determine the coupon campaign is indeed cannibalizing organic traffic, how would you restructure the experiment to measure the true incremental lift of the coupons?

Hints

Hint 1

Start by looking for technical signals such as crash rates, latency, or API error rates before moving to user behavioral patterns.

Hint 2

Consider the difference between 'intent' and 'conversion'; look at the funnel from app open to ride request to see where the drop-off occurs.

Hint 3

To measure incrementality, think about how a randomized holdout group (control group) can help you isolate the effect of the coupon versus general market trends.

Follow-up questions

  • What counter-metrics would you track to ensure that increasing coupon distribution doesn't negatively impact long-term unit economics?
  • How would you account for network effects or marketplace imbalance if the coupon drives a surge in demand that outstrips driver supply?
  • If the coupon campaign is deemed successful for retention but detrimental to short-term margins, what framework would you use to balance these competing priorities?