Diagnose and reverse an adoption-rate decline
GoogleData Scientist · OnsiteLast updated
This question evaluates a data scientist's competency in product analytics and experimentation, covering precise metric definition and validation, instrumentation and event quality checks, rapid triage of adoption regressions, hypothesis generation, experiment design, and causal inference.
Google Meet’s enterprise adoption rate (weekly active unique users ÷ total enabled users) has fallen from 38% to 31% over the last 4 calendar weeks. You are the on-call analyst. (a) Precisely define and validate “adoption rate” (event definitions, de-duplication, bot filtering, identity across devices, time zones, attribution windows). How would you backfill and reconcile with historical dashboards? (b) Lay out a 24-hour triage plan to distinguish instrumentation issues, seasonality, and true behavioral change; include concrete queries/checks, holdout benchmarks, and sanity ratios. (c) Propose at least three falsifiable hypotheses segmented by customer tier, geography, client platform (web/iOS/Android), and feature usage (recording/meeting size). What evidence would support or refute each? (d) Design experiments to reverse the decline (e.g., onboarding nudge, performance fix, reminder). Specify primary/secondary metrics, guardrails, MDE, power, sample size, duration, and ramp plan; explain how you’d choose variants under limited traffic. (e) Show how you’d separate causal impact from external shocks (holidays, competitor launches) using diff-in-diff, synthetic control, or interrupted time series. What assumptions must hold, and how will you test them? (f) Identify risks like metric gaming, delayed conversions, and Simpson’s paradox. What monitoring and drill-downs prevent false wins and ensure reproducibility?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.