Katkat
Back

Diagnose and fix low conversion rigorously

InstacartData Scientist · HR Screen
MediumData ScientistHR ScreenAnalytics & Experimentation

Last updated

This question evaluates a Data Scientist's competency in analytics and experimentation, covering instrumentation validation, funnel and segmentation analysis, causal inference methods (holdouts/DiD), system reliability diagnostics, statistical power and sample-size estimation, and the ability to synthesize findings into executive-level impact statements. It is commonly asked to determine whether a candidate can distinguish measurement or product regressions from marketing-mix shifts and to demonstrate both conceptual understanding of causal inference and practical application of data querying, metric definition, and experiment design in the Analytics & Experimentation domain.

Checkout conversion dropped from 42% to 35% week‑over‑week after a new promo banner shipped; traffic volume is flat, marketing mix shifted 10% toward paid social, and payment failures rose slightly. Outline a step‑by‑step plan to isolate root cause: validate instrumentation, define conversion precisely, build a funnel, segment by device/geo/new vs. returning, inspect latency and errors, and use holdouts or diff‑in‑diff to separate promo impact from channel shifts. Specify the minimal queries/metrics you’d pull, the power you’d need for an A/B rollback or fix‑forward test, and the executive‑level insight you’d present, including expected revenue impact and next actions.