Katkat

Investigate visit–report correlation causality

TikTokData Scientist · Technical Screen

Last updated

This question evaluates a candidate's competency in causal inference, experimental design, diagnostic analytics, and statistical modeling with ad-level observational data.

You observe a positive correlation between ad page visits and the probability an ad is reported as bad. Investigate whether more page visits cause more reports, or whether exposure/selection confounds the relationship.

Provide: (1) at least three concrete hypotheses (e.g., pure exposure effect, targeting bias to sensitive cohorts, low-quality ads drive both attention and reporting, instrumentation effects), and the falsifiable prediction each makes; (2) a diagnostic plan including exact metrics and cuts: per-visit report rate vs. visit volume buckets; within-ad fixed-effects trends; cohorting by traffic source; funnel-based conditional probabilities; (3) a statistical model to estimate effect size controlling for exposure: e.g., ad-day logistic or Poisson regression with an offset log(visits), ad fixed effects, hour-of-day and geo controls; report the coefficient interpretation and how you would check overdispersion and multicollinearity; (4) at least one quasi-experimental approach (e.g., holdout geos, randomized delivery caps, or an IV using exogenous traffic shocks like site outages) and how you’d run a parallel A/A to validate; (5) criteria for action: define thresholds for an operational alert (e.g., ≥20% lift in per-visit report rate with p<0.01 after multiple-testing control) and what product/policy levers you’d recommend under each outcome.