Katkat

Diagnose metric drop in Ads Manager

TikTokData Scientist · Onsite

Last updated

This question evaluates a Data Scientist's competency in operational analytics, metric integrity, root-cause analysis, and experiment interpretation, covering concepts such as logging latency, sample ratio assessment, and product instrumentation awareness.

Your Ads Manager dashboard shows a 15% relative drop in ad-creation completion rate on 2025-06-10 versus its prior 7-day baseline. The metric is defined as completed_creations / initiated_creations per day at the advertiser level. Describe a step-by-step triage and deep-dive plan to: (1) verify the drop (data quality checks, AA tests, sample ratio mismatch, logging latency); (2) localize the issue (slice by geo, platform, entry point, advertiser tenure/spend tier, funnel step, experiment flags, recent UI/BE releases); (3) determine root cause (product change vs. traffic mix vs. seasonality vs. incidents) using specific analyses you would run; (4) quantify impact and propose immediate mitigations; and (5) define decision criteria for rollback vs. continue. Be specific about queries/cuts you would run and guardrail metrics you would check.