Diagnose metric drop in Ads Manager
TikTokData Scientist · OnsiteLast updated
This question evaluates a Data Scientist's competency in operational analytics, metric integrity, root-cause analysis, and experiment interpretation, covering concepts such as logging latency, sample ratio assessment, and product instrumentation awareness.
Your Ads Manager dashboard shows a 15% relative drop in ad-creation completion rate on 2025-06-10 versus its prior 7-day baseline. The metric is defined as completed_creations / initiated_creations per day at the advertiser level. Describe a step-by-step triage and deep-dive plan to: (1) verify the drop (data quality checks, AA tests, sample ratio mismatch, logging latency); (2) localize the issue (slice by geo, platform, entry point, advertiser tenure/spend tier, funnel step, experiment flags, recent UI/BE releases); (3) determine root cause (product change vs. traffic mix vs. seasonality vs. incidents) using specific analyses you would run; (4) quantify impact and propose immediate mitigations; and (5) define decision criteria for rollback vs. continue. Be specific about queries/cuts you would run and guardrail metrics you would check.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.