Test 15s to 60s video length change
TikTokData Scientist · OnsiteLast updated
This question evaluates a data scientist's competency in experimental design, causal inference, metric definition and instrumentation, power/MDE calculations, interference and contamination handling on multi‑sided platforms, and rollout planning.
Product proposes changing default video length from 15s to 60s. Design the experiment. Specify: (1) unit of randomization and exposure rules to avoid contamination (creators, viewers, or geo; how to handle multi-sided interactions); (2) primary outcomes and guardrails (e.g., completion rate, total watch time per user, creator retention, session length, ad revenue per mille, latency/crash rate); (3) traffic allocation, ramp schedule, and expected MDE with power assumptions; (4) how you will detect novelty and learning effects and set experiment duration; (5) checks for SUTVA/interference and cluster variance; and (6) a rollout plan if results pass. Be concrete about metric definitions and instrumentation.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.