Katkat

Test 15s to 60s video length change

TikTokData Scientist · Onsite

Last updated

This question evaluates a data scientist's competency in experimental design, causal inference, metric definition and instrumentation, power/MDE calculations, interference and contamination handling on multi‑sided platforms, and rollout planning.

Product proposes changing default video length from 15s to 60s. Design the experiment. Specify: (1) unit of randomization and exposure rules to avoid contamination (creators, viewers, or geo; how to handle multi-sided interactions); (2) primary outcomes and guardrails (e.g., completion rate, total watch time per user, creator retention, session length, ad revenue per mille, latency/crash rate); (3) traffic allocation, ramp schedule, and expected MDE with power assumptions; (4) how you will detect novelty and learning effects and set experiment duration; (5) checks for SUTVA/interference and cluster variance; and (6) a rollout plan if results pass. Be concrete about metric definitions and instrumentation.