Design experiments for marketplace balance

You propose a new supplier prioritization (ranking) policy intended to increase order completion in a two-sided marketplace with known interference between users and suppliers. (a) Experiment design: Choose an appropriate design (user-level randomization, supplier-level randomization, geo-clustered test, or switchback). Justify your choice given network effects, supply constraints, and spillovers. Define exposure and eligibility, bucketing, and how you will prevent cross-contamination. (b) Metrics: Define the primary metric (e.g., completion rate or GMV per session) and guardrails (e.g., cancellation rate, supplier wait time, fairness across cities/suppliers). Specify event triggers and the exact aggregation level. (c) Power: Baseline completion rate is 60%; the minimum detectable effect is +2 percentage points (absolute). Compute the per-variant sample size for a two-sided test with α=0.05 and power=0.80 under independent Bernoulli outcomes. Then adjust for clustering with a design effect of 1.2 and state the final sample size. Show formulas. (d) Analysis: Detail how you will handle heterogeneous treatment effects (by city, time-of-day, supplier capacity quartile), sequential monitoring without inflating Type I error, and variance reduction (e.g., CUPED or covariate adjustment). Explain how you would detect and correct for marketplace rebalancing artifacts (e.g., improvements in treated geos causing degradations elsewhere). (e) Rollout: Propose a ramp plan and a fallback decision rule if guardrails breach for two consecutive days.