Katkat
Back

Choose target customers and define success metrics

StripeData Scientist · Onsite
MediumData ScientistOnsiteAnalytics & Experimentation

Last updated

This question tests the ability to design an experimental rollout strategy for a new financial product, requiring a balance between revenue growth, risk mitigation, and statistical validity.

The Problem

Stripe is launching an 'Instant Payout' feature that allows small-business merchants to receive funds within minutes rather than days for a small fee. You are tasked with designing the pilot rollout to the first 10% of existing small-business merchants.

Your Task

  1. Segmentation: Define a scoring model to identify the 'ideal' 10% of merchants for this initial pilot. State the features you would prioritize (e.g., volume, dispute history, payout velocity) and why.
  2. Experiment Design: Propose a testing framework (e.g., A/B test, switchback, or cluster randomization). Explain how you would calculate the required sample size and minimum detectable effect (MDE) to validate the feature's profitability.
  3. Success Metrics & Guardrails: Define primary metrics for success and identify critical guardrails. How do you measure the risk of fraudulent actors exploiting the instant payout feature?
  4. ROI Calculation: Provide a high-level formula for the return on investment (ROI) of this rollout, accounting for the cost of capital, operational support, and expected fee revenue.

Assumptions

  • You have access to historical transaction, payout, and dispute data for all merchants.
  • The product carries a risk of 'day-zero' losses if a merchant closes their account after receiving an instant payout but before the underlying transactions clear.

Provide your logic and clearly state all assumptions made regarding merchant behavior or financial risk.

Hints

Hint 1

Consider using a risk-adjusted score that weighs high transaction volume against the merchant's history of chargebacks.

Hint 2

For the experiment design, think about whether a randomized control trial (RCT) at the merchant level is feasible or if network effects make a geo-based or time-based approach safer.

Hint 3

When defining guardrails, focus on leading indicators of fraud, such as sudden spikes in refund requests or anomalous high-velocity activity.

Follow-up questions

  • How would your strategy change if we were targeting large enterprise merchants instead of small businesses?
  • If the pilot shows high adoption but a slight increase in chargeback rates, how would you decide whether to terminate, pivot, or scale the product?
  • How would you account for the 'adverse selection' effect, where merchants who most urgently need cash are also the ones most likely to be high-risk?