Katkat
Back

Design an experiment to launch fractional shares

RobinhoodData Scientist · Onsite
EasyData ScientistOnsiteAnalytics & Experimentation

Last updated

This question evaluates a candidate's competency in experimentation design, product analytics, statistical power and sample-size calculation, causal inference, and risk identification within a brokerage product context.

Context

You work on a brokerage/investing app. The team is considering launching fractional share trading (users can buy/sell fractions of a stock/ETF instead of whole shares).

Task

  1. Explain the expected benefits of fractional shares for the business and for users.
  2. Propose an experimentation plan to evaluate the launch:
  • What is the primary success metric?
  • What diagnostic metrics help explain movement?
  • What guardrail metrics ensure you don’t harm users or the business?
  • Define the unit of randomization (user/account/household/etc.) and the treatment/control experience.
  • Call out key risks: interference/network effects, novelty effects, seasonality, and any selection bias.
  1. Sample size / power
  • Show how you would estimate the required sample size for the primary metric (state assumptions such as baseline rate/variance, MDE, (\alpha), and power).
  • If you need to adjust the sample size mid-experiment, explain a statistically valid approach.
  1. Constraint scenario
  • If the required sample size is too large (you don’t have enough traffic or time), what do you do? Provide at least 3 practical options (design, metrics, variance reduction, ramp strategy, or alternative inference methods) and discuss tradeoffs.

Output

Provide a structured written plan, including metric definitions, assumptions used in power/MDE calculations, and a final recommendation on whether/how to launch.