Experiment on increasing order notifications

Context: Notifications team wants to increase order-related push notifications to drive more sessions and conversions. Tasks:

1. Propose an experiment design to measure incremental impact of higher notification frequency and/or smarter timing. Specify randomization unit (user), exposure caps, time-of-day stratification, and controls for cross-channel effects (email/SMS). Consider multi-armed or bandit variants and explain when you would or would not use them.
2. Define primary success metrics (incremental orders, revenue, profit), and guardrails (opt-out rate, uninstall rate, complaint rate, session quality). Include per-user and per-notification level metrics and the aggregation strategy.
3. Measure long-term effects: design a holdout or staggered-rollout with a 4–8 week follow-up, addressing novelty effects, fatigue, and decay. Describe how you will estimate persistent lift (e.g., switchback tests, long-lived holdouts, synthetic controls).
4. Handle interference and repeated exposure: propose analysis that accounts for saturation and diminishing returns (e.g., dose-response curves, exposure-weighted treatment, instrumental variables via randomized send/no-send at trigger level).
5. Define power/MDE assumptions, stopping rules, and a decision framework that balances short-term lift vs long-term retention risk. Include a concrete rollback criterion.