Apply sequential testing without p-hacking
MetaData Scientist · Onsite
Last updated
This question evaluates a candidate's competency in sequential hypothesis testing, alpha‑spending rules (Pocock vs O’Brien–Fleming), always‑valid inference methods such as mixture‑SPRT and e‑values, and strategies for controlling error rates across multiple metrics.
You want continuous monitoring with early stopping for efficacy or futility. a) Contrast Pocock vs. O’Brien–Fleming alpha‑spending for two interim looks and report the corresponding critical z‑values. b) Explain how mixture‑SPRT or e‑values yield always‑valid inference under arbitrary peeking. c) Describe how you would integrate sequential monitoring with multiple metrics without inflating error rates.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.