Determine Metrics to Measure Free-Trial Impact on Subscriptions
OpenAIData Scientist · Online Assessment
Last updated
Determine Metrics to Measure Free-Trial Impact on Subscriptions evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
Scenario
A/B test that offers free users a free trial of the paid plan; dataset includes user_id, treatment assignment, sign-ups and cancels.
Question
Which primary metrics would you track to measure the free-trial’s impact on subscription behaviour? Define each metric precisely. Calculate the lift in conversion to paid subscription between treatment and control and test whether the difference is statistically significant. Estimate the treatment effect on churn (cancels) within 30 days. Is the effect practically meaningful? Given your results, would you recommend rolling the free-trial out to all users? Justify your answer.
Hints
Choose consistent denominators, run two-sample tests, and discuss both statistical and business significance.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.