Katkat
Back

Evaluate Probability of Positive User Comments and Model Performance

MetaData Scientist · Onsite
MediumData ScientistOnsiteStatistics & Math

Last updated

Evaluate Probability of Positive User Comments and Model Performance evaluates statistical assumptions, formulas, estimation strategy, uncertainty, edge cases, and interpretation in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Scenario

You are an analyst evaluating user comments and model performance on a social-media platform.

Question

If the probability that a single user comment is positive is p, what is the probability that two independent comments are both positive? Assuming each response is i.i.d., if the first three responses from a user are positive, what is the probability that the fourth response is also positive? Model A returns a positive response in 80 % of cases while Model B returns a positive response in 90 % of cases. At the 5 % significance level, can you conclude Model B is better? Detail the statistical test, null/alternative hypotheses, test statistic and decision rule.

Hints

Use the multiplication rule for independent events and a two-proportion z-test with a pooled estimate of p.