Katkat
Back

Compute Bayes probability for fake accounts

MetaData Scientist · Onsite
EasyData ScientistOnsiteStatistics & Math

Last updated

This question tests the ability to apply Bayes' Theorem in a practical classification scenario, specifically accounting for low base-rate bias and conditional probabilities.

Scenario

Meta is running an automated detector to identify fake accounts. Suppose that 0.1% of all newly created accounts are fake. The detector has a sensitivity (true positive rate) of 95%—meaning it correctly flags 95% of fake accounts. It also has a false positive rate of 2%, meaning it incorrectly flags 2% of legitimate (non-fake) accounts as fake.

Task

  1. Calculate the probability that an account is actually fake given that the detector flagged it.
  2. Suppose an account is flagged by two independent detectors, both with the same sensitivity and false positive rates as described above. What is the probability that the account is fake, given that both detectors flagged it?

Constraints and Assumptions

  • Assume that each detector's error is independent given the status of the account (fake or real).
  • Provide the result as a percentage rounded to two decimal places.
  • Show the logic used for both calculations.

Hints

Hint 1

Use Bayes' Theorem: P(A|B) = [P(B|A) * P(A)] / P(B), where P(B) is the total probability of being flagged.

Hint 2

For the first part, remember that P(Flagged) = P(Flagged|Fake) * P(Fake) + P(Flagged|Real) * P(Real).

Hint 3

For the second part, treat the dual-flag event as a single condition where the probability of being flagged by both detectors is the product of their individual conditional probabilities (assuming independence).

Follow-up questions

  • How does the prevalence (base rate) of fake accounts significantly impact the Precision (Positive Predictive Value) of your detector?
  • If we wanted to increase our Precision to 50%, what would we need to change regarding the detector's thresholds or our operating point?
  • What are the real-world implications of 'independence' between detectors; i.e., why might two different models often fail in the same way?