Katkat
Back

Estimate fake-account prevalence with capture-recapture

MetaData Scientist · Onsite
MediumData ScientistOnsiteStatistics & Math

Last updated

This question tests the ability to apply the Lincoln-Petersen estimator to estimate a hidden population size and addresses the limitations of the assumptions underlying capture-recapture methods.

Scenario

Assume you are working on estimating the total number of fake accounts on a social media platform. You have two independent detection systems:

  1. System A: Uses a machine learning classifier based on account metadata.
  2. System B: Uses a graph-based algorithm analyzing suspicious connection patterns.

Let N be the total number of fake accounts. From your logs, you observe the following data:

  • Accounts flagged by System A only (n_a): 800
  • Accounts flagged by System B only (n_b): 600
  • Accounts flagged by both systems (n_both): 200

Task

  1. Calculate the estimated total number of fake accounts (N_hat) using the Lincoln-Petersen estimator.
  2. Calculate the estimated prevalence if the total number of accounts on the platform is 1,000,000.
  3. Discuss the statistical bias introduced if the two systems are not actually independent (e.g., if both systems rely on a shared, flawed user-registration feature).

Hints

Hint 1

Recall the formula for the Lincoln-Petersen estimator: N = (M * C) / R, where M is the size of the first capture, C is the size of the second capture, and R is the recaptures.

Hint 2

Define M as (n_a + n_both) and C as (n_b + n_both), then treat n_both as the overlap R.

Hint 3

If the systems are positively correlated (e.g., both detect the same 'easy' cases), the overlap R will be larger than expected under independence, leading to an underestimation of N.

Follow-up questions

  • How would you adjust your estimate if you know System A has a high false-positive rate but System B is highly precise?
  • What statistical methods could you use to validate the independence assumption between the two detection models?
  • If you added a third detection system (System C), how would the calculation complexity change and what benefits would it provide?