Katkat

Detect fake accounts and measure their impact

MetaAnalytics Engineer · Onsite

Last updated

This question tests the ability to design an analytical framework for identifying fraudulent behavior and quantifying the financial or engagement impact of those accounts on a platform.

Scenario

You are an Analytics Engineer at a social media platform. The product team suspects a surge in bot activity and automated fake accounts that inflate advertising metrics and degrade user experience. You have access to a massive event-level dataset containing user login timestamps, IP addresses, device IDs, account creation dates, and ad-click logs.

Task

  1. Detection Strategy: Propose a methodology to identify fake accounts. Define at least three behavioral or technical signals (features) that would distinguish a bot from a legitimate user.
  2. Impact Measurement: How would you calculate the 'negative impact' of these fake accounts on two specific metrics: (a) Ad Revenue (e.g., invalid clicks) and (b) Daily Active Users (DAU) inflation.
  3. Validation: Suppose you implement a removal policy that deletes detected fake accounts. How would you design an experiment (or quasi-experiment) to measure the true impact of this removal on the platform's ecosystem without negatively impacting legitimate user engagement metrics?

Constraints

  • Assume the platform logs billions of events daily.
  • You must account for false positives—avoiding the deletion of legitimate users is critical.
  • The measurement of impact should distinguish between 'organic' noise and 'malicious' bot behavior.

Expected Output

  • Describe the technical approach for feature engineering.
  • Explain the mathematical framework for attribution (i.e., how do you subtract bot activity from total metrics?).
  • Detail the experimental design, including treatment/control group definition and potential confounding variables.

Hints

Hint 1

Consider clustering accounts based on shared device IDs or geographic anomalies during peak login hours.

Hint 2

For revenue impact, look at the ratio of ad clicks to user interaction depth (time on page or scroll depth) as a filtering heuristic.

Hint 3

For the experiment, consider a 'Holdout' or 'Shadow Deletion' group where you track the performance of the platform with and without the detected bot population.

Follow-up questions

  • How would you handle a situation where a coordinated bot network evolves to mimic human behavior (e.g., randomizing login times and browsing habits)?
  • What are the operational risks if the model used to flag fake accounts has a high false-positive rate on power users?
  • How does your detection strategy change if the malicious actor uses residential proxy services to rotate IP addresses?