Detect fake accounts and measure their impact
MetaAnalytics Engineer · OnsiteLast updated
This question tests the ability to design an analytical framework for identifying fraudulent behavior and quantifying the financial or engagement impact of those accounts on a platform.
Scenario
You are an Analytics Engineer at a social media platform. The product team suspects a surge in bot activity and automated fake accounts that inflate advertising metrics and degrade user experience. You have access to a massive event-level dataset containing user login timestamps, IP addresses, device IDs, account creation dates, and ad-click logs.
Task
- Detection Strategy: Propose a methodology to identify fake accounts. Define at least three behavioral or technical signals (features) that would distinguish a bot from a legitimate user.
- Impact Measurement: How would you calculate the 'negative impact' of these fake accounts on two specific metrics: (a) Ad Revenue (e.g., invalid clicks) and (b) Daily Active Users (DAU) inflation.
- Validation: Suppose you implement a removal policy that deletes detected fake accounts. How would you design an experiment (or quasi-experiment) to measure the true impact of this removal on the platform's ecosystem without negatively impacting legitimate user engagement metrics?
Constraints
- Assume the platform logs billions of events daily.
- You must account for false positives—avoiding the deletion of legitimate users is critical.
- The measurement of impact should distinguish between 'organic' noise and 'malicious' bot behavior.
Expected Output
- Describe the technical approach for feature engineering.
- Explain the mathematical framework for attribution (i.e., how do you subtract bot activity from total metrics?).
- Detail the experimental design, including treatment/control group definition and potential confounding variables.
Hints
Hint 1
Consider clustering accounts based on shared device IDs or geographic anomalies during peak login hours.
Hint 2
For revenue impact, look at the ratio of ad clicks to user interaction depth (time on page or scroll depth) as a filtering heuristic.
Hint 3
For the experiment, consider a 'Holdout' or 'Shadow Deletion' group where you track the performance of the platform with and without the detected bot population.
Follow-up questions
- How would you handle a situation where a coordinated bot network evolves to mimic human behavior (e.g., randomizing login times and browsing habits)?
- What are the operational risks if the model used to flag fake accounts has a high false-positive rate on power users?
- How does your detection strategy change if the malicious actor uses residential proxy services to rotate IP addresses?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.