Katkat
Back

Design Metrics to Track and Analyze Spam Impact

MetaData Scientist · Technical Screen
MediumData ScientistTechnical ScreenAnalytics & Experimentation

Last updated

This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Design Metrics to Track and Analyze Spam Impact states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Scenario

Product team wants metrics and experiment design to reduce spam without harming normal user experience.

Question

Without a classifier table, what alternative metrics would you track to monitor spam and overall user experience? If report rate declines, what potential causes could explain it and what extra metrics would you examine? When running an anti-spam A/B test where spammers are rare, how would you select test and control groups?

Hints

Consider message volume, unique senders, report-per-message, acceptance rate, stratified sampling, power.