Design a Spam Review Workflow with Classifiers, AI Agents, and Human Labelers
GoogleData Analyst · Onsite
Last updated
Design a spam review workflow combining classifiers, AI agents, and human labelers with accuracy, throughput, and label-quality evaluation.
Design a Spam Review Workflow with Classifiers, AI Agents, and Human Labelers
Design a spam detection system that combines a classifier, an AI agent, and human labelers. The main objective is to improve the accuracy and efficiency of human review. Explain the responsibilities of each component, the routing of cases, and how you would evaluate the workflow. Discuss useful AI capabilities and their limitations without assuming an agent's output is reliable ground truth.
Hints
Hint 1
Separate assistance from authority: Consider which evidence an agent can organize and which consequential decisions still require verified rules or human judgment.
Follow-up questions
- How would you detect whether the agent makes reviewers faster but less accurate?
- How would you learn about spam that the classifier never sends to review?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.