Katkat
Back

Design a harmful video content moderation system

OpenAIMachine Learning Engineer · Onsite
HardMachine Learning EngineerOnsiteML System Design

Last updated

This question evaluates a candidate's competency in designing scalable, multi-modal machine learning systems for content moderation, covering architecture, model strategy, labeling and human-in-the-loop workflows, metrics, auditability, and defenses against adversarial evasion.

Question

Design an end-to-end system to detect and moderate harmful videos on a large platform.

Requirements

  • Detect multiple policy categories (violence, self-harm, hate, sexual content, etc.).
  • Operate at upload time and for already-published content.
  • Combine automated decisions with human review.
  • Minimize both false negatives (missed harmful content) and false positives (wrongful removals).
  • Provide appeals and auditability.

Deliverables

Architecture, model strategy (multi-modal), labeling/review workflow, metrics, and how you handle adversarial evasion.