Katkat
Back

Design a Content Moderation Platform

ByteDanceSoftware Engineer · Onsite
MediumSoftware EngineerOnsiteML System Design

Last updated

This problem evaluates your ability to design a scalable, multi-modal content moderation system that filters user-uploaded media using both automated ML inferences and human review workflows.

System Design: Content Moderation Platform

Design a global content moderation platform for a video-sharing application. The platform must process millions of video uploads per day and determine if content violates community guidelines (e.g., violence, hate speech, or nudity).

Functional Requirements

  1. Real-time Assessment: Videos must be screened as quickly as possible upon upload.
  2. Multi-modal Analysis: Analyze both video frames and audio tracks.
  3. Human-in-the-loop: High-uncertainty predictions must be routed to human reviewers.
  4. Auditability: Maintain an immutable log of moderation decisions and associated model metadata for compliance.

Non-functional Requirements

  1. Low Latency: Provide an initial moderation signal within 5 seconds.
  2. Scalability: Handle significant traffic spikes (e.g., viral trends).
  3. Accuracy: Minimize false negatives for prohibited content.

Expected Output

  1. High-level Architecture: Diagram of services (ingestion, feature extraction, inference, queueing, and human review).
  2. ML Pipeline: Strategies for serving models and updating them with new training data.
  3. Data Model: Briefly describe the schema for storing content metadata and moderation decisions.
  4. Trade-offs: Discuss the latency vs. throughput trade-offs in multi-modal classification.

Hints

Hint 1

Consider an asynchronous architecture where video ingestion is decoupled from the intensive ML inference processes.

Hint 2

Think about how to handle 'gray area' content using probability thresholding and routing to a dedicated queue for manual human review.

Hint 3

Address the cold-start problem and how you would efficiently ingest and label new edge cases to improve model performance over time.

Follow-up questions

  • How would you handle a scenario where a specific category of prohibited content (e.g., a new viral meme format) requires an immediate, site-wide update to the moderation logic?
  • If the inference cost is too high, what sampling or heuristic-based strategies would you implement to reduce compute requirements?
  • How would you design a feedback loop to ensure that human reviewers' decisions are used to re-train the models and correct model bias?