Katkat
Back

Design comment ranking

RedditMachine Learning Engineer · Onsite
HardMachine Learning EngineerOnsiteML System Design

Last updated

This problem evaluates your ability to design a scalable comment ranking system that balances engagement, relevance, and safety, while addressing the challenges of low-latency inference and data freshness.

Scenario

You are tasked with building a real-time comment ranking system for a high-traffic platform like Reddit. Unlike global post ranking, comment ranking must handle hierarchical threads and significantly higher volumes of item-to-user interactions.

Functional Requirements

  • Surface "meaningful" discussions at the top of threads.
  • Minimize toxic or spam content exposure.
  • Ensure users see diverse viewpoints while maintaining relevance.

Non-Functional Requirements

  • Latency: Ranking must happen in under 100ms for a thread page load.
  • Freshness: New comments should be rankable within seconds of posting.
  • Scale: Handle millions of active threads and billions of daily interactions.

Deliverables

  1. High-level architecture: Describe the pipeline from comment ingestion to serving the ranked list.
  2. Model strategy: Define your candidate generation, ranking model architecture (e.g., Two-Tower, Multi-Task Learning), and how you handle labels (e.g., upvotes, replies, report rates).
  3. Features: Propose key features for users, comments, and the context of the thread.
  4. Trade-offs: Discuss how to balance freshness with compute cost and how to handle the cold-start problem for new comments.

Hints

Hint 1

Consider using a multi-stage approach: a fast candidate retrieval phase followed by a more complex, heavy-duty ranking model (re-ranking).

Hint 2

Think about how to balance explicit signals (upvotes/downvotes) with implicit signals (dwell time, reply depth, block/report actions) in your label design.

Hint 3

Address the cold-start problem for new comments by utilizing content-based features (NLP embeddings) before engagement data accumulates.

Follow-up questions

  • How would you modify your architecture to ensure that controversial but healthy debate is not unfairly suppressed by a model that prioritizes positive sentiment?
  • How do you implement and validate a system to prevent 'feedback loops' where popular comments get exponentially more visibility regardless of quality?
  • If the latency budget was reduced to 20ms, which components of your model would you sacrifice or optimize, and why?