Katkat
Back

Design a Social Feed

MercorMachine Learning Engineer · Technical Screen
EasyMachine Learning EngineerTechnical ScreenSystem Design

Last updated

Design an end-to-end personalized social feed system capable of serving low-latency ranked content while handling hybrid fanout, multi-stage ML candidate retrieval, and stable pagination.

Scenario

Design the feed generation and ranking architecture for a personalized social network (similar to X/Twitter or LinkedIn). Users can publish short text/media posts, follow other users, and view an algorithmic home feed that blends fresh content from followed accounts with recommended discovery posts.

Requirements

Functional Requirements

  • Post Creation & Deletion: Users can publish new posts and delete existing posts.
  • Social Graph: Users can follow and unfollow other creators.
  • Personalized Feed: Users can view an infinite-scrolling feed combining followed content and discovery content, ranked by expected engagement.
  • Pagination: Support consistent, stable pagination as new posts are published in real time.

Non-Functional Requirements

  • Low Latency: p99 feed load latency under 200 ms.
  • High Availability: Eventual consistency for post delivery; high availability for feed reads.
  • Scalability: 100 million Daily Active Users (DAU), average 10 feed reads per user per day, 50 million total posts created per day (read-heavy, ~20:1 read-to-write ratio).

What Your Design Should Cover

  1. Core Architecture & Data Flow:

    • High-level component diagram showing ingestion, storage, feed generation, and serving.
    • Database choices for user posts, social graph, and pre-materialized feeds.

  2. Fanout Strategy (Push vs. Pull vs. Hybrid):

    • How to handle standard users vs. high-follower accounts ("celebrity problem").
    • When and where feed materialization occurs.

  3. ML Candidate Generation & Ranking Pipeline:

    • Two-stage architecture: Candidate retrieval (in-network inbox + out-of-network vector search) followed by scoring (heavy ranking model).
    • Feature store integration (real-time user context vs. offline batch features).
    • Hydration step (fetching author metadata, post assets, like counters) after ranking.

  4. Operational & Scale Trade-offs:

    • Handling immediate post deletions and privacy changes (e.g., blocked users).
    • Stable cursor-based pagination strategy to avoid duplicates and dropped posts.

Hints

Hint 1

Consider how high-follower accounts make a pure fanout-on-write model unsustainable, and analyze why a hybrid push-pull approach resolves write amplification.

Hint 2

Split feed generation into candidate retrieval (reducing millions of items to thousands), scoring/ranking (reducing thousands to tens), and late hydration of rich metadata.

Hint 3

Use cursor-based pagination tied to item scores or ranking timestamps rather than offset-based pagination to avoid duplicate or skipped posts as new items arrive.

Follow-up questions

  • How would you ensure that a deleted post or an updated privacy setting is immediately purged from user feeds if feed IDs are already cached in Redis?
  • How can you incorporate real-time session signals (e.g., user skipped three video posts in a row) into the ranking model without re-running full candidate retrieval?
  • How do you combine multiple predicted probabilities (e.g., p(click), p(like), p(dwell_time > 10s), p(hide)) into a final ranking score while balancing creator diversity?