Katkat
Back

Design an RSS News Feed Service

ConfluentSoftware Engineer · Onsite
MediumSoftware EngineerOnsiteSystem Design

Last updated

This question evaluates a candidate's ability to design a scalable RSS news feed service, emphasizing system architecture, relational schema and API design, including database table and index planning, ingestion and serving paths, deduplication/idempotency, and feed generation trade-offs.

Design an RSS news feed service.

The service lets users subscribe to RSS sources, ingests articles from those sources on a schedule, stores article metadata, and serves each user a personalized feed of articles drawn from the sources they follow. You do not need to draw diagrams, but you must clearly explain the main components, the storage model, and the request flows.

This interview emphasizes two areas above all others: database table design and API design. Spend the bulk of your time there. Be concrete — name the tables, their columns, primary keys, the indexes that make the hot queries fast, and the endpoints with their request/response shapes.

Constraints & Assumptions

  • Single logical service; you may assume a relational primary store (e.g. PostgreSQL) with the option to offload feed items to a key-value/wide-column store at scale.
  • Eventual consistency is acceptable: a newly published article need not appear in feeds instantly — seconds-to-minutes of lag is fine.
  • Sources number in the hundreds of thousands; a popular source may be followed by a large fraction of users (fanout hot spot).
  • Feed reads dominate traffic and must be low-latency; ingestion is bursty (some sources publish many items at once).
  • RSS metadata is unreliable: guid may be missing, published_at may be absent or wrong, URLs may change, and the same article may be syndicated by multiple sources.
  • You may assume an external auth layer supplies an authenticated user_id.

Clarifying Questions to Ask

  • What is the read:write ratio, and roughly how many sources does a typical user subscribe to (tens? thousands)? This drives fanout-on-read vs. fanout-on-write.
  • How fresh must feeds be — is minutes-of-lag acceptable, or is near-real-time required?
  • Do we need full-text article content and search, or only metadata (title, summary, link)?
  • What per-user state must we track (read / saved / hidden), and is read-state required to be strongly consistent?
  • Are there a few "celebrity" sources with millions of subscribers we must plan a fanout hot spot around?
  • Do we control the article publishers, or are these arbitrary third-party RSS feeds (affecting fetch politeness, rate limits, and trust)?

Hints

Hint 1

Where to start: Split the system into two paths and design for each independently: the **ingestion path** (scheduler → fetch RSS → parse → store articles) and the **serving path** (a user requests their feed). Which of the two does your traffic pattern say must be the cheapest? Let that decision drive the schema.

Hint 2

Data model: Sketch the entities before the columns. Which tables do you need so that "give me one user's feed" is answerable, and how do you model the relationship between users and the sources they follow? For each of your hot queries — listing a user's subscriptions, finding who follows a source, reading recent articles for a source — ask what single index would let that query avoid a sort or a scan.

Hint 3

Dedup and idempotency: RSS fetches get retried, and the same article often reappears across re-fetches (and across syndicating sources). What property must an article insert have so a retry is harmless? What field would you key dedup on first — and what's your fallback when that field is missing or unstable? Then ask which layer (application vs. database) should *enforce* "no duplicates."

Hint 4

Feed generation — the key design decision: The central question is *when* a user's feed is assembled: at read time (merge recent articles across their subscribed sources on request) or at write time (materialize per-user rows as each article lands). Weigh read cost vs. write amplification — especially what a single source followed by a huge fraction of users does to each approach. State which you'd pick by default and what would make you switch.

Hint 5

Pagination: The naive choice (`OFFSET`) breaks when new articles arrive mid-scroll — rows duplicate or get skipped. What stable key could you page on instead so the boundary stays fixed as data shifts, and how does that key line up with the index backing the feed read?

Follow-up questions

  • A single source suddenly gains millions of subscribers and publishes a burst of articles. How does your fanout-on-write path avoid melting? (Consider lazy/hybrid fanout, batching, and treating celebrity sources as pull-only.)
  • A user unsubscribes from a source they were already materialized into under fanout-on-write. How do you keep stale articles out of their feed without an expensive synchronous delete?
  • Two different sources syndicate the identical article. How do you detect the duplicate at ingestion time, and how do you decide which copy a user sees?
  • How would you add full-text search over article titles and summaries, and where does that index live relative to the primary store?