Katkat
Back

Design a Pastebin Service: Requirements, Entities, APIs and End-to-End Architecture

Goldman SachsSoftware Engineer · Onsite
EasySoftware EngineerOnsiteSystem Design

Last updated

Design a scalable text sharing (Pastebin) service that generates unique short URLs for submitted text snippets. This question assesses API design, data modeling, storage decoupling, key generation strategies, and caching mechanisms under high read concurrency.

Scenario

Design a Pastebin-like text sharing service where users can paste plain text or code snippets, receive a unique shortened URL, and share that URL with others. Recipients can access the original text via the shortened link until the snippet expires or is manually deleted.

Requirements

Functional Requirements

  • Create Paste: Users can submit a text payload and optionally specify an expiration time (e.g., 10 minutes, 1 day, 1 month, or never) and a custom alias.
  • Read Paste: Given a valid short key, retrieve and display the original text content.
  • Delete Paste: Anonymous users cannot delete after creation, but authenticated creators can explicitly delete their pastes.
  • Expiration: Pastes must no longer be accessible once their expiration timestamp has passed.

Non-Functional Requirements

  • High Availability and Low Latency: Reads should be sub-50ms; the read-to-write ratio is heavily read-skewed (approx. 10:1 to 50:1).
  • Durability: Uploaded snippets must not be lost before expiration.
  • Uniqueness: Short URL keys must be unique and non-guessable if marked private.

Scale & Numbers

  • Write Traffic: 10 million new pastes per day (~115 writes/sec average, ~500 writes/sec peak).
  • Read Traffic: 100 million reads per day (~1,150 reads/sec average, ~5,000 reads/sec peak).
  • Size Limits: Maximum paste size is 2 MB; average paste size is 10 KB.
  • Storage Volume: Approximately 100 GB of new text content daily (~36.5 TB/year without purges).

What Your Design Should Cover

  1. API Design: Define RESTful endpoints for creating, retrieving, and deleting pastes, including input/output payloads and HTTP status codes.
  2. Data Model & Storage Strategy: Outline the database schema. Discuss why separating text content (blob storage / object store) from metadata (relational or NoSQL database) is beneficial.
  3. Key Generation Strategy: Contrast pre-generating keys (Key Generation Service / token range allocation) vs. hashing on write (e.g., Base62 of counter or MurmurHash with collision resolution).
  4. End-to-End Architecture: Walk through read and write paths including load balancers, application services, caching layers (Redis/Memcached), and background cleanup jobs for expired records.
  5. Edge Cases & Trade-offs: Handle hot pastes, rate limiting, and eventual consistency between cache and primary storage.

Hints

Hint 1

Storing large text bodies directly inside a relational or document database index causes table bloat; consider storing metadata and actual content in separate storage systems.

Hint 2

Using a simple counter or Base62 hash of an auto-incrementing ID makes URLs easily guessable and creates write-lock bottlenecks; look into pre-generated token pools or distributed ID generators.

Hint 3

Active deletion of expired rows during peak hours can degrade database performance; explore lazy expiration on read paired with asynchronous background batch cleanup.

Follow-up questions

  • How would you handle a viral paste that receives tens of thousands of requests per second without overwhelming the storage layer?
  • If a corporate compliance policy requires client-side end-to-end encryption, how does that impact your API and server-side architecture?
  • How would you implement rate limiting and abuse detection to prevent malicious users from spamming large blobs or performing denial-of-service attacks?