Katkat
Back

Prevent a Cache Stampede on a Hot Key

ByteDanceSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSystem Design

Last updated

This question tests the ability to design robust cache-aside mechanisms to prevent database overload during cache expiration events for high-traffic keys.

System Design: Preventing Cache Stampede

You are designing a distributed service that retrieves metadata for viral video content. You use a cache-aside pattern (e.g., Redis) to store this metadata. During peak traffic, a 'hot key' (a video ID with millions of requests per minute) expires, causing a cache miss. Thousands of concurrent requests then hit the backend database simultaneously, potentially causing a database crash—a phenomenon known as a cache stampede.

Requirements

  1. Design a mechanism to ensure only one instance performs the database fetch when a hot key expires.
  2. Handle scenarios where the database fetch might be slow or fail entirely.
  3. Define how to keep the system responsive even if the cache is empty.

Your answer should address:

  • Synchronization Strategy: How to ensure only one process updates the cache (e.g., mutexes, semaphore tokens, or request coalescing).
  • Failure Handling: What happens if the sole worker responsible for updating the cache fails or times out?
  • Freshness Contract: How do you balance the need for fresh data with the need for high availability during surges?
  • Trade-offs: Contrast the use of 'soft expiration' (serving stale data while revalidating) versus blocking mechanisms.

Hints

Hint 1

Consider how a distributed lock or an atomic 'check-and-set' operation can limit backend access to a single request.

Hint 2

Evaluate 'stale-while-revalidate'—storing an extra 'logical' expiration time in the cache object to allow background updates while serving old data.

Hint 3

Think about how adding randomized jitter to expiration times or request delays can distribute the load if cache misses occur across a cluster.

Follow-up questions

  • How would you monitor the system to detect if a stampede is actively occurring versus a legitimate spike in traffic?
  • If the database is down, how should your system behave to avoid returning constant 500 errors to users?
  • How does your strategy change if you need to guarantee 'read-your-writes' consistency for the user updating the metadata?