Katkat
Back

Design a Global Small-Image Caching Service

WaymoSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSystem Design

Last updated

This problem tests the ability to design a highly available, globally distributed caching layer that handles high-throughput small-image delivery while maintaining data consistency and resilience against origin failures.

Design a Global Small-Image Caching Service

Design a caching architecture for a platform that serves small images (thumbnails, icons, metadata assets) globally. The system must support hundreds of thousands of requests per second with low latency.

Requirements

Functional Requirements:

  • GET /image/{id}: Fetch an image.
  • POST/PUT /image/{id}: Update an image (invalidates existing caches).
  • Support versioning to handle stale client requests.

Non-Functional Requirements:

  • Latency: Sub-50ms p99 latency for global users.
  • Availability: High availability; the system must remain functional even if the primary database (origin) experiences intermittent downtime.
  • Burst Protection: Prevent 'thundering herd' problems when a popular image's cache expires.

Considerations

  • Cache Hierarchy: How should you structure local, regional, and global caches?
  • Consistency: How do you handle cache invalidation globally while maintaining eventual consistency?
  • Staleness: How do you handle cases where the origin database is down?
  • Scale: Explain how the architecture handles traffic spikes for specific 'hot' images.

Expectations

Provide a high-level architecture diagram or description, explain the data flow, discuss trade-offs in consistency models, and detail how the system reacts to a regional outage or an origin-side failure.

Hints

Hint 1

Consider a multi-tiered approach: Edge (CDN/PoP) caches for speed, regional caches for hit rate, and the origin for source of truth.

Hint 2

To handle origin outages, implement 'stale-while-revalidate' or 'graceful degradation' policies where expired objects are served while a background fetch is attempted.

Hint 3

Use a distributed message bus or Pub/Sub mechanism to propagate invalidation events to regional nodes to minimize the window of inconsistency.

Follow-up questions

  • How would you change your design if images were highly personalized per user versus being public assets?
  • If the origin database is experiencing high load during cache misses, how can you implement request collapsing (also known as request coalescing)?
  • How would you measure cache effectiveness, and what metrics would you monitor to detect a cache poisoning attack?