Katkat
Back

Design a simple API rate limiter

MicrosoftSoftware Engineer · Onsite
HardSoftware EngineerOnsiteSystem Design

Last updated

This problem tests the ability to design a distributed rate-limiting service that maintains low latency and high accuracy across multiple application server instances.

System Design: Distributed API Rate Limiter

Design a rate-limiting service that restricts the number of requests a client can make to your API within a specific time window (e.g., 100 requests per minute per API key).

Functional Requirements

  • Enforce rate limits based on unique API keys.
  • Support multiple distinct API endpoints with different limit configurations.
  • Return a standard HTTP 429 (Too Many Requests) status code when the limit is exceeded.

Non-Functional Requirements

  • Low Latency: The rate-check logic should not add significant overhead to the request path (target < 10ms).
  • Accuracy: The system should handle concurrent requests from the same client correctly.
  • Scalability: The system must support thousands of requests per second across a cluster of application servers.
  • Fault Tolerance: A failure in the rate-limiting component should not result in a total system outage (fail-open or fail-closed behavior).

Requirements for your design:

  1. Define the API schema/interface for the client application.
  2. Choose a storage backend for storing counters and timestamps, justifying your choice.
  3. Propose a high-level architecture. How do application servers communicate with the rate limiter?
  4. Discuss how you handle race conditions when multiple nodes receive requests for the same user simultaneously.

Hints

Hint 1

Consider the trade-offs between different rate-limiting algorithms like Fixed Window, Sliding Window Log, and Sliding Window Counter.

Hint 2

Using an in-memory data store like Redis can provide the necessary speed; consider using Lua scripts or atomic operations (INCR/EXPIRE) to avoid race conditions.

Hint 3

Think about how the architecture changes if you decide to perform rate-limiting locally on each application server versus a centralized service.

Follow-up questions

  • How would you handle a 'bursty' traffic pattern where a client sends many requests in the first second of a minute?
  • If you move to a multi-region deployment, how do you handle synchronization of rate-limit counters across geographical boundaries?
  • How would you implement a 'graceful degradation' strategy if the centralized rate-limiting store becomes unavailable?