Katkat
Back

Design Multi-Dimensional Request Rate Limiting

RobloxSoftware Engineer · Technical Screen
EasySoftware EngineerTechnical ScreenSystem Design

Last updated

Design a distributed, low-latency rate limiting service capable of enforcing multi-dimensional limits (e.g., by user ID, IP address, and endpoint) across a large-scale gaming platform.

Scenario

At a large-scale multiplayer platform, public APIs and internal microservices face heavy traffic, varying from normal user interactions to bot attacks and runaway client scripts. You need to design a distributed rate-limiting system that inspects requests and enforces multi-dimensional quotas before requests reach upstream backend services.

"Multi-dimensional" means a single request may need to satisfy multiple rules simultaneously, such as:

  • Max 120 requests/minute per authenticated user_id.
  • Max 600 requests/minute per origin ip_address.
  • Max 10 requests/second per (user_id, endpoint) pair for sensitive endpoints (e.g., item purchases or matchmaking requests).

Requirements

Functional Requirements

  • Rule Evaluation: Check an incoming request against all matching rule dimensions and return a boolean decision (allow or deny).
  • Quota Metadata: Return standard rate-limiting headers in the response (e.g., X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset).
  • Dynamic Configuration: Allow operators to register, update, and deprecate rate limit rules without restarting the service.

Non-Functional Requirements

  • Low Latency: Limit check processing should add under 3 ms p99 overhead to request processing.
  • High Availability: 99.99% availability. If the rate limiter experiences partial failures, the system should support configurable fail-open or fail-closed behavior.
  • Scale: Handle 500,000 incoming requests per second distributed globally.

What Your Design Should Cover

  1. Rate Limiting Algorithm: Compare algorithms (e.g., Token Bucket, Leaky Bucket, Sliding Window Counter) and justify your choice for multi-dimensional evaluation.
  2. Architecture & Integration: Explain where the rate limiter sits (API Gateway filter, sidecar, or centralized service) and how it handles rule matching.
  3. Data Model & Storage: Define the key schema, storage tier (e.g., Redis Cluster, in-memory cache), and atomic operations needed to track counts.
  4. Trade-offs & Edge Cases: Address race conditions, network partition behavior, and strategies to prevent hot keys during high-traffic events.

Hints

Hint 1

Consider composing hierarchical cache keys such as `rl:{rule_id}:{dimension_value}` to store counts independently for each dimension.

Hint 2

A sliding window counter implemented via Redis hashes or Lua scripts provides a good balance between memory efficiency and burst accuracy.

Hint 3

To meet strict latency targets under heavy traffic, explore local in-memory batching or token reservation at the edge proxy layer.

Follow-up questions

  • If the centralized Redis cluster becomes temporarily unreachable, how should your system balance service protection with user experience?
  • How would you optimize Redis round-trips when a single incoming request matches four different rate limiting dimensions?
  • How can you prevent a sudden spike of traffic from a single celebrity user or viral game event from creating a hot partition in your caching layer?