Solve Throughput, Rate Limiting, and LFU
MicrosoftSoftware Engineer · OnsiteLast updated
This problem tests your ability to design a system with O(1) time complexity for cache operations and time-window-based request tracking.
Problem Statement
You are tasked with building a high-performance request handling component for a web service. The component must support three primary operations:
- LFU Cache (Least Frequently Used): Design a cache that stores key-value pairs. When the cache reaches capacity, it must evict the entry with the lowest frequency. If there is a tie, evict the least recently used entry among those with the same frequency. All operations (
getandput) must run inO(1)time. - Fixed Window Rate Limiting: Implement a method that determines if a specific user ID is allowed to make a request, given a limit of
Nrequests perTseconds. - Throughput Calculation: Given a log of request timestamps as an array of integers, calculate the maximum number of requests processed in any rolling window of size
Wmilliseconds.
Implementation Signature
class RequestManager:
def __init__(self, capacity: int, limit: int, window_size: int):
# capacity: for LFU cache
# limit: max requests per window
# window_size: duration for rate limiting and throughput analysis
pass
def get_cache(self, key: int) -> int: ...
def put_cache(self, key: int, value: int) -> None: ...
def is_allowed(self, user_id: int, current_time: int) -> bool: ...
def max_throughput(self, timestamps: list[int]) -> int: ...
Constraints
capacity,limit,window_sizeare positive integers up to 10^5.timestampsis sorted in non-decreasing order.
Example
- LFU: If capacity is 2,
put(1,1),put(2,2),get(1),put(3,3), key 2 should be evicted because it was used 0 times (or less than 1). - Throughput:
timestamps = [100, 200, 300, 500, 600],window_size = 250. The window [300, 550] contains 2 requests, but [100, 300] contains 3. Max is 3.
Hints
Hint 1
To achieve O(1) for LFU, use a combination of a hash map for key storage and a doubly-linked list or a map of linked lists to track frequencies.
Hint 2
For rate limiting, consider using a dictionary to map user IDs to a deque of timestamps, clearing out entries older than the current time minus the window size.
Hint 3
For the sliding window throughput, use a two-pointer approach to maintain the window boundaries within the sorted timestamp array.
Follow-up questions
- How would you adapt the LFU implementation to handle multi-threaded access while minimizing lock contention?
- If the request timestamps are not perfectly sorted due to network jitter, how would your throughput calculation change?
- How does a sliding window rate limiter differ from a token bucket algorithm in terms of burstiness and memory usage?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.