Implement idempotent request handling with idempotency keys
MicrosoftSoftware Engineer · OnsiteLast updated
This problem tests your ability to design a thread-safe idempotency layer that ensures an API endpoint processes identical requests exactly once, even in the presence of concurrent execution.
Problem Statement
You are building a middleware component for a payment service. To ensure reliability, clients attach a unique string idempotency_key to their POST requests. Your goal is to implement an IdempotencyManager class that ensures a specific operation is executed at most once per key.
Requirements
execute(key, task): If thekeyhas not been seen, execute thetask(a callable) and store its result. If thekeyhas been seen, return the previously stored result without re-executing the task.- Concurrency: Multiple threads may call
executewith the same key simultaneously. Only one thread should execute the task, while others should block until the result is available. - TTL (Time-to-Live): Store results for a maximum duration (e.g., 10 minutes) to free up memory.
Class Interface
class IdempotencyManager:
def __init__(self, ttl_seconds: int):
# Initialize storage and TTL settings
def execute(self, key: str, task: Callable) -> Any:
# Returns the result of the task execution
Constraints
- The system must be thread-safe.
- Use
threadingprimitives (e.g., Locks, Conditions). - Consider the memory footprint of storing results.
Example
# Setup
manager = IdempotencyManager(ttl_seconds=600)
# Execution
result1 = manager.execute("tx_123", lambda: expensive_compute())
result2 = manager.execute("tx_123", lambda: expensive_compute())
# result2 should be identical to result1, and expensive_compute()
# should have been called exactly once.
Edge Cases
- What happens if the
taskfails or raises an exception during the first execution? - How do you handle rapid cleanup of expired keys to prevent memory leaks?
Hints
Hint 1
Use a dictionary to store result states, where each entry is protected by a lock or a condition variable.
Hint 2
Represent the state of a key as either 'Pending', 'Completed', or 'Failed' to differentiate between ongoing work and finished results.
Hint 3
Implement a cleanup mechanism using a background thread or a periodic check to remove expired entries based on their insertion timestamp.
Follow-up questions
- How would you adapt this design to work across multiple distributed server instances instead of a single process?
- If the task takes a long time, how can you prevent a 'thundering herd' where many threads wait on the same key result simultaneously?
- How would you handle a situation where a client sends a second request with a different payload but the same idempotency key?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.