Katkat

Implement idempotent request handling with idempotency keys

MicrosoftSoftware Engineer · Onsite

Last updated

This problem tests your ability to design a thread-safe idempotency layer that ensures an API endpoint processes identical requests exactly once, even in the presence of concurrent execution.

Problem Statement

You are building a middleware component for a payment service. To ensure reliability, clients attach a unique string idempotency_key to their POST requests. Your goal is to implement an IdempotencyManager class that ensures a specific operation is executed at most once per key.

Requirements

  1. execute(key, task): If the key has not been seen, execute the task (a callable) and store its result. If the key has been seen, return the previously stored result without re-executing the task.
  2. Concurrency: Multiple threads may call execute with the same key simultaneously. Only one thread should execute the task, while others should block until the result is available.
  3. TTL (Time-to-Live): Store results for a maximum duration (e.g., 10 minutes) to free up memory.

Class Interface

class IdempotencyManager:
    def __init__(self, ttl_seconds: int):
        # Initialize storage and TTL settings

    def execute(self, key: str, task: Callable) -> Any:
        # Returns the result of the task execution

Constraints

  • The system must be thread-safe.
  • Use threading primitives (e.g., Locks, Conditions).
  • Consider the memory footprint of storing results.

Example

# Setup
manager = IdempotencyManager(ttl_seconds=600)

# Execution
result1 = manager.execute("tx_123", lambda: expensive_compute())
result2 = manager.execute("tx_123", lambda: expensive_compute())

# result2 should be identical to result1, and expensive_compute() 
# should have been called exactly once.

Edge Cases

  • What happens if the task fails or raises an exception during the first execution?
  • How do you handle rapid cleanup of expired keys to prevent memory leaks?

Hints

Hint 1

Use a dictionary to store result states, where each entry is protected by a lock or a condition variable.

Hint 2

Represent the state of a key as either 'Pending', 'Completed', or 'Failed' to differentiate between ongoing work and finished results.

Hint 3

Implement a cleanup mechanism using a background thread or a periodic check to remove expired entries based on their insertion timestamp.

Follow-up questions

  • How would you adapt this design to work across multiple distributed server instances instead of a single process?
  • If the task takes a long time, how can you prevent a 'thundering herd' where many threads wait on the same key result simultaneously?
  • How would you handle a situation where a client sends a second request with a different payload but the same idempotency key?