Katkat
Back

Review a PR for thread-safe request handling

J.P. MorganSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSoftware Engineering Fundamentals

Last updated

This problem tests your ability to identify race conditions and thread-safety issues in a concurrent service and suggests robust design patterns for handling shared resources.

Scenario

You are reviewing a pull request for a RateLimiterService designed to handle concurrent outgoing API requests for a high-frequency trading platform. The developer has implemented the following class:

import time
import threading

class RateLimiterService:
    def __init__(self, limit):
        self.limit = limit
        self.counter = 0
        self.last_reset = time.time()

    def should_allow_request(self):
        now = time.time()
        if now - self.last_reset > 1:
            self.counter = 0
            self.last_reset = now
        
        if self.counter < self.limit:
            self.counter += 1
            return True
        return False

The Task

  1. Identify the thread-safety vulnerabilities in the provided should_allow_request method.
  2. Explain why these vulnerabilities are problematic in a high-concurrency production environment.
  3. Propose a thread-safe refactored version of this class, ideally minimizing lock contention if possible.

Constraints & Expectations

  • Your solution should handle multiple threads calling should_allow_request simultaneously without exceeding the limit.
  • Provide an explanation of the trade-offs between using synchronization primitives (e.g., threading.Lock) versus atomic operations (e.g., threading.local or atomic counters).
  • Consider how the design handles drift or precision issues when multiple threads attempt to reset the counter at the exact same time.

Hints

Hint 1

Look for the 'check-then-act' race condition where multiple threads might pass the `counter < limit` check simultaneously.

Hint 2

Consider how the `last_reset` timestamp logic might lead to inconsistent states when multiple threads enter the reset block at the same time.

Hint 3

Evaluate whether a fine-grained lock is necessary or if an atomic increment operation would suffice for the counter logic.

Follow-up questions

  • How would your approach change if this service needed to be distributed across multiple instances (e.g., using Redis)?
  • What are the performance implications of 'Lock' contention in a system with thousands of requests per second?
  • How would you implement a 'backpressure' mechanism if the `should_allow_request` returns False?