Katkat
Back

Find the Production Bugs in a Dictionary Cache Around an Expensive Computation

MercorSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSoftware Engineering Fundamentals

Last updated

Review a short Python function that caches the results of an expensive computation in a dictionary, and identify what breaks in a long-running, concurrent production service. Tests reasoning about memory growth, eviction policy, concurrent cache misses, error handling, and input validation.

The interviewer shows you a short piece of code that caches the results of an expensive function, and asks you to find the problems with it. The original snippet is not preserved. The version below reconstructs the same pattern, an in-process dictionary cache in front of a function named expensive_computation, shown here in Python, and it is the code you should review.

Assume it runs inside a long-lived, multi-threaded web service in which many requests call get_result concurrently.

_cache = {}

def get_result(key):
 if key in _cache:
 return _cache[key]
 result = expensive_computation(key)
 _cache[key] = result
 return result

List every problem you can find, explain why each one matters in production, and propose a fix.

Constraints and Clarifications

  • expensive_computation is slow compared with the rest of the request, and it can fail.
  • You may change the function's internals, but callers must keep calling get_result(key).

Clarifying Questions

  • How many distinct keys exist, and how large is each result?
  • Can the result for a given key change over time, so that cached entries need a time-to-live?
  • Does expensive_computation signal failure by raising an exception, by returning an error value, or both? Are its failures transient?
  • Is the service one process with many threads, several processes, or several machines?

Hints

Hint 1

Run it for a month: Picture this function after weeks inside a busy service that sees many distinct keys. What happens to `_cache`?

Hint 2

Mind the gap: Walk through what each concurrent caller does between the membership check and the assignment, and how long that gap can last.

Hint 3

Beyond the happy path: Ask what `expensive_computation` can do other than return a good value, and what kinds of `key` callers might pass in.

Follow-up questions

  • Implement a bounded least-recently-used version and state the time complexity of each operation.
  • How would you share this cache across several service instances, and what new failure modes appear?
  • When is caching a failure (negative caching) the right choice, and for how long?