Debug Python LRU Cache-Key Construction
AnthropicSoftware Engineer · OnsiteLast updated
Implement and debug a thread-safe, bounded LRU memoization decorator that generates canonical cache keys across equivalent argument patterns, distinguishes conflicting types, and avoids caching exceptions.
Problem Statement
Standard naive memoization decorators often construct cache keys directly from (args, tuple(sorted(kwargs.items()))). This creates multiple failure modes:
- Calling
fn(1, 2)versusfn(a=1, b=2)produces different keys despite invoking identical semantics. - Omitting an argument that takes its default value produces a different key than passing the default explicitly.
- Python's default hashing considers
True == 1and1 == 1.0, causing unintentional cache collisions across distinct types. - Passing unhashable nested structures (e.g.,
dict,list,set) raisesTypeErrorunless normalized into an immutable, type-tagged form. - Unhandled exceptions raised by the wrapped function must not be cached, and locks must never remain held on failure.
Implement a decorator canonical_lru(maxsize=128) that wraps a Python function with an LRU cache addressing these issues.
Requirements
- Canonical Signature Binding: Use
inspect.signatureto bind incoming*argsand**kwargs, apply parameter defaults, and produce an identical parameter mapping regardless of whether arguments are provided positionally or by keyword. - Type-Distinguishing Key Serialization: Recursively freeze arguments into a hashable representation. Keys must distinguish:
int,float, andbool(e.g.,1,1.0, andTruemust yield distinct sub-keys).listvstuple(e.g.,[1, 2]vs(1, 2)must not collide).dictkeys and values (order-independent for dict keys).set/frozensetelements (order-independent).
- LRU Eviction: Maintain at most
maxsizeentries. Accessing an existing entry marks it as most recently used. When capacity is exceeded, evict the least recently used item. Ifmaxsize <= 0, cache nothing. - Exception Safety & Concurrency: Ensure thread-safe cache operations using a lock. If the wrapped function raises an exception, do not record a cache entry and ensure the lock is properly released.
Interface
import inspect
import threading
from collections import OrderedDict
from functools import wraps
from typing import Any, Callable
def canonical_lru(maxsize: int = 128) -> Callable:
"""Decorator that caches function calls with canonical argument keys."""
...
Example
calls = 0
@canonical_lru(maxsize=4)
def process(a, b=10, *, flags=None):
global calls
calls += 1
return (a, b, flags)
# All three calls must hit the same cache entry:
process(1) # calls = 1 (Cache Miss)
process(1, 10) # calls = 1 (Cache Hit)
process(b=10, a=1, flags=None) # calls = 1 (Cache Hit)
# Type distinctions must not collide:
process(1, 10.0) # calls = 2 (Cache Miss, int vs float)
process(True, 10) # calls = 3 (Cache Miss, bool vs int)
Hints
Hint 1
Use inspect.signature(fn).bind(*args, **kwargs) followed by bound_args.apply_defaults() to obtain a consistent mapping of all parameter names to values.
Hint 2
To prevent type collisions such as 1 vs 1.0 or list vs tuple, tag each value with its exact type (e.g., (type(v), freeze(v))) when recursively freezing collections.
Hint 3
Use an OrderedDict protected by a threading.Lock, and invoke the target function outside of or prior to acquiring the lock if you want to avoid holding the lock during slow computations.
Follow-up questions
- How does your implementation handle functions that accept *args (var-positional) or **kwargs (var-keyword)?
- Holding a single lock across cache lookup, execution, and insertion can cause high contention; how would you optimize concurrency if function execution is slow?
- How would you handle cyclic references in nested data structures passed as arguments?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.