Katkat

Debug Python LRU Cache-Key Construction

AnthropicSoftware Engineer · Onsite

Last updated

Implement and debug a thread-safe, bounded LRU memoization decorator that generates canonical cache keys across equivalent argument patterns, distinguishes conflicting types, and avoids caching exceptions.

Problem Statement

Standard naive memoization decorators often construct cache keys directly from (args, tuple(sorted(kwargs.items()))). This creates multiple failure modes:

  1. Calling fn(1, 2) versus fn(a=1, b=2) produces different keys despite invoking identical semantics.
  2. Omitting an argument that takes its default value produces a different key than passing the default explicitly.
  3. Python's default hashing considers True == 1 and 1 == 1.0, causing unintentional cache collisions across distinct types.
  4. Passing unhashable nested structures (e.g., dict, list, set) raises TypeError unless normalized into an immutable, type-tagged form.
  5. Unhandled exceptions raised by the wrapped function must not be cached, and locks must never remain held on failure.

Implement a decorator canonical_lru(maxsize=128) that wraps a Python function with an LRU cache addressing these issues.

Requirements

  1. Canonical Signature Binding: Use inspect.signature to bind incoming *args and **kwargs, apply parameter defaults, and produce an identical parameter mapping regardless of whether arguments are provided positionally or by keyword.
  2. Type-Distinguishing Key Serialization: Recursively freeze arguments into a hashable representation. Keys must distinguish:
    • int, float, and bool (e.g., 1, 1.0, and True must yield distinct sub-keys).
    • list vs tuple (e.g., [1, 2] vs (1, 2) must not collide).
    • dict keys and values (order-independent for dict keys).
    • set / frozenset elements (order-independent).
  3. LRU Eviction: Maintain at most maxsize entries. Accessing an existing entry marks it as most recently used. When capacity is exceeded, evict the least recently used item. If maxsize <= 0, cache nothing.
  4. Exception Safety & Concurrency: Ensure thread-safe cache operations using a lock. If the wrapped function raises an exception, do not record a cache entry and ensure the lock is properly released.

Interface

import inspect
import threading
from collections import OrderedDict
from functools import wraps
from typing import Any, Callable

def canonical_lru(maxsize: int = 128) -> Callable:
    """Decorator that caches function calls with canonical argument keys."""
    ...

Example

calls = 0

@canonical_lru(maxsize=4)
def process(a, b=10, *, flags=None):
    global calls
    calls += 1
    return (a, b, flags)

# All three calls must hit the same cache entry:
process(1)                       # calls = 1 (Cache Miss)
process(1, 10)                   # calls = 1 (Cache Hit)
process(b=10, a=1, flags=None)   # calls = 1 (Cache Hit)

# Type distinctions must not collide:
process(1, 10.0)                 # calls = 2 (Cache Miss, int vs float)
process(True, 10)                # calls = 3 (Cache Miss, bool vs int)

Hints

Hint 1

Use inspect.signature(fn).bind(*args, **kwargs) followed by bound_args.apply_defaults() to obtain a consistent mapping of all parameter names to values.

Hint 2

To prevent type collisions such as 1 vs 1.0 or list vs tuple, tag each value with its exact type (e.g., (type(v), freeze(v))) when recursively freezing collections.

Hint 3

Use an OrderedDict protected by a threading.Lock, and invoke the target function outside of or prior to acquiring the lock if you want to avoid holding the lock during slow computations.

Follow-up questions

  • How does your implementation handle functions that accept *args (var-positional) or **kwargs (var-keyword)?
  • Holding a single lock across cache lookup, execution, and insertion can cause high contention; how would you optimize concurrency if function execution is slow?
  • How would you handle cyclic references in nested data structures passed as arguments?