Katkat

Implement a Lazy Tee Iterator in Python

GoogleSoftware Engineer · Technical Screen

Last updated

This problem tests your understanding of iterator protocols, generator patterns, and memory-efficient data buffering for multi-consumer streams.

Lazy Tee Iterator

Implement a function lazy_tee(iterable, n=2) that takes an input iterable and returns a tuple of n independent iterators. Each returned iterator must provide access to all items from the source iterable.

Your implementation must be lazy: if one consumer stops iterating, the source should not be advanced beyond what is required by the other consumers. Because iterators consume data linearly, you must implement a buffering mechanism to store items that have been read from the source but not yet consumed by all n iterators.

Requirements

  • The source iterator should only be advanced when necessary.
  • Memory usage should be proportional to the "lag" between the fastest and slowest consumer, not the total size of the iterable.
  • Once an item has been consumed by all n iterators, it should be removed from the buffer to allow for garbage collection.

Function Signature

def lazy_tee(iterable, n=2):
    # Returns a tuple of n independent iterators
    pass

Examples

# Basic usage
it = iter([1, 2, 3])
a, b = lazy_tee(it)
print(list(a))  # [1, 2, 3]
print(list(b))  # [1, 2, 3]

# Uneven consumption
it = iter([1, 2, 3, 4])
a, b = lazy_tee(it)
next(a)
print(list(b))  # [1, 2, 3, 4]
print(next(a))  # 2

Edge Cases

  • The input iterable might be empty.
  • One of the returned iterators might never be used.
  • One of the returned iterators might consume the entire stream before the others are even started.

Hints

Hint 1

Use a shared central list or deque to store items as they are pulled from the underlying source.

Hint 2

Maintain a list of pointers or indices, one for each consumer, to track their current position in the shared buffer.

Hint 3

Periodically prune the start of the buffer by checking if the slowest consumer has moved past the oldest stored item.

Follow-up questions

  • How does your implementation handle a scenario where one iterator is abandoned (garbage collected) while the others are still active?
  • What are the performance implications if the input iterable is a generator that produces items at high latency?
  • Can you extend this implementation to handle a dynamic number of consumers added at runtime?