Implement a Lazy Tee Iterator in Python
GoogleSoftware Engineer · Technical ScreenLast updated
This problem tests your understanding of iterator protocols, generator patterns, and memory-efficient data buffering for multi-consumer streams.
Lazy Tee Iterator
Implement a function lazy_tee(iterable, n=2) that takes an input iterable and returns a tuple of n independent iterators. Each returned iterator must provide access to all items from the source iterable.
Your implementation must be lazy: if one consumer stops iterating, the source should not be advanced beyond what is required by the other consumers. Because iterators consume data linearly, you must implement a buffering mechanism to store items that have been read from the source but not yet consumed by all n iterators.
Requirements
- The source iterator should only be advanced when necessary.
- Memory usage should be proportional to the "lag" between the fastest and slowest consumer, not the total size of the iterable.
- Once an item has been consumed by all
niterators, it should be removed from the buffer to allow for garbage collection.
Function Signature
def lazy_tee(iterable, n=2):
# Returns a tuple of n independent iterators
pass
Examples
# Basic usage
it = iter([1, 2, 3])
a, b = lazy_tee(it)
print(list(a)) # [1, 2, 3]
print(list(b)) # [1, 2, 3]
# Uneven consumption
it = iter([1, 2, 3, 4])
a, b = lazy_tee(it)
next(a)
print(list(b)) # [1, 2, 3, 4]
print(next(a)) # 2
Edge Cases
- The input iterable might be empty.
- One of the returned iterators might never be used.
- One of the returned iterators might consume the entire stream before the others are even started.
Hints
Hint 1
Use a shared central list or deque to store items as they are pulled from the underlying source.
Hint 2
Maintain a list of pointers or indices, one for each consumer, to track their current position in the shared buffer.
Hint 3
Periodically prune the start of the buffer by checking if the slowest consumer has moved past the oldest stored item.
Follow-up questions
- How does your implementation handle a scenario where one iterator is abandoned (garbage collected) while the others are still active?
- What are the performance implications if the input iterable is a generator that produces items at high latency?
- Can you extend this implementation to handle a dynamic number of consumers added at runtime?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.