Katkat
Back

Python Generators, yield, Lazy Evaluation and How Decorators Work

EngelhartSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSoftware Engineering Fundamentals

Last updated

A Python interview question on generators and decorators. It asks what a generator is, what yield does, why generators use less memory and what lazy evaluation means, then what a decorator actually does under the hood, testing precise language mechanics rather than memorized definitions.

Explain what generators and decorators are in Python and why you would use them. These came as a quick oral sequence in the fundamentals half of a technical screen for a junior quantitative developer role, and each answer drew a follow-up on the mechanism underneath.

squares_list = [x * x for x in range(10_000_000)]
squares_gen = (x * x for x in range(10_000_000))

@timed
def load_prices(path): ...

Clarifying Questions

  • When comparing a generator with an "ordinary iterator", is that a hand-written class with __iter__ and __next__, or a sequence such as a list?
  • For decorators, should the answer include decorators that take arguments, or only the basic form?
  • Should the memory discussion include approximate sizes, or only the reason?

Part 1 — Generators, yield and lazy evaluation

  1. What is a generator?
  2. What advantages does a generator have over an ordinary iterator or a sequence?
  3. What does yield do?
  4. Why is a generator more memory efficient?
  5. What is lazy evaluation?

What This Part Should Cover

  • A generator as a function whose execution is suspended and resumed, and the exact role of yield
  • How it satisfies the iterator protocol with far less code than a hand-written iterator class
  • Memory use and lazy evaluation, with a concrete use such as streaming a large file
  • The limitations: a single pass, no length or indexing, deferred errors

Part 2 — Decorators

  1. Have you used decorators? Give an example.
  2. What is a decorator really doing?

What This Part Should Cover

  • What the @ syntax is equivalent to, and when the decorator runs compared with the wrapped function
  • A correctly written wrapper that passes arguments and the return value through
  • Preserving the wrapped function's metadata, and decorators that take arguments
  • Real examples from the standard library or your own code

Hints

Hint 1

Where the function pauses: Call `next()` on a generator twice and track what happens to the function's local variables in between. Then count how many values exist in memory at once, compared with the list.

Hint 2

Remove the @ sign: Rewrite `@timed` placed above `def load_prices` as an ordinary assignment statement, and ask what object the name `load_prices` refers to afterwards.

Follow-up questions

  • You loop over the same generator twice. What does the second loop produce, and how do you handle a computation that needs two passes?
  • Write a decorator that retries a function up to a given number of times when it raises a given exception. Where do those settings go?
  • A function opens a file with `with` and returns a generator expression over its lines. What goes wrong when the caller iterates?
  • How would you chain generators to read a very large CSV file, filter its rows and keep a running average without loading the whole file?