Katkat

Design a Resumable Iterator with Checkpoint and Restore

OpenAIMachine Learning Engineer · Technical Screen

Last updated

This question evaluates a candidate's ability to design serializable, stateful abstractions in both in-memory and I/O-bound contexts. It tests understanding of iterator patterns, checkpoint-restore semantics, and test-driven development — competencies commonly assessed in engineering roles requiring systems thinking and API design rigor.

Design a resumable iterator: an iterator that can serialize its current position into a compact, opaque checkpoint and later be reconstructed from that checkpoint so that iteration continues from exactly where it left off — even in a brand-new process.

You will build this in two stages, and you are expected to write the tests first, then implement against them. Your test suite must be detailed enough that a correct implementation passes immediately on the first run.

Stage 1 — List iterator

Implement ResumableListIterator over an in-memory sequence.

It must support:

  • has_next() -> bool — whether another element remains.
  • next() -> T — return the next element and advance; raise StopIteration (or your language's equivalent) if none remain.
  • save_state() -> State — return a checkpoint capturing the current position. The checkpoint must be a value that can be stored and passed around (e.g. an integer, a string, or a small serializable object). It must not be a live reference to the iterator's internal mutable state.
  • A constructor / factory that builds an iterator from (source, state) so that resuming from a saved checkpoint yields exactly the elements that had not yet been returned at the moment save_state() was called.

Resume semantics, precisely:

  • If you call save_state() after returning the first k of n elements, then build a new iterator from the same source plus that checkpoint, the new iterator must return elements k, k+1, ..., n-1 (0-indexed) in order and then report no more.
  • Saving at the very start (before any next()) resumes from the first element. Saving after the last element resumes into an empty iterator (has_next() is false).
  • save_state() is a read-only operation: calling it must not advance the iterator or change what the original iterator yields next.

Stage 2 — File iterator (follow-up)

Implement ResumableFileIterator that yields the lines of a text file one at a time, with the same has_next / next / save_state / resume-from-state contract as Stage 1.

Additional requirements specific to files:

  • The checkpoint must let a fresh iterator (in a new process, with the file reopened) resume at the next unread line without re-reading or re-emitting any line already returned.
  • Do not require loading the whole file into memory; the iterator must stream so that it works on a file far larger than available RAM. Your checkpoint should therefore be small and must not embed the file's contents.
  • State the assumptions your checkpoint relies on (for example, that the underlying file is not modified between save and resume), and design the checkpoint so that resuming lands on a well-defined line boundary rather than mid-line.

Required output / deliverables

  1. A test suite covering at minimum: resume from the start, from the middle, from the end (empty resume), and a round trip where you iterate part-way, checkpoint, reconstruct, and assert the remaining sequence equals the expected tail. Include a test asserting save_state() does not mutate the original iterator.
  2. The ResumableListIterator implementation passing those tests.
  3. The ResumableFileIterator implementation passing line-based versions of the same tests, plus a test that resume continues at the correct line boundary on a multi-line file.

Constraints & Assumptions

  • Elements/lines are returned in their original order; no skipping, no duplication across a save/resume boundary.
  • A checkpoint produced by save_state() is opaque to callers and portable: it can be serialized (e.g. to JSON or a primitive) and used to reconstruct the iterator later, possibly in a different process.
  • For the file iterator, assume the file is encoded in a single, known text encoding (e.g. UTF-8) and is not concurrently modified between checkpoint and resume.
  • Aim for $O(1)$ additional memory per next() for the file iterator (streaming), and a checkpoint whose size is independent of how many elements have already been consumed.