Katkat

Batch-Apply Pillow Image Transforms, Then Parallelize: Processes vs Threads

AnthropicSoftware Engineer · Onsite

Last updated

A Python coding exercise that applies a given sequence of Pillow transforms to a folder of cat images and then asks you to cut the total processing time. It tests clean per-image code, reasoning about CPU-bound work and the GIL, choosing between processes and threads, and sizing a worker pool.

You are given a set of cat images and must apply a series of image transforms to each one. The interviewer fixes the image library, Pillow, and the starter material shows how to call each required transform in Pillow. A provided test harness runs your code over all the images, checks the results, and reports how long it took to process all of them.

For practice, treat the transforms as a fixed sequence of standard Pillow operations (for example a resize, a rotation and a grayscale conversion) applied in order to every image. In the interview, the exact list and the exact calls come from the starter material.

Constraints and Clarifications

  • Use Pillow for all image work; the call for each transform is given.
  • A faster version must produce exactly the same output images as the simple version, because the tests check the results.
  • The number of images, their sizes and the number of CPU cores on the test machine are not given. Ask, or state your assumptions.

Clarifying Questions

  • Is the same transform sequence applied to every image, or does each image have its own list?
  • Where should outputs be written, and how should they be named? May an output overwrite its input?
  • If one image is corrupt or cannot be opened, should the whole run fail, or should it continue and report the failure?
  • Roughly how many images are there, how large are they, and how many CPU cores does the test machine have?

Part 1 — Apply the transforms

Implement the processing: for every input image, open it, apply the transforms in order using the given Pillow calls, and save the result. Make the provided tests pass.

What This Part Should Cover

  • Correct use of the given Pillow calls in the required order, including image modes and formats on save
  • Resource handling: closing files and keeping only one image's data in memory at a time
  • A clean per-image unit of work and a driver over all inputs
  • Handling of unreadable or unusual images

Part 2 — Make it faster

The harness reports the total time. Reduce it. If you parallelize the work, be ready for two questions from the interviewer:

  • Why did you choose multiple processes rather than multiple threads, or the reverse?
  • If you use processes, how many workers do you start, and why that number?

Clarifying Questions for this Part

  • Are all the cores the operating system reports available to this process, or is it limited, for example by a container CPU quota?
  • Does the reported time include starting the workers?

What This Part Should Cover

  • Whether the workload is CPU-bound or I/O-bound, and how to check
  • The global interpreter lock and what it means for threads versus processes, including where Pillow does and does not release it
  • Reasoning for the worker count, confirmed by measurement
  • Overheads: process start-up, data sent between processes, batching of tasks, and memory per worker

Hints

Hint 1

Keep one image's work self-contained: Write the work for a single image, from opening the file to saving the result, as one unit that shares no state with the others. Testing, error handling and Part 2 all become easier.

Hint 2

Find where the time goes: Before choosing a tool, decide whether the per-image work is limited by the CPU or by waiting on the disk, and what that means for Python threads.

Hint 3

Sizing the pool: Think about how many workers the machine can actually run at the same moment, and what each extra worker costs in start-up time and memory.

Follow-up questions

  • The image set grows to millions of files spread across many machines. What changes?
  • Profiling shows most of the time is spent decoding and encoding JPEG files. What would you try?
  • On an 8-core machine, 8 workers turn out slower than 6. What could explain it?
  • How would you make the run resumable if it crashes halfway through?