Scale a Thumbnail-and-Watermark Pipeline With Worker Processes

A photo-sharing backend needs to turn every freshly uploaded photo into a thumbnail stamped with a small corner watermark, for use in feeds and previews. Start with the single-photo operation, then scale it across a batch using multiple worker processes.

Constraints and assumptions

• Each input photo comes with an explicit job spec: source location, target thumbnail dimensions, a watermark intensity value, and a destination location. Nothing about size, intensity, or destination is implicit or hardcoded.
• Photos are independent of each other, but disk I/O and memory are shared resources across the whole batch.
• One failing or corrupt photo must never take down the batch; it should be reported and the rest should keep going.
• For this exercise, represent an image as a 2D grid of integer pixel values rather than depending on an external imaging library, so the pipeline is self-contained and easy to test; the same design carries over directly to a real library like Pillow.

Part 1 — A correct single-photo operation

Write a function that takes one job spec, reads the source image, resizes it to the target dimensions, stamps the watermark into its bottom-right corner, writes the result to the destination, and returns a result or error record — never raises past its own boundary.

def process_image(spec: dict) -> dict:
    """spec has keys: id, src_path, dst_path, target_size (w, h), watermark_value.
    Returns {'id': ..., 'status': 'ok'} or {'id': ..., 'status': 'error', 'error': str}."""

Think about what happens if the pipeline crashes mid-write: should a half-written output file ever be mistaken for a finished one?

Part 2 — Scale it across worker processes

def run_pipeline(specs: list[dict], max_workers: int) -> list[dict]:
    """Process every spec, bounded to at most max_workers processes in flight, and
    return one result/error record per spec, regardless of individual failures."""

Decide what crosses the process boundary to each worker, and why that should be small.

What this should cover

• Explicit transform order (resize, then watermark) and no hidden default dimensions.
• Output publication that can't be confused with a truncated, in-progress file.
• A bounded worker pool, not one process per photo.
• Small, serializable job descriptions passed to workers, rather than shipping decoded pixel buffers through interprocess communication.
• One photo's failure producing an error record, not an exception that kills the batch.