Katkat
Back

Design Individual and Bulk Merchant Product Updates with Image Uploads

PinterestSoftware Engineer · Onsite
HardSoftware EngineerOnsiteSystem Design

Last updated

This problem tests the ability to design a distributed system capable of handling high-throughput asynchronous updates, atomic transactional integrity for partial batch failures, and robust image processing pipelines.

System Design: Merchant Catalog Management

Pinterest merchants frequently update their product catalogs. These updates can be single-item patches or bulk CSV/JSON uploads containing thousands of products. Each update may include high-resolution image URLs that must be processed, resized, and stored in a CDN.

Requirements

Functional Requirements:

  1. Atomic Updates: Support partial updates to existing products (e.g., change price, update stock).
  2. Bulk Processing: Accept large file uploads. If a bulk upload fails partially (e.g., 500 items succeed, 100 fail), provide the merchant with a detailed report of failures.
  3. Image Pipeline: Validate, resize, and persist images from external URLs provided in the product metadata.
  4. Status Tracking: Provide an API for merchants to check the status of a bulk job.

Non-Functional Requirements:

  1. Scalability: Handle spikes in catalog updates during peak retail seasons.
  2. Consistency: Ensure product inventory data remains consistent across search and discovery feeds.
  3. Reliability: Implement retry mechanisms for transient network failures during image fetching.

The Task

Design a system architecture to handle these requirements. Provide:

  • API Design: Define the endpoints for single updates and bulk job submission.
  • Data Model: How to store the product metadata and the state of bulk jobs.
  • Processing Pipeline: Describe how you move from receiving a file to updating the merchant's live catalog.
  • Concurrency: How to handle multiple updates to the same product ID happening concurrently.

Hints

Hint 1

Consider using an asynchronous task queue or message broker (like Kafka or RabbitMQ) to decouple the ingestion API from the heavy lifting of image processing.

Hint 2

For bulk jobs, implement a state machine that tracks progress; treat each product update within the batch as an idempotent operation to allow safe retries.

Hint 3

To handle concurrency, utilize optimistic locking or a versioning field in your database to ensure that older updates do not overwrite newer ones.

Follow-up questions

  • How would you handle a scenario where a merchant uploads a catalog with 100,000 items, and the image processing server crashes mid-job?
  • If a merchant updates the same product price across two different bulk jobs submitted simultaneously, how do you resolve the final state?
  • How would you implement rate limiting to protect the system from malicious merchants uploading massive files or external sites being overwhelmed by image-fetch requests?