Katkat
Back

Design a Product Image Upload Pipeline for Sellers

AmazonSoftware Engineer · Onsite
MediumSoftware EngineerOnsiteSystem Design

Last updated

Design an asynchronous, distributed image processing and delivery pipeline for an e-commerce seller portal, focusing on direct-to-cloud uploads, state management, moderation workflows, and resilient global caching.

Scenario

In an e-commerce marketplace like Amazon, millions of third-party sellers manage product listings that require high-resolution imagery. Sellers need to upload, reorder, replace, and delete product images across multiple image slots (such as the primary MAIN image and supplementary gallery shots).

Design a reliable, scalable image processing and delivery pipeline that ingests raw seller uploads, validates and transforms them into standardized formats, performs automated moderation, and serves approved images globally to buyers.

Requirements

Functional

  • Direct Upload: Support secure, direct-to-storage upload for image files up to 25 MB without routing raw binaries through primary API gateways.
  • Processing & Moderation: Asynchronously validate image formats/dimensions, generate standardized variants (e.g., thumbnail 100x100, zoom 2000x2000 in WebP/AVIF), and run automated policy moderation (detecting watermarks, prohibited text, or offensive content).
  • Lifecycle & Ordering: Allow sellers to assign, replace, and reorder image slots for a product listing (ASIN). Unapproved or failing images must never become visible to buyers.
  • Delivery: Serve approved variants with low latency across global buyer regions.

Non-Functional & Scale

  • Write Scale: 50 million images uploaded or updated daily (~600 uploads/sec avg, ~2,500/sec peak).
  • Read Scale: 5 billion image views daily (~60,000 views/sec avg).
  • Processing SLA: 95% of uploaded images should be processed, moderated, and available within 30 seconds.
  • Reliability & Concurrency: Retry-safe and idempotent processing. Protect against race conditions when a seller rapidly updates or swaps images.

What Your Answer Should Cover

  1. System Architecture: High-level workflow from pre-signed upload URL generation to event-driven processing and persistent storage.
  2. API & Data Model: Key endpoints for initiating uploads and updating listing image order; schema modeling image state transitions (e.g., PENDING_UPLOAD, PROCESSING, APPROVED, REJECTED, ACTIVE).
  3. Concurrency & Idempotency: Mechanisms to handle duplicate callbacks, retries, and concurrent reordering requests from multiple seller sessions.
  4. CDN & Caching Strategy: Cache invalidation vs. immutable versioned URLs when sellers replace existing images.

Hints

Hint 1

Use pre-signed S3/Blob storage URLs so clients upload directly to object storage, triggering event notifications to your processing pipeline.

Hint 2

Decouple image files using content-addressable storage (SHA-256 hashes) from the product listing's logical slots to simplify deduplication and CDN caching.

Hint 3

Use optimistic concurrency control (version checks) on the product metadata record to safely handle race conditions during reordering and slot replacements.

Follow-up questions

  • How would you handle cache invalidation at the edge if you use stable URLs (e.g., `/products/{asin}/main.jpg`) versus content-hashed URLs?
  • If automated moderation rejects an image after intermediate derivatives have already been written to storage, how do you handle rollback and quarantine?
  • How would you prioritize processing queues during massive seller upload surges (e.g., Prime Day preparations) to ensure urgent updates are not delayed?