Katkat
Back

Design a Product Tagging Pipeline

SnapchatMachine Learning Engineer · Onsite
MediumMachine Learning EngineerOnsiteML System Design

Last updated

This problem tests your ability to design a scalable multi-modal machine learning pipeline that processes diverse content types to generate standardized taxonomy tags for an e-commerce platform.

Design a Product Tagging Pipeline

Snapchat is expanding its e-commerce capabilities. You are tasked with designing a system that automatically tags user-uploaded product content (images, videos, and titles) with labels from a predefined taxonomy (e.g., "Footwear", "Electronics", "Summer Seasonal").

Functional Requirements:

  • The system must ingest various media types: product images, short-form video clips, and text descriptions.
  • The output must be a ranked list of relevant tags from a taxonomy of approximately 5,000 labels.
  • Tags should be generated with high precision for search and recommendation purposes.

Non-Functional Requirements:

  • Latency: Initial tags should be suggested to the user within 500ms of upload.
  • Scale: The system must handle 10,000 requests per second during peak hours.
  • Consistency: Tags must be consistent across different media types (e.g., an image of a shoe and the word "sneakers" should yield similar results).

Design Constraints:

  • Assume a pre-existing infrastructure for media storage (S3-like) and message queues (Kafka-like).
  • You do not need to build the core model architecture from scratch, but you must describe how to utilize pre-trained models or embeddings.

Deliverables:

  1. Data Flow: Describe the pipeline from user upload to tag assignment.
  2. Model Strategy: Explain how to leverage multi-modal signals (Image-Text-Video) to resolve ambiguities.
  3. Evaluation: Define how you measure the performance of the system in production.
  4. Scalability: Explain how you would handle heavy traffic spikes.

Hints

Hint 1

Consider an embedding-based approach where text, images, and video frames are projected into a shared high-dimensional vector space.

Hint 2

Think about using a two-stage retrieval process: a fast candidate generation step followed by a more precise re-ranking model.

Hint 3

Address the cold-start problem for new products by emphasizing the use of available text metadata and visual features.

Follow-up questions

  • How would you handle a scenario where the taxonomy changes or expands to include new categories?
  • What strategies would you use to detect and mitigate bias in the tagging system, such as models consistently mislabeling products from certain demographic groups?
  • How would you implement an active learning loop to improve model performance using user-corrected tags?