Katkat
Back

Design An Ad Creative Relationship Processing System

ByteDanceSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSystem Design

Last updated

This question tests your ability to design a distributed asynchronous processing pipeline that manages entity relationships, handles API retries, and maintains state consistency across multiple dependent components.

System Design: Ad Creative Relationship Processor

You are tasked with building a back-end system for an advertising platform that processes 'Creative Plans.' A Creative Plan consists of a set of video assets, target keywords, and corresponding ad copy.

Your system receives a payload containing these elements and must call an external Machine Learning Model API for each (Video, Keyword, Copy) triplet to generate a 'Relationship Score.' The external API is unreliable and periodically returns 5xx errors or timeouts.

Functional Requirements:

  1. Submission: Accept a batch request containing a Creative Plan ID and a list of N asset triplets.
  2. Processing: Orchestrate calls to the external model API for each triplet.
  3. Tracking: Provide an endpoint to check the status of a Creative Plan. A plan is considered 'Complete' only when all triplets have successfully received a score.
  4. Consistency: Ensure the system is idempotent; re-submitting the same Creative Plan should not trigger duplicate external API calls.

Non-Functional Requirements:

  • Reliability: Implement a robust retry mechanism for failed external API calls.
  • Scale: The system must handle up to 10,000 Creative Plans per hour, with each plan containing up to 50 triplets.
  • Observability: Track individual task status (Pending, In-Progress, Success, Failed).

Deliverables:

  • Define the database schema to store the relationships and track task state.
  • Describe the high-level architecture, specifically how you will handle asynchronous processing.
  • Explain how you manage retries and ensure that a plan's state correctly transitions to 'Complete'.

Hints

Hint 1

Consider using a state machine pattern to track the status of individual triplets and the parent plan.

Hint 2

A message queue (e.g., Kafka or RabbitMQ) can decouple the ingestion service from the worker services responsible for calling the external API.

Hint 3

Use a distributed lock or a unique request identifier in your database to ensure that retries and re-submissions do not lead to duplicate external processing.

Follow-up questions

  • How would you handle a scenario where the external API becomes chronically slow and causes a massive backlog in your message queue?
  • If the external model requires a specific order of processing (e.g., videos before keywords), how would you adjust the architecture?
  • How would you implement a 'dead-letter queue' for triplets that consistently fail after the maximum number of retry attempts?