Katkat
Back

Design a Tier-Aware AI Video Generation Service

OpenAISoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSystem Design

Last updated

This problem tests your ability to design a distributed, asynchronous task processing system that balances resource fairness, reliability in the face of compute failures, and tiered prioritization.

Problem Statement

Design a video generation service that converts user-provided prompts into video files using an AI model. The system must support two tiers: 'Premium' (high priority) and 'Free' (lower priority). Video generation is resource-intensive, takes several minutes, and is prone to GPU-related failures.

Requirements

  • Scalability: The system must handle thousands of concurrent requests while GPUs remain a scarce, finite resource.
  • Reliability: If a worker node crashes mid-generation, the task must be recoverable and re-queued without creating duplicate partial artifacts.
  • Tier-Awareness: Premium requests must be processed before Free requests, but the system must prevent 'starvation' of Free-tier users.
  • Idempotency: Users should be able to retry job submissions without triggering redundant expensive computations.

System Constraints

  • Assume an asynchronous request-response pattern (client polls or receives a webhook).
  • Assume a fleet of GPU workers that are ephemeral and frequently crash.
  • You have access to a distributed object store (S3-like) and a key-value store (Redis-like) or database.

Deliverables

  1. High-level architecture diagram or component list.
  2. Proposed database schema for managing job states (e.g., Pending, Processing, Completed, Failed).
  3. Strategies for managing the worker lease mechanism and handling partial video artifacts.
  4. Design for the scheduling logic to ensure fairness across tiers.

Hints

Hint 1

Use a message queue with visibility timeouts to handle the worker heartbeat and task re-queuing upon failure.

Hint 2

Implement a 'Claim-Check' pattern where the worker writes the final output to persistent storage and updates the database record atomically.

Hint 3

Introduce a 'Priority Queue' strategy where workers look at high-tier queues first, but periodically poll lower-tier queues to ensure no job is indefinitely delayed.

Follow-up questions

  • How would you handle a sudden 10x spike in traffic to prevent the entire system from collapsing?
  • If a user wants to cancel a video generation request that is already mid-processing, how would you orchestrate that signal?
  • How do you ensure that the cost of storing generated videos does not become a bottleneck for your system architecture?