Katkat

Design an Ad Pacing System

NetflixSoftware Engineer · Onsite

Last updated

This question evaluates a candidate's skill in designing large-scale, low-latency distributed advertising systems, including pacing algorithms, budgeting and spend tracking, concurrency control, event processing, and observability.

Design an advertising pacing system for a large-scale video / streaming advertising platform.

Advertisers create campaigns with budgets, flight dates (start/end), targeting constraints, and delivery goals such as impressions, clicks, or spend. The platform receives ad opportunities (ad requests) in real time and, for each one, must decide which campaigns are eligible to serve — while pacing delivery smoothly over the campaign lifetime rather than exhausting the budget early.

Your design should address how to:

  • Spread campaign spend / impressions over time instead of front-loading the budget.
  • Handle real-time ad requests with low latency.
  • Track spend, impressions, and remaining budget accurately.
  • Re-pace when traffic volume changes (diurnal cycles, spikes, dips).
  • Prevent overspending across many concurrent serving nodes.
  • Support nested budget scopes: campaign-level, ad-group-level, and daily budgets.

Cover the data model, APIs, services, storage, streaming pipelines, and monitoring. Make the separation between the fast serving path and the slower pacing control loop explicit, and walk through requirements, end-to-end architecture, the pacing control loop, the real-time serving path, overspend prevention, event handling, scaling, and failure modes.

Constraints & Assumptions

State your assumptions explicitly; a reasonable starting point:

  • Scale: ~1M+ ad requests/sec at peak across the fleet; hundreds of thousands of active campaigns; events (impressions/clicks/spend) at a similar rate to requests.
  • Latency: the per-request ad-decision budget is tight (single-digit to low tens of milliseconds, p99). Pacing and budget checks happen inside this budget.
  • Accuracy: hard budget overspend must be bounded (e.g. within a small single-digit percent worst case, ideally tighter); billing/spend ultimately reconciles against an authoritative ledger.
  • Traffic is non-uniform: opportunity volume varies by hour, day, region, and content — uniform "linear" pacing alone under- or over-delivers.
  • Events can be late, out-of-order, or duplicated (client retries, network, replay).

Clarifying Questions to Ask

  • What is the delivery goal we're pacing to — spend, impressions, clicks, or conversions — and can a single campaign mix goals?
  • Is serving auction-based (we set a bid into a marketplace) or a direct/owned-inventory decision? This changes whether bid multipliers are even a lever.
  • What overspend tolerance is acceptable, and is the budget a hard cap or a soft target with reconciliation?
  • What's the relationship between the budget scopes (does an ad-group budget sum up to the campaign budget, or are they independent caps)?
  • What are the consistency requirements on spend tracking — is eventually-consistent accounting with later reconciliation acceptable, or must it be strongly consistent?
  • What's the expected number of candidate campaigns per request, and the p99 latency SLO for the ad decision?

Hints

Hint 1

Where to start: The request path and the budgeting logic don't have the same latency budget. Think about which decisions *must* happen inside the per-request deadline and which can run on a slower cadence, and what state each side reads vs. writes.

Hint 2

Pacing signal: Pacing is a feedback loop: define some notion of where a campaign's spend *should* be by now and compare it to where it actually is, then act on the gap. Ask yourself what makes that "should" target wrong when traffic isn't uniform over the flight, and what dials the serving path can actually turn in response.

Hint 3

Overspend under concurrency: The dangerous case is many serving nodes deciding to spend the same campaign's budget at the same instant. Think about whether you really need a synchronous global check on every impression, or whether nodes can be handed something to spend locally — and what that choice does to your worst-case overspend.

Hint 4

Nested budgets & late events: One serve can count against several budgets at once (ad-group, daily, lifetime, …). Think about what eligibility means when multiple caps apply, and separately about how delayed, out-of-order, or replayed spend events could corrupt your counters — and what property of event processing protects against that.

Follow-up questions

  • A single campaign is responsible for a large fraction of total spend and concentrates on one hot region — how do you pace it without that one counter becoming a serving bottleneck?
  • The traffic forecast is badly wrong for the next few hours (e.g. an unexpected live-event traffic spike). How does the controller react, and what protects you from over-delivering before the next control tick?
  • Mid-flight, an advertiser cuts a campaign's daily budget in half. How quickly and safely does that propagate to every serving node, and what prevents a brief window of overspend?
  • How would you extend this to **frequency capping** (per-user impression limits) on top of pacing, and how do the two interact when deciding eligibility?