Process One Million Quote Updates per Second

Scenario

You are building a core market-data engine for an automated proprietary trading desk. The system connects to multiple equity exchanges (e.g., Venue A, Venue B, Venue C) that publish top-of-book quotes (Best Bid and Offer / BBO) for a universe of traded instruments. When an exchange's bid exceeds another exchange's ask for the same instrument, a cross-exchange arbitrage condition exists, and the trading system must detect it immediately.

Design a low-latency, in-process software component that consumes these updates, maintains the current extrema (global highest bid and lowest ask across venues), and dispatches execution signals with sub-microsecond latency.

Requirements

Functional Requirements

• Quote Ingestion: Ingest top-of-book updates QuoteUpdate(instrument_id, venue_id, bid_price, bid_size, ask_price, ask_size, timestamp_ns).
• Extrema Tracking: Maintain the per-venue state and compute the global Best Bid (max(bid_price)) and Best Ask (min(ask_price)) per instrument in real time.
• Arbitrage Detection: Detect if global_best_bid.price > global_best_ask.price across different venues, and emit an ArbitrageOpportunity event containing the buy venue, sell venue, spread, and max executable volume.

Non-Functional Requirements & Scale

• Throughput: Sustained 1,000,000 quote updates per second, with burst capacity up to 3,000,000 updates per second.
• Latency: P99 processing latency under 5 microseconds from ingestion to signal emission; zero dynamic heap allocations on the critical path.
• Scale: Up to 10,000 distinct instruments across up to 16 distinct exchanges.
• Concurrency & Correctness: Safe under multi-threaded ingestion where multiple exchange network threads push updates concurrently.

Key Areas to Cover

1. Data Structures & Memory Layout: Detail how per-instrument and per-venue state is represented. Explain preallocation, cache-line alignment, and how cache invalidation is avoided.
2. Concurrency & Threading Model: Choose between shared-state locking, lock-free primitives, or single-writer/actor-per-instrument models (e.g., partitioning instruments across worker threads via core pinning and ring buffers).
3. Extrema Update Algorithm: How to efficiently update the best bid/ask when an incoming quote either improves the extrema or degrades an existing top-of-book quote.
4. Failure Modes & Trade-offs: Handle stale quotes, sequence reordering, and the trade-off between lock-free synchronization vs. thread-pinned sharding.