Katkat
Back

Design low-latency trading infrastructure

OptiverSoftware Engineer · Onsite
HardSoftware EngineerOnsiteSystem Design

Last updated

Design an end-to-end low-latency market making and execution architecture split between an exchange-colocated environment and a remote operational office.

Scenario

You are tasked with designing the trading infrastructure for an automated market-making firm trading equity options and futures on a major exchange (e.g., CME or Eurex). The system must ingest real-time market data feeds, calculate fair values, evaluate risk limits, and emit passive and aggressive orders with minimal and deterministic tick-to-trade latency.

The deployment is split into two environments:

  1. Exchange Colocation Facility: Bare-metal servers hosted in the exchange data center running the core trading loop.
  2. Remote Office / Headquarters: Development, research, monitoring, manual risk controls, and post-trade processing.

Requirements

Functional

  • Market Data Feed Handler: Ingest multicast exchange data (e.g., direct market feeds via UDP Multicast) and rebuild the limit order book.
  • Pricing & Quoting Engine: Generate dynamic two-sided quotes and respond to market-moving events.
  • Pre-trade Risk Management: Enforce sub-microsecond risk checks (e.g., position limits, maximum notional per clip, price banding, message rate limits).
  • Order Routing & Execution: Send new orders, cancels, and replaces over the exchange protocol (e.g., binary protocol or FIX over TCP/IP) and handle execution reports.
  • Remote Control & Telemetry: Enable human traders in the remote office to adjust strategy parameters, toggle kill-switches, and monitor health/positions in real time without impacting the hot path.

Non-Functional

  • Tick-to-Trade Latency: Median tick-to-trade under 2 microseconds; 99.9th percentile under 5 microseconds.
  • Jitter & Determinism: Minimal tail latency, zero GC pauses, strict CPU core isolation.
  • Throughput: Peak processing of 5 million market data messages per second; order submission bursts up to 50,000 orders per second.
  • Fault Tolerance & Safety: Automatic failover or fail-safe (immediate quote cancellation on disconnect) to avoid rogue quoting.
  • Precision Time Synchronization: Sub-microsecond timestamping across all components for compliance and order book sequencing.

What Your Design Should Cover

  • Physical & Hardware Architecture: Kernel bypass (e.g., Solarflare OpenOnload, DPDK), NIC timestamping, PTP (IEEE 1588), NUMA pinning, and cache-line considerations.
  • Software Architecture: Separation of the critical path (hot path) vs. monitoring/management (cold path), IPC mechanisms (e.g., lock-free shared memory ring buffers), and memory management.
  • Networking & Protocols: Protocol selection (UDP multicast, TCP, binary framing) and wide-area network (WAN) connectivity between colocation and remote offices.
  • Safety & Resilience: Drop-copy handling, cancel-on-disconnect (COD) mechanisms, remote kill-switch propagation, and state reconciliation.

Hints

Hint 1

Separate the execution hot path completely from any blocking calls, heap allocations, and remote network I/O using lock-free single-producer single-consumer ring buffers.

Hint 2

Consider how hardware features like PTP hardware timestamping at the NIC, CPU pinning, and kernel bypass eliminate operating system overhead and non-deterministic jitter.

Hint 3

For the remote office boundary, rely on an asynchronous drop-copy or publish-subscribe bus to ensure WAN degradation or disconnection never stalls the colocated trading engine.

Follow-up questions

  • If the WAN link between the remote office and the exchange colocation goes down, how does the colocated trading engine decide whether to continue trading, widen spreads, or pull quotes?
  • How would you implement zero-copy multicast packet capture and replay for deterministic offline backtesting and post-trade compliance?
  • How do you design a pre-trade risk check layer that guarantees zero latency overhead on the happy path while ensuring safety invariants (e.g., maximum short gamma) are never violated?