Katkat
Back

Design a Real-Time Asset Price Service

CoinbaseBackend Engineer · Onsite
MediumBackend EngineerOnsiteSystem Design

Last updated

Design a high-throughput, low-latency price ingestion and distribution service that normalizes multi-exchange market data, reconciles conflicting ticks, and delivers live prices to internal services and retail clients.

Scenario

You are tasked with designing the core Real-Time Asset Price Service for a cryptocurrency exchange. The service aggregates tick-by-tick trade and quote data from external liquidity venues and exchanges, derives canonical reference prices (e.g., volume-weighted mid-market prices), and serves both internal trading engines and millions of active end users.

The system must start with a single upstream provider and cleanly scale to ingest dozens of disparate feeds with varying data formats, network latencies, and reliability profiles.

Requirements

Functional Requirements

  • Multi-Source Ingestion: Ingest raw order book top-of-book (BBO) and last-traded-price feeds from multiple third-party exchanges via WebSockets and FIX protocols.
  • Normalization & Provenance: Normalize diverse source payloads into a unified schema, tracking source timestamps, local receipt timestamps, and source IDs for regulatory auditability.
  • Price Aggregation / Reconciliation: Compute a canonical price per pair (e.g., BTC-USD) in near-real-time using a deterministic algorithm (e.g., volume-weighted average or median with outlier rejection).
  • Fan-Out & Delivery: Expose real-time prices to internal systems (risk engine, order execution) via streaming RPC/pub-sub and external web/mobile clients.
  • Fallback & Circuit Breaking: Detect stale or misbehaving upstream feeds and gracefully degrade to healthy sources.

Non-Functional Requirements

  • Latency: End-to-end ingestion-to-fan-out p99 latency < 50ms for internal subscribers; < 200ms for external clients.
  • Freshness & Accuracy: Reject or flag ticks older than a configured threshold (e.g., 5 seconds clock drift or network lag).
  • High Availability: 99.99% uptime. System must survive single availability zone failures without serving stale prices.

Scale & Traffic Assumptions

  • Asset Pairs: 1,000 active trading pairs across 10 upstream venues.
  • Ingestion Volume: ~50,000 raw market ticks/sec aggregate peak across all providers.
  • Internal Subscribers: ~100 internal microservices consuming via dedicated message streams.
  • External Subscribers: Up to 2,000,000 concurrent connected web and mobile clients during peak market volatility events.

Key Focus Areas

  • Architecture: Feed handlers, message bus, price derivation engine, and fan-out gateway layer.
  • Protocol Trade-offs: Detailed comparison of polling, Server-Sent Events (SSE), and WebSockets for client delivery.
  • Backpressure & Ordering: Handling slow consumers and out-of-order network packets across distributed pipelines.
  • Data Model & Caching: Schema for ticks and canonical prices, plus in-memory caching strategies.

Hints

Hint 1

Decouple protocol-specific feed adapters from downstream normalization using an internal partitioned event log or message bus.

Hint 2

For client fan-out to millions of connections, decouple connection termination from price calculation using a dedicated edge gateway tier that coalesces high-frequency updates.

Hint 3

Resolve clock skew and out-of-order ticks by enforcing monotonic versioning based on ingestion sequence IDs alongside source timestamps.

Follow-up questions

  • How would you prevent a sudden flash crash on a single low-liquidity upstream exchange from poisoning your canonical reference price?
  • If millions of retail clients are connected via WebSockets, how do you handle backpressure when a client's cellular connection drops to high packet loss?
  • How would your architecture change if internal algorithmic execution engines required deterministic p99 latency under 2 milliseconds?