Katkat
Back

Design a User Behavior Tracking (Clickstream Analytics) System

RipplingSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSystem Design

Last updated

Design a scalable clickstream analytics pipeline capable of ingesting high-throughput user interaction events, handling duplicate and out-of-order data, and supporting both real-time aggregated dashboards and raw behavioral query paths.

Scenario

A modern enterprise workforce management platform requires an end-to-end User Behavior Tracking (clickstream analytics) system. The system must collect granular client events (e.g., page views, button clicks, workflow transitions, error triggers) from web and mobile applications. These events power real-time product analytics dashboards for internal teams, audit logging for enterprise customers, and feature adoption tracking.

Requirements

Functional Requirements

  • Client Ingestion: Provide an ingestion API and client SDK strategy to collect structured and semi-structured interaction events.
  • Delivery & Deduplication: Ensure reliable event transport despite unreliable client network connections, including client-side batching and deduplication.
  • Real-Time Aggregations: Support low-latency dashboard queries for metric aggregations (e.g., DAU/MAU, top actions per tenant, conversion funnels) with freshness under 1 minute.
  • Raw & Ad-Hoc Analytics: Support deep-dive user journey queries (e.g., chronological event trace for a specific user session or audit investigation).
  • Multi-Tenancy: Strict data isolation between enterprise organizations.

Non-Functional Requirements

  • High Availability & Durability: 99.99% ingestion availability; no event loss once acknowledged.
  • Low Latency: Ingestion response < 50ms; dashboard queries < 1s.
  • Scalability: Horizontal scaling for both ingestion spikes and data retention.

Scale & Traffic Estimations

  • Users: 10,000 enterprise tenants; 5 million Daily Active Users (DAU).
  • Event Volume: Average 100 events/user/day $\approx$ 500 million events/day.
  • Throughput: Average $\approx$ 6,000 events/sec; peak traffic factor of 4x $\approx$ 25,000 events/sec.
  • Payload: Average payload size is 1 KB $\rightarrow$ Peak ingest bandwidth $\approx$ 25 MB/sec.
  • Retention: Raw events retained for 90 days in hot storage and archived to cold storage; aggregates retained indefinitely.

Key Areas to Cover

  1. API & Client SDK Design: Schema design, client-side buffering, offline caching, and retry mechanisms.
  2. Architecture: End-to-end pipeline covering edge ingestion, message bus, stream processing, and persistence layers.
  3. Storage Strategy: Trade-offs between columnar OLAP databases, distributed log stores, and time-series engines for aggregate vs. per-user event logs.
  4. Data Integrity & Consistency: Handling late-arriving events, clock skew, and at-least-once delivery duplicates.

Hints

Hint 1

Consider decoupling event ingestion from processing using a distributed commit log to absorb traffic spikes and allow independent consumers for batch and streaming pipelines.

Hint 2

Evaluate whether a single database can serve both high-write append-only raw events and fast multidimensional aggregate queries, or if a Lambda/Kappa architecture with a specialized OLAP store (e.g., ClickHouse) is needed.

Hint 3

Address client reliability by generating unique event IDs and timestamps at the client, but validate and reconcile them against server ingestion timestamps to guard against clock skew.

Follow-up questions

  • How would you handle significant client clock skew or late-arriving events when computing fixed-window aggregates (e.g., tumbling 5-minute event counts)?
  • How would you design tenant-level rate limiting at the ingestion gateway to prevent a single misconfigured client from degrading pipeline throughput for other tenants?
  • If an enterprise customer invokes a GDPR 'Right to be Forgotten' request, how would you delete or anonymize that user's events across both raw event storage and pre-aggregated rollups?