Katkat
Back

Design an analytics dashboard for an AI chat app

SalesforceSoftware Engineer · Onsite
MediumSoftware EngineerOnsiteSystem Design

Last updated

This question tests the ability to architect a scalable observability pipeline for high-throughput AI chat interactions, focusing on metrics ingestion, data aggregation, and multi-dimensional query performance.

Problem Statement

You are asked to design an analytics dashboard for an AI-powered chat application. The system needs to track key performance indicators (KPIs) such as total chat volume, average latency per response, token usage per request, and model-specific error rates.

The system handles approximately 5,000 requests per second across different regions and supports hundreds of enterprise tenants. The dashboard must allow users to slice data by model version, geographic region, client platform (web/mobile), and tenant ID.

Functional Requirements

  1. Real-time ingestion of chat events (latency, model tokens, success/failure).
  2. A dashboard to visualize these metrics aggregated by time windows (1m, 1h, 1d).
  3. Support for filtering and grouping by dimensions (e.g., "Show me error rates for GPT-4 in North America for Tenant X").

Non-functional Requirements

  1. High availability and low latency for dashboard queries.
  2. Scalable data storage that handles multi-dimensional cardinality.
  3. Retention policy: 30 days of granular data, 1 year of daily aggregates.

Design Requirements

Provide a high-level architecture diagram or description covering:

  • Instrumentation strategy (how to capture events in the chat service).
  • Ingestion and streaming pipeline (buffer/message queue choices).
  • Data storage and aggregation strategy (Time-series vs. OLAP).
  • How you would handle high-cardinality dimensions (e.g., unique Tenant IDs).

Hints

Hint 1

Consider a push-based model where the chat service emits events asynchronously to a message broker like Kafka to decouple ingestion from the core application path.

Hint 2

Evaluate the difference between a dedicated Time Series Database (like Prometheus) and an OLAP database (like ClickHouse or Druid) for handling high-cardinality filtering.

Hint 3

Think about using materialized views or pre-aggregation layers to ensure that dashboard queries for long time ranges don't scan raw events.

Follow-up questions

  • How would you handle the storage and query requirements if the data volume increased by 100x?
  • If a specific tenant reports a spike in errors, how would you design an alerting system based on this analytics pipeline?
  • How would you ensure data consistency between the raw logs used for auditing and the aggregated metrics used for the dashboard?