Katkat
Back

Design a chatbot (system design)

OpenAISoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenML System Design

Last updated

This question evaluates system design skills for ML-powered chatbots with a front-end focus, testing understanding of client-side privacy and storage lifetimes, streamed response delivery, stateless relay architectures, authentication boundaries, rate limiting, and failure-mode handling.

Question

Design an AI chatbot system with a front-end focus, under the following constraints:

  1. User messages and conversation history are stored only in the user's browser — never in any server-side database.
  2. Bot responses must be streamed to the user (tokens appear incrementally).
  3. Refreshing the page starts a brand-new conversation (no persisted history).
  4. User credential / authentication handling must be addressed on the client without leaking provider secrets.

In your design, cover:

  • The overall architecture (client SPA, edge/relay layer, backend LLM provider) and the end-to-end data flow.
  • How streaming is implemented, and the trade-offs of Server-Sent Events (SSE) vs. WebSockets.
  • Session management without a server database (where conversation state and tokens live, and how refresh resets them).
  • Security for user credentials and provider API keys (OAuth/OIDC + PKCE, token lifetimes, scopes, CORS/CSP/XSS).
  • Rate limiting and abuse prevention when you have no durable server-side store.
  • Failure modes and retry strategy (network drops mid-stream, 429s, 5xx, timeouts, token expiry, oversized context).
  • The trade-offs of the no-database approach (privacy and cost vs. lost continuity, larger per-request payloads, weaker analytics/personalization).