Katkat
Back

Design the Frontend of an AI Chat Application With Streaming Replies

WaymoFrontend Engineer · Onsite
MediumFrontend EngineerOnsiteSystem Design

Last updated

Frontend-focused system design for an AI chat application. It covers component structure, conversation and message state, streaming replies, stop and retry, safe rendering of model output, performance on long conversations, accessibility and the backend API contract.

In this system design round for a senior frontend role, you are asked to design an AI chat application with the focus on the frontend. A user writes a message, the model's reply appears in the conversation, and the user can continue the conversation or come back to earlier ones.

Design the client: its component structure, state, communication with the backend, and how it behaves under slow responses, failures and long conversations. Describe the API contract the frontend needs from the backend, but you do not need to design the model serving itself.

Clarifying Questions

  • Should replies stream in as they are generated, or appear all at once?
  • Which platforms: desktop web only, or mobile web and native apps too?
  • Is there a history of multiple conversations, and must it sync across devices and tabs?
  • What can a reply contain: formatted text, code blocks, tables, images? Can users attach files?
  • Can the user stop a reply, regenerate it, or edit an earlier message?
  • How long can a conversation get, and are there usage limits the UI must communicate?

Hints

Hint 1

Replies take a while: Think about what the user should see during the seconds between sending a message and the reply being complete, and what they may want to do meanwhile.

Hint 2

The reply is untrusted text: Consider what the reply may contain once it is rendered as rich text.

Follow-up questions

  • The user edits a message from earlier in the conversation. How do you model the resulting branches in state and in the UI?
  • The same conversation is open in two tabs. How do you keep them consistent?
  • How would you measure perceived responsiveness, for example time until the first words of a reply appear?
  • The network drops in the middle of a long reply. What does the user see, and what happens when the connection returns?