Katkat
Back

Design a Multi-Agent Chat Backend

HebbiaSoftware Engineer · Onsite
MediumSoftware EngineerOnsiteSystem Design

Last updated

This question evaluates a candidate's ability to design scalable, reliable backend architectures that integrate LLMs and multi-agent orchestration, covering distributed systems, streaming responses, conversation context management, tool integrations, observability, and secure handling of sensitive financial data.

Design the backend for an LLM-powered chat product used by customers in a financial-technology setting. The interviewer will show a simple demo of a chat UI, but your scope is the backend.

The system should support:

  • A user sending a chat message and receiving a streamed assistant response.
  • Conversation history and context management.
  • A multi-agent orchestration layer, where different agents may handle tasks such as planning, retrieval, calculation, compliance checks, and final response generation.
  • Tool calls to internal services or external APIs.
  • Reliability, observability, and safe handling of sensitive financial data.

Describe the architecture, main services, data model, request flow, scaling strategy, and failure handling.