Katkat
Back

Architect an AI Chatbot for Enterprise Users

AmazonSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSystem Design

Last updated

This question tests your ability to design a scalable, enterprise-grade AI chatbot system that handles high throughput, ensures data privacy, and integrates with existing business data stores.

Problem Statement

Design an enterprise-level AI chatbot system that allows employees to query internal documentation (e.g., HR policies, technical wikis, and financial reports). The system must handle high request volumes, ensure that users only access documents they have permission to see, and maintain low latency for responses.

Functional Requirements

  • Users can send natural language queries and receive AI-generated answers.
  • The system must cite the source documents used to generate the answer.
  • Support for multi-turn conversations (maintaining context).

Non-Functional Requirements

  • Low latency (target < 2 seconds per query).
  • High availability and scalability for 10,000+ internal users.
  • Strict data privacy: access control lists (ACLs) must be respected based on the user's role.

Deliverables

  1. High-level system architecture diagram (described in text).
  2. Storage strategy for documents and vector embeddings.
  3. Approach for enforcing per-document access control.
  4. Strategy for ensuring the chatbot provides accurate information (minimizing hallucinations).

Hints

Hint 1

Consider how a Retrieval-Augmented Generation (RAG) pipeline can be adapted to filter context based on user permissions.

Hint 2

Think about the trade-offs between pre-calculating vector embeddings and the need to refresh data from frequently changing internal wikis.

Hint 3

Discuss how to handle asynchronous processing if a user query requires searching through a massive corpus of documents.

Follow-up questions

  • How would you handle document updates or deletions to ensure the vector database stays in sync with the source of truth?
  • What metrics would you monitor to evaluate the quality of the AI responses and system latency?
  • If the enterprise moves to a multi-tenant model (e.g., different departments having isolated knowledge bases), how does your design change?