Katkat
Back

Design AI-Powered Document Search

WorkdaySoftware Engineer · Onsite
MediumSoftware EngineerOnsiteML System Design

Last updated

This question assesses the ability to design an end-to-end, scalable pipeline for ingesting, processing, and indexing heterogeneous documents (PDFs, images, text files) to support AI-driven semantic search.

Problem Statement

Design a scalable document ingestion and indexing system for an enterprise platform. Users upload various document formats (e.g., scanned PDFs, Word docs, images of receipts) which must be made searchable via both keyword matches and semantic similarity (vector search).

System Requirements

Functional

  1. Support asynchronous ingestion of large batches of documents.
  2. Extract text via OCR for images/scanned PDFs.
  3. Generate embeddings for semantic search.
  4. Extract key metadata (e.g., invoice number, date, sender) using an NLP model.

Non-Functional

  1. Fault tolerance: If an OCR model or embedding service fails for a specific document, the system must support retries without duplicating data.
  2. Scalability: The system must handle spikes in ingestion volume.
  3. Monitoring: Track the processing pipeline latency and identify documents stuck in the queue.

Expected Output

  • Provide a high-level architecture diagram or description of the services.
  • Describe the data flow from document upload to final indexing in the vector database.
  • Discuss the choice of storage (e.g., Object Store, Metadata DB, Vector DB).
  • Explain how to handle retries and partial failures effectively.

Hints

Hint 1

Use a message queue (e.g., Kafka or SQS) to decouple ingestion from the intensive processing tasks like OCR and model inference.

Hint 2

Implement an idempotency strategy by using document hashes or unique task IDs to ensure retries do not result in duplicate index entries.

Hint 3

Separate your storage concerns by keeping raw documents in an object store (e.g., S3), extracted metadata in a relational database, and embeddings in a vector-optimized database.

Follow-up questions

  • How would you handle a scenario where an OCR model performs poorly on a specific document—how do you flag it for human review without stopping the pipeline?
  • What strategies would you implement to update the vector index when the underlying machine learning model is retrained and embeddings need to be refreshed?
  • How would you measure the search quality, and how do you incorporate user feedback (e.g., clicks on search results) to improve future retrieval?