Katkat
Back

Design a PDF Ingestion and Retrieval-Augmented Generation Pipeline

NikeSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenML System Design

Last updated

This problem tests the ability to design an end-to-end RAG pipeline capable of handling complex document parsing, chunking strategies for citations, and efficient synchronization of updates.

System Design: PDF RAG Pipeline

Design a scalable system that ingests PDFs and provides a Retrieval-Augmented Generation (RAG) service. The system must support accurate citations to specific page numbers even when documents contain complex layouts, tables, and scanned images.

Functional Requirements:

  1. Ingestion: Handle selectable text, OCR for scans, and table parsing.
  2. Retrieval: Provide context with explicit page-level citations.
  3. Updates: Allow users to update or delete documents, ensuring the index reflects the latest version.

Non-Functional Requirements:

  1. Accuracy: Minimize hallucination and ensure citation-to-text alignment.
  2. Latency: Ingestion can be asynchronous; retrieval should be sub-second.
  3. Scalability: Support thousands of documents.

Requirements for your response:

  • Parsing Strategy: How do you handle non-linear structures like multi-column layouts and tables?
  • Indexing: Define the data model for storing chunks and their metadata (page number, document ID).
  • Retrieval Logic: How do you ensure the LLM returns accurate page numbers during generation?
  • Lifecycle Management: Describe the workflow for document versioning and purging stale chunks.

Constraints:

  • Assume a volume of 5,000 PDF documents (avg 50 pages each).
  • System must support high-concurrency read operations.

Hints

Hint 1

Consider a multimodal parsing approach using LayoutLM or similar vision-language models to handle tables and headers correctly.

Hint 2

Store page identifiers as metadata in your vector database and include them in the LLM prompt context to force citation generation.

Hint 3

Implement a versioning scheme (e.g., DocID + Version) and a 'delete-and-reindex' flow to maintain search consistency during document updates.

Follow-up questions

  • How would you measure the performance of your retrieval component in terms of citation accuracy?
  • If a document contains repeating headers or footers on every page, how would you prevent this noise from degrading the embedding quality?
  • How would you implement access control so users only retrieve information from PDFs they are authorized to view?