Katkat
Back

Design a Jira+Confluence RAG assistant

AtlassianMachine Learning Engineer · Onsite
MediumMachine Learning EngineerOnsiteML System Design

Last updated

This question evaluates competencies in designing retrieval-augmented generation systems, including ingestion and indexing, retrieval and ranking strategies, prompt-based generation and citation handling, permission-aware access control, and monitoring for freshness and latency.

You are asked to design a simple Retrieval-Augmented Generation (RAG) system that answers employee questions using content from two internal products: Jira (issues, comments) and Confluence (pages, spaces).

Design an MVP that supports “cross-product” questions such as:

  • “What is the current status of project X and what design doc explains the latest decision?”
  • “Summarize the open P0 bugs for service Y and link the relevant runbook page.”

Requirements and constraints:

  • Data sources: Jira issues (title/description/comments/metadata) and Confluence pages (title/body/metadata).
  • Must respect document-level permissions (a user should only retrieve what they can access).
  • Latency target: interactive Q&A (a few seconds).
  • Results should cite sources (links to Jira issues / Confluence pages).
  • Handle updates: Jira changes frequently; Confluence changes less frequently.

Describe:

  1. Ingestion and indexing (chunking, embeddings, metadata)
  2. Retrieval strategy (filters, ranking, hybrid search)
  3. Prompting / generation and citation handling
  4. Permission enforcement approach
  5. Evaluation and monitoring (offline + online)