Katkat
Back

Design an ML-powered search system

AtlassianMachine Learning Engineer · Onsite
HardMachine Learning EngineerOnsiteML System Design

Last updated

This question tests the ability to design a scalable, low-latency, and relevant search system for internal enterprise documents, focusing on the integration of hybrid search (keyword + semantic) and ranking strategies.

Problem Statement

Atlassian needs to design a search engine for its enterprise knowledge base (e.g., Confluence pages, Jira tickets). Employees frequently struggle to find relevant information due to the high volume of content and context-specific terminology.

Functional Requirements

  1. Search for documents based on user queries.
  2. Support both keyword-based matching and semantic similarity.
  3. Provide personalized results based on user teams and past document interactions.

Non-Functional Requirements

  1. Latency: Search results should return within 200ms.
  2. Freshness: New content or updates must be searchable within 60 seconds.
  3. Scale: System should support 100 million documents and 5,000 queries per second.

Your Task

Design the architecture for this search system. Include:

  • The data indexing pipeline and document representation.
  • The retrieval and ranking strategy (e.g., multi-stage ranking).
  • How to incorporate personalization into the ranking model.
  • Strategies to manage index freshness while maintaining high retrieval performance.
  • Monitoring and evaluation metrics to ensure search quality over time.

Hints

Hint 1

Consider a hybrid approach using BM25 for keyword retrieval and dense vector embeddings (like Transformers) for semantic matching.

Hint 2

Think about using a multi-stage architecture: a fast candidate retrieval phase followed by a more computationally expensive 're-ranker' model.

Hint 3

Address the cold-start problem and how to represent user context effectively without introducing excessive latency.

Follow-up questions

  • How would you handle the 'query intent' ambiguity when a user enters a very short or domain-specific acronym?
  • If the index size grows to 1 billion documents, how would you optimize your vector search component?
  • What A/B testing strategy would you implement to measure the impact of a new ranking model on user productivity?