Katkat
Back

Design an OOD detection system

OpenAIMachine Learning Engineer · Technical Screen
MediumMachine Learning EngineerTechnical ScreenML System Design

Last updated

This question tests the ability to design an end-to-end system for detecting out-of-distribution (OOD) data in production LLMs to ensure reliability and safety.

Scenario

You are tasked with designing an OOD detection system for a customer-facing LLM-based chatbot. The chatbot provides financial advice based on a specific knowledge base of corporate documents. As the system scales, users are increasingly querying topics outside the domain (e.g., medical advice or casual social interaction). You need to build a pipeline to identify these OOD queries before they reach the LLM or before the response is served to the user.

Requirements

  • Define the architecture for real-time OOD detection.
  • Discuss the trade-offs between precision and recall in a financial context.
  • Describe the monitoring strategy for drift detection over time.
  • Propose a method for incorporating human-in-the-loop feedback to refine the OOD thresholds.

Constraints & Assumptions

  • Assume a request volume of 5,000 queries per second.
  • Latency budget for detection must be under 50ms.
  • The model should be adaptable to new domains without full retraining.

Expected Deliverables

  • A high-level system diagram description.
  • Selection of features or metrics (e.g., perplexity, embedding distance, logit-based scores) to trigger OOD alerts.
  • Strategy for handling OOD data: reject the query, route to a human, or route to a general-purpose model.

Hints

Hint 1

Consider utilizing a secondary, smaller 'detector' model trained specifically on the distribution of your known knowledge base.

Hint 2

Think about the difference between semantic OOD (different topics) versus covariate shift (changes in user language style).

Hint 3

Evaluate the cost of false positives (refusing valid financial queries) versus false negatives (providing advice on unauthorized topics).

Follow-up questions

  • How would you measure the performance of your OOD detector if you don't have labeled 'OOD' data in production?
  • If you decide to update your knowledge base, how does your OOD system need to change to avoid flagging new, valid documents as OOD?
  • How would you integrate an 'uncertainty quantification' score from the primary LLM into your decision logic?