Katkat
Back

Design How an LLM Detects Questions Beyond Its Training Cutoff and Says So

CohereMachine Learning Engineer · Onsite
MediumMachine Learning EngineerOnsiteML System Design

Last updated

Design how a large language model recognizes questions that depend on information after its training cutoff and tells the user where its knowledge ends. The question tests data construction for calibrated abstention, prompt versus fine-tuning versus retrieval trade-offs, and evaluating both over-refusal and hallucination.

When a chat assistant such as ChatGPT-3.5 is asked about something recent, it may reply that its training data only goes up to a certain point (for example, 2022) and that it therefore cannot answer. How would you implement this behavior in a large language model you are building?

Constraints and Clarifications

  • Assume you control the model's training pipeline, including a pretraining corpus in which most documents carry a date and the later fine-tuning stages, as well as the serving stack.
  • The model is retrained periodically, so the cutoff date will change.

Clarifying Questions

  • Should the model refuse outright, or answer with what it knew as of its cutoff and a clear caveat?
  • Is the product allowed to use retrieval or web search, which could answer the question instead of declining it?
  • Does the model receive the current date at inference time?
  • Which failure is more costly for this product: declining a question it could have answered, or confidently answering with outdated information?

Hints

Hint 1

Two separate problems: Knowing where your own knowledge ends is a different problem from recognizing that a particular question depends on something after that point. Solve them separately.

Hint 2

Where the behavior lives: Consider each place the behavior could come from: the prompt at serving time, the training data, a separate component in front of the model, or a tool. Each has a different cost when the cutoff moves.

Follow-up questions

  • A model's knowledge usually thins out gradually over the last months before its nominal cutoff. How does that affect the date you tell it and its answers about that period?
  • How would you measure over-refusal on questions about stable facts that merely mention a recent year?
  • If you later add web search, how does this behavior change, and what should the model say when the search fails?