Explain KL Divergence in Language Models
GoogleMachine Learning Engineer · Technical Screen
Last updated
Review KL divergence from its probability definition through directionality, support mismatch, sample estimates, and LLM regularization.
Explain KL Divergence in Language Models
Define Kullback-Leibler divergence, explain its directionality and support requirements, and describe why modern language-model training or post-training uses it.
Constraints & Assumptions
- Use probability distributions over the same events or tokens.
- KL divergence is not symmetric and is not a metric.
- Distinguish a token-level estimate from the ideal sequence-level objective.
Clarifying Questions to Ask
- Which distribution is the reference and which is being optimized?
- Is the goal regularization, distillation, variational inference, or monitoring?
- How is the expectation estimated from samples?
Hints
Hint 1
Name the expectation: Write which distribution supplies samples and which log-probability ratio is averaged.
Follow-up questions
- What changes when reverse KL is used instead of forward KL?
- Why can a sampled KL estimate be negative even though true KL is nonnegative?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.