Katkat
Back

Fine-Tune a Language Model on Question-Answer Pairs

VoleonMachine Learning Engineer · Technical Screen
HardMachine Learning EngineerTechnical ScreenMachine Learning

Last updated

Review the full workflow for supervised language-model fine-tuning on question-answer pairs, including masking, leakage control, evaluation, and rollout.

Fine-Tune a Language Model on Question-Answer Pairs

Explain how to fine-tune a pretrained language model using question-and-answer pairs, from data construction and token masking through optimization, evaluation, and deployment safeguards.

Constraints & Assumptions

  • Training and evaluation pairs must be separated without near-duplicate leakage.
  • The intended model may be decoder-only or encoder-decoder; state which format is used.
  • User or proprietary data requires an explicit privacy and licensing boundary.

Clarifying Questions to Ask

  • Should loss apply to the question tokens, answer tokens, or both?
  • What behavior is the fine-tune intended to change?
  • Which baseline and held-out tasks must be preserved?

Hints

Hint 1

Align labels with behavior: Show exactly which tokens produce training loss and how inference formatting matches the training template.

Follow-up questions

  • How would you handle answers with several valid phrasings?
  • When would retrieval or prompting be preferable to fine-tuning?