Katkat
Back

Explain LLM fine-tuning and generative models

GoogleSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenMachine Learning

Last updated

This question tests your conceptual understanding of generative model architectures and the practical trade-offs involved in adapting large-scale language models to specific tasks.

Part 1: Generative Architectures

Explain the high-level structural differences between Variational Autoencoders (VAEs) and modern Transformer-based Large Language Models (LLMs). Specifically, address the following:

  1. How does the training objective differ (e.g., reconstruction loss vs. next-token prediction)?
  2. In a VAE, why do we introduce a latent space bottleneck, and how does this affect the model's ability to generate coherent long-form text compared to an LLM?

Part 2: Fine-Tuning and PEFT

You are tasked with adapting a pre-trained LLM (like Llama-3 or GPT-4) to summarize medical documents. You have a limited dataset of 500 expert-reviewed summaries and strict latency requirements.

  1. Compare full fine-tuning with Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA. Why might PEFT be preferred in this scenario?
  2. What are the risks of 'catastrophic forgetting' or 'hallucination' when fine-tuning on a small, specialized domain dataset, and how would you mitigate them?
  3. If your fine-tuned model begins to output unsafe or biased medical advice, what strategies would you implement to improve safety without retraining the entire model?

Hints

Hint 1

Consider how the latent space in a VAE forces a probabilistic mapping of data, whereas an LLM models the conditional distribution of sequences directly.

Hint 2

Think about the difference between updating billions of parameters vs. freezing the backbone and injecting low-rank matrices.

Hint 3

Discuss techniques like RLHF (Reinforcement Learning from Human Feedback), DPO (Direct Preference Optimization), or system-level guardrails.

Follow-up questions

  • How would you evaluate the performance of a medical summary model beyond standard metrics like ROUGE or BLEU?
  • If inference latency is the primary bottleneck, how do quantization and knowledge distillation compare as optimization strategies?
  • Explain the 'temperature' parameter in generative models: how does it affect the trade-off between creativity and factual accuracy?