Explain Core ML Training, Evaluation, and Transformer Concepts

Scenario

You are tasked with building and evaluating a real-time multimodal content moderation model for short-form user posts (e.g., text captions and short video transcripts). In this domain, violating content accounts for roughly 0.1% of all submitted posts (1 in 1,000). The downstream system requires both high precision to avoid false censorship and low latency during inference.

Walk through your technical approach and reasoning across the following four core machine learning areas:

1. Generalization and Overfitting

• Explain how you diagnose whether your model is suffering from high variance versus high bias using training and validation loss curves.
• Contrast three distinct techniques for combating overfitting in deep neural networks (e.g., data augmentation, explicit regularization such as weight decay / dropout, and early stopping). Detail how weight decay mathematically affects parameter updates when using standard SGD versus Adam (AdamW).

2. Severe Class Imbalance

• With a 99.9% negative class distribution, accuracy is uninformative. Explain the mathematical definitions and practical trade-offs between Precision, Recall, PR-AUC, and ROC-AUC in this setting.
• Describe at least two training-time adjustments (e.g., focal loss, class-weighted cross-entropy, dynamic batch negative sampling) and how you determine the operational classification decision threshold instead of defaulting to 0.5.
• Explain how probability calibration (e.g., Platt scaling, isotonic regression) impacts post-processing and threshold stability.

3. Optimization Dynamics

• Contrast Stochastic Gradient Descent with Momentum against adaptive learning rate optimizers like Adam and AdamW.
• Explain how moving averages of gradients (first moment) and squared gradients (second moment) alter the effective update step, and discuss why learning rate warm-up is crucial when training large Transformer-based backbones.

4. Transformer Architecture and Inference Efficiency

• Explain the scaled dot-product self-attention mechanism: Attention(Q, K, V) = softmax(Q K^T / sqrt(d_k)) V. Why is the scaling factor 1 / sqrt(d_k) necessary?
• Differentiate between bidirectional attention masking and causal attention masking.
• Explain the concept of Key-Value (KV) caching during autoregressive text generation, detailing the memory footprint and latency benefits relative to naive recomputation.