Katkat
Back

Explain Core ML Fundamentals

UiPathMachine Learning Engineer · Technical Screen
EasyMachine Learning EngineerTechnical ScreenMachine Learning

Last updated

This question tests your conceptual understanding of how fundamental components of neural networks and predictive models affect optimization, generalization, and performance.

Machine Learning Fundamentals Discussion

During a technical screening, an interviewer may ask you to explain the trade-offs and mathematical intuition behind key machine learning concepts. Prepare clear, concise answers for the following scenarios:

1. Loss Functions and Optimization

Explain the mathematical relationship between the derivative of the Mean Squared Error (MSE) loss and the update rule in Gradient Descent. How does the choice of loss function (e.g., MSE vs. Cross-Entropy) influence the training dynamics of a classification model?

2. Regularization and Overfitting

Consider a model that is significantly overfitting your training data. Compare L1 (Lasso) and L2 (Ridge) regularization. Under what circumstances would you prefer one over the other, and how do they geometrically differ in their impact on weight vectors?

3. Neural Network Components

Why do we use non-linear activation functions like ReLU instead of linear ones in hidden layers? Additionally, explain the mechanism of 'Dropout' during training. How does it act as a regularizer, and how does its behavior change between training and inference phases?

4. Similarity Metrics

In the context of recommendation systems or clustering, when is Cosine Similarity preferred over Euclidean Distance? Provide an example scenario where Euclidean Distance fails to capture the desired relationship between two data points.

5. Feature Engineering

If you are working with high-cardinality categorical features (e.g., thousands of unique zip codes), what are the potential pitfalls of One-Hot Encoding, and what alternative encoding strategies might you employ?

Hints

Hint 1

Focus on the 'why'—explain not just what a technique is, but how it alters the optimization surface or the model's capacity.

Hint 2

For regularization, discuss how L1 can induce sparsity by forcing coefficients to zero, whereas L2 penalizes large magnitudes without necessarily zeroing them out.

Hint 3

Think about the scaling of data: Euclidean distance is sensitive to magnitude, whereas Cosine similarity focuses strictly on the orientation of the vectors.

Follow-up questions

  • How does the learning rate affect the convergence of Gradient Descent, and what are the signs that your learning rate is too high versus too low?
  • Can you explain the vanishing gradient problem in deep networks and how architectural choices like Batch Normalization or ReLU help mitigate it?
  • If you have an imbalanced dataset, why might accuracy be a misleading metric, and what metrics would you choose instead?