Katkat
Back

Explain precision/recall and compute NN output

CoinbaseMachine Learning Engineer · Online Assessment
HardMachine Learning EngineerOnline AssessmentMachine Learning

Last updated

This question tests the ability to calculate classification performance metrics, explain ensemble model dynamics, and perform manual forward propagation through a small neural network architecture.

Part 1: Classification Metrics

Given a binary classification model evaluating fraud detection, the confusion matrix results are: True Positives (TP) = 80, False Positives (FP) = 20, True Negatives (TN) = 850, and False Negatives (FN) = 50.

Calculate the Precision, Recall, and F1-Score. If the cost of a False Negative is 10x higher than a False Positive, would you prioritize increasing Precision or Recall? Explain your reasoning.

Part 2: Ensemble Learning

You are tasked with improving model robustness. Explain the fundamental difference between Bagging and Boosting in terms of how they affect model variance and bias. Why does Bagging typically require low-bias base learners, while Boosting benefits from high-bias (weak) learners?

Part 3: Neural Network Forward Propagation

Consider a single-layer neural network with 2 inputs (x1, x2), 1 hidden neuron, and 1 output.

  • Inputs: x1 = 0.5, x2 = -0.2
  • Weights to hidden neuron: w1 = 0.8, w2 = 0.3, bias = 0.1
  • Activation function: ReLU(z) = max(0, z)
  • Weight to output: w_out = 1.2, bias_out = 0.0

Calculate the final output value. Show the intermediate hidden layer value before and after the ReLU activation.

Constraints:

  • Assume standard arithmetic rules.
  • Round all floating-point results to 3 decimal places.

Hints

Hint 1

Recall is the ratio of TP to the total number of actual positives, while Precision is the ratio of TP to the total predicted positives.

Hint 2

Bagging reduces variance by averaging models trained on subsets, whereas Boosting reduces bias by sequentially correcting errors of previous models.

Hint 3

To compute the output, first calculate the linear combination of inputs, apply the activation, and then pass that result through the final layer's weight.

Follow-up questions

  • How does the F1-score change if the model's threshold is shifted to be more conservative?
  • How would you handle class imbalance in the training data if the number of fraud cases (positives) was extremely low compared to legitimate transactions?
  • If we replaced the ReLU activation with a Sigmoid function, how would that affect the potential for vanishing gradients during backpropagation?