Katkat
Back

Explain What a Neural Network Is and How It Learns

Axq CapitalSoftware Engineer · HR Screen
MediumSoftware EngineerHR ScreenMachine Learning

Last updated

Explain what a neural network is: how units and layers turn weighted inputs into outputs, why non-linear activations are essential, and how the weights are learned with a loss function, backpropagation, and gradient descent. It tests core machine learning fundamentals, including overfitting and when simpler models are the better choice.

The interviewer asked: "What is a neural network?"

Give an explanation that a technical interviewer would accept as complete, covering what the model computes from its inputs and how it learns its parameters from data.

Clarifying Questions

  • Does the interviewer want an intuitive explanation, the mathematical formulation, or both?
  • Should the answer stay with plain feed-forward networks, or also mention architectures such as convolutional, recurrent and transformer networks?
  • Should the explanation be tied to a particular kind of data or prediction task?

Hints

Hint 1

Start from one unit: Describe what a single unit computes from its inputs before stacking units into layers.

Hint 2

Stacking alone is not enough: Consider what a stack of layers could represent if every layer computed only a weighted sum of its inputs.

Hint 3

Where the weights come from: Explain how the weights get their values, starting from a measure of how wrong the network's outputs are.

Follow-up questions

  • Why does backpropagation cost only a small constant multiple of a forward pass, rather than one forward pass per weight?
  • Your dataset is small and noisy. What risks does a neural network bring, and what would you try first?
  • Why did deep networks with sigmoid activations train poorly, and what changed that?
  • On tabular data, neural networks are often matched or beaten by gradient-boosted trees. Why might that be?