Katkat

Debug a PyTorch U-Net shape mismatch

AppleMachine Learning Engineer · Technical Screen

Last updated

This question tests your ability to calculate the output dimensions of convolutional layers in a U-Net architecture and resolve shape mismatches caused by cropping or padding requirements during feature map concatenation.

Problem Statement

You are building a U-Net for image segmentation where each encoder block reduces the spatial resolution, and each decoder block performs an up-convolution. You are using 'valid' padding (no padding) in your convolutional layers.

You have encountered a RuntimeError during the skip-connection concatenation step in the first up-sampling block because the encoder feature map and the decoder feature map have slightly different spatial dimensions.

The Setup

  • Input image size: (H, W) = (572, 572)
  • Encoder: A standard block applies two 3x3 convolutions. With no padding, each 3x3 convolution reduces the dimensions by 2 (e.g., 572 -> 570 -> 568).
  • Max Pooling: A 2x2 max pool follows, reducing dimensions by half (e.g., 568 -> 284).

Your Task

Given the EncoderBlock output (after the first max pooling) and the DecoderBlock (after up-sampling), write a function get_crop_indices(target_shape, source_shape) that returns the cropping coordinates required to center-crop the larger feature map to match the smaller one.

def get_crop_indices(target_shape, source_shape):
    # target_shape: (H_target, W_target)
    # source_shape: (H_source, W_source)
    # Returns: (start_h, end_h, start_w, end_w)
    pass

Example

If the encoder map is (392, 392) and the decoder map is (388, 388):

  • The difference is 4 pixels in each dimension.
  • You should crop 2 pixels from each side to align them.

Constraints

  • Assume source_shape is always greater than or equal to target_shape.
  • The difference between dimensions is always even.
  • Work in PyTorch tensor indexing logic.

Hints

Hint 1

Calculate the difference between the source dimension and the target dimension for both height and width.

Hint 2

Divide the difference by 2 to determine the number of pixels to discard from each side.

Hint 3

Use Python slicing syntax to create the crop: `tensor[:, :, start_h:end_h, start_w:end_w]`.

Follow-up questions

  • How would you modify the model architecture to avoid manual cropping by using padding instead?
  • If the input dimensions are odd, how does the 'no-padding' constraint affect the ability to maintain perfect feature alignment?
  • Explain the trade-offs between using 'same' padding (via padding arguments) versus 'valid' padding in segmentation tasks.