Design multi-GPU matrix multiplication
GoogleMachine Learning Engineer · Technical Screen
Last updated
This question evaluates proficiency in multi-GPU parallelism and system-level ML engineering, covering data partitioning, inter-GPU communication primitives, compute scheduling and overlap, memory layout and buffer reuse, numerical precision trade-offs, synchronization, scalability, and failure handling.
Design and implement computing C = A × B across two GPUs when A and B must reside on both devices. Specify data partitioning (row/column/block tiling), communication primitives (e.g., all-reduce, all-gather, point-to-point), compute scheduling (tiled GEMM with overlap of compute and communication), memory layout and buffer reuse, numerical precision, synchronization, how you aggregate and return C, and discuss scalability and failure handling.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.