Katkat
Back

Optimize Tensor Runtime Kernels

WaymoMachine Learning Engineer · Technical Screen
MediumMachine Learning EngineerTechnical ScreenSoftware Engineering Fundamentals

Last updated

This question evaluates competency in optimizing machine learning framework runtimes for accelerators, covering profiling and performance analysis of tensor kernels, memory layout design, operator scheduling and kernel fusion while weighing trade-offs among throughput, latency, memory usage, numerical correctness, and maintainability.

You are responsible for optimizing an ML framework runtime on an accelerator. A model has high latency and occasional memory pressure. Explain how you would profile and optimize tensor kernels, memory layout, operator scheduling, and kernel fusion. Include tradeoffs between throughput, latency, memory usage, numerical correctness, and maintainability.