Design and benchmark optimized inference pipelines

Describe PyTorch Dynamo (aka TorchDynamo) in the context of accelerating inference: what it does, how it captures/compiles graphs, and how it relates to TorchInductor and backends like TensorRT. List several techniques to speed up inference (briefly): forms of parallelism (data/model/pipeline), effective batching, operator fusion, quantization, kernel autotuning, CUDA Graphs, overlapping compute and data transfer (e.g., pinned memory/streams), sparsity, caching, and graph-level compilers. How would you design a fair inference benchmark? Specify metrics (latency p50/p95/p99, throughput, GPU/SM utilization, memory), test setup (GPU model, precision, batch size and sequence length, warmup and iteration counts, concurrency), baselines (vanilla PyTorch), and how to report absolute and percentage improvements.