Design a batch inference API

Design an inference service API where clients POST a job and later poll for results. Requirements: accept single or batch inputs; return a job ID on submission; provide status endpoints (queued, running, succeeded, failed); no streaming required. Specify request/response schemas, idempotency keys, timeout and retry behavior, and rate limits. Describe the job queue, workers, and storage of intermediate and final results; how you would scale workers, batch efficiently, and utilize accelerators; and how you would implement observability, error handling, and partial failures within a batch.