Design an ML inference orchestration platform

Your company exposes multiple ML models as independent services (e.g., classification, embeddings, re-ranking). Design a platform that orchestrates these models into end-to-end workflows for external users. Describe request routing/orchestration, data flow between models, schemas and validation, intermediate storage/caching, final results storage, model/version management, failure handling and retries, monitoring/traceability, scaling and deployment, and multi-tenant auth/rate limiting. Justify when to use RPC versus REST between services and the trade-offs in latency, schema evolution, observability, and backward compatibility.