Explain how to make robust HTTP API calls

Problem Statement

You are building a wrapper library for a high-traffic third-party HTTP API. This service is occasionally flaky, enforces rate limits, and requires strict adherence to request integrity. Your task is to design a robust fetch_data(url, payload) function that effectively manages the complexities of a distributed network environment.

Requirements

1. Resilience: Handle transient network errors and 5xx status codes gracefully.
2. Rate Limiting: Respect the server's Retry-After header and implement a strategy to avoid being throttled.
3. Idempotency: Ensure that retrying non-idempotent requests (like POST) does not cause unintended side effects on the server.
4. Observability: Briefly explain how you would track the health of these calls in a production environment.

Implementation Context

Assume the following function signature:

def fetch_data(url: str, method: str, data: dict, headers: dict) -> dict:
    """
    Performs an HTTP request with retry logic and error handling.
    Returns the parsed JSON response or raises a custom exception.
    """
    pass

Constraints

• Do not use external retry libraries; implement the logic manually.
• The client must handle 429 Too Many Requests and 503 Service Unavailable status codes.
• Define how you would differentiate between errors that should be retried and those that should fail immediately.