Design production-ready dedup service

Design a production-ready file deduplication service. Outline the architecture (ingest, chunking, indexing, storage, and metadata), APIs, and read/write workflows. Explain strategies for consistency, idempotency, fault isolation, failure recovery, and disaster recovery; how to run backfills and compaction/garbage collection safely; index sharding and rebalancing; deployment, rollout/rollback, and schema/version migration plans. Define monitoring, alerting, and SLOs; capacity planning and cost controls (compute, storage, network); privacy and compliance considerations (e.g., encryption, access control, GDPR); and techniques to minimize impact on production workloads (e.g., rate limiting, backpressure, priority queues).