Design batch and streaming ETL architecture

Design an end-to-end data platform that supports both daily batch processing and near-real-time streaming for product analytics. Specify the ingestion sources and formats; schema design for raw, staging, and modeled layers; table partitioning and clustering; strategies for idempotency, deduplication, and handling late/out-of-order events; update patterns (append-only vs upsert/merge), slowly changing dimensions (SCD1/SCD 2), and backfills; orchestration, dependency management, and failure recovery; aggregation strategies for daily/hourly/rolling-window metrics; data quality checks and SLAs; and trade-offs between latency, cost, and complexity.