Katkat
Back

Design Secure Token Authorization and Large-File Transfer

J.P. MorganSoftware Engineer II · Onsite
MediumSoftware Engineer IIOnsiteSystem Design

Last updated

Design a secure, enterprise-grade file transfer service that supports resumable multi-gigabyte uploads/downloads alongside fine-grained, token-based authorization and key management.

Scenario

Financial institutions regularly ingest and distribute massive data payloads—such as daily transaction logs, audit dumps, and regulatory risk reports—spanning hundreds of megabytes to tens of gigabytes. You are tasked with designing a secure, high-throughput file transfer service that enforces fine-grained authorization, immediate access revocation, cryptographic integrity validation, and resumable streaming over unreliable networks.

Requirements

Functional Requirements

  • Token-Based Authorization: Issue short-lived, cryptographically signed tokens containing scoped claims (tenant ID, target resource path, allowed actions read/write, and expiration). Support public key rotation and immediate token revocation.
  • Resumable Large-File Upload/Download: Enable clients to upload and download files up to 50 GB in chunks. If a transfer fails mid-flight, it must resume from the last successful chunk without restarting from byte zero.
  • Integrity Verification: Ensure chunk-level and file-level integrity checks (e.g., SHA-256 / MD5 checksums) to detect corruption or tampering.
  • Bounded Server Memory: Streaming through API proxies/gateways must operate under strict, bounded memory limits (O(1) memory per active connection) rather than buffering entire payloads in memory.

Non-Functional Requirements

  • High Availability & Concurrency: Support up to 5,000 concurrent active file streams with 99.99% availability.
  • Auditability & Security: Enforce end-to-end encryption (TLS 1.3 in transit, AES-256 at rest). Every file access and policy check must produce immutable audit logs.
  • Low Latency for Auth: Token validation must add less than 5 ms of overhead per request.

Scale & Performance Assumptions

  • Daily upload/download volume: 100,000 files/day.
  • Average file size: 500 MB; peak file size: 50 GB.
  • Standard chunk size: 8 MB to 64 MB.
  • Peak network ingress/egress: ~40 Gbps aggregate bandwidth.

Key Discussion Areas

  1. Authentication & Authorization Architecture: Token format (e.g., JWT vs. PASETO), signature verification with JWKS (JSON Web Key Set), rotation mechanisms, and handling distributed revocation (e.g., Redis blocklist, short TTLs + bloom filters).
  2. Transfer Workflow & API Design: Endpoints for initiating transfer sessions, uploading chunks with byte ranges and checksums, finalizing uploads, and range-based parallel downloads.
  3. Data Flow & Storage: Direct-to-storage transfers via pre-signed URLs vs. reverse-proxy streaming architectures; trade-offs in security control vs. network bandwidth.
  4. Failure Handling: Network timeouts, deduplication of idempotent chunk retries, and out-of-order chunk assembly.

Hints

Hint 1

Consider whether data streams directly through your application servers or if clients should receive time-limited, scoped pre-signed URLs to stream directly to an underlying object store.

Hint 2

For token revocation with asymmetric JWTs, balance performance and real-time invalidation using short token expirations paired with a fast distributed cache (e.g., Redis) storing revoked token IDs (jti).

Hint 3

Design upload sessions with explicit manifest state tracking in a relational or document database to handle chunk-level checksums, out-of-order delivery, and reassembly.

Follow-up questions

  • How would you handle a malicious client that reserves an upload session for a 50 GB file but sends chunks at 1 KB/s (Slowloris-style attack)?
  • If your underlying storage is S3-compatible object storage, how do you handle Multipart Upload aborted parts to prevent dangling storage costs?
  • How does your architecture handle key rotation without invalidating in-flight file transfer sessions that were initiated under an older key?