Katkat
Back

Design Cloud File Storage

HarveySoftware Engineer · Onsite
MediumSoftware EngineerOnsiteSystem Design

Last updated

This question tests your ability to design a scalable cloud file storage and collaboration platform, focusing on the architectural separation between metadata and blob storage, chunking, deduplication, and granular access control.

Scenario

Design a cloud file storage service (similar to Google Drive, Box, or Dropbox) tailored for enterprise document management. Users can upload, organize, version, and share files within hierarchical folder structures, while administrators enforce granular access control and audit logging.

Requirements

Functional Requirements

  • File Operations: Users can upload, download, update, and delete files of varying sizes (from 1 KB up to 10 GB).
  • Folder Hierarchy & Metadata: Support arbitrary directory structures, renaming, moving files/folders, and tag-based search.
  • Versioning: Preserve past versions of files when updates occur, allowing rollbacks.
  • Permissions & Sharing: Fine-grained access control (Owner, Editor, Viewer) at the file or folder level with inheritance.

Non-Functional Requirements

  • Durability: 99.999999999% (11 9s) durability for file contents.
  • Availability: 99.99% availability for both metadata and blob read operations.
  • Low Latency: Directory listing and metadata operations should complete in under 50 ms at p95.
  • Resilience & Resumability: Uploads of large files must handle transient network disconnects without restarting from byte zero.

Scale Assumptions

  • User Base: 50 million monthly active users, 10 million daily active users.
  • Storage Volume: 2 billion total files, growing by 20 million files per day.
  • Average File Size: 2 MB (with a long tail up to 10 GB).
  • Read/Write Ratio: 10:1 read-to-write ratio on metadata; 5:1 on blob downloads vs. uploads.

What Your Solution Should Cover

  1. API Design: Core endpoints for resumable file uploads, downloads, and hierarchical metadata navigation.
  2. Storage Architecture: Decoupling metadata storage from raw blob storage; evaluating content-addressable chunk storage and block-level deduplication.
  3. Data Model: Schema for file systems (nodes, versions, chunks) and permission inheritance models (e.g., materialized paths or closure tables).
  4. Reliability & Consistency: Handling concurrent file writes, ensuring atomic metadata updates, and maintaining data integrity across distributed stores.

Hints

Hint 1

Decouple the file payload from its directory tree metadata early on so that metadata operations like moves and renames do not touch the underlying storage blocks.

Hint 2

For large file uploads and deduplication, consider splitting files into fixed or content-defined chunks (e.g., using Rabin fingerprinting or fixed 4 MB blocks) hashed with SHA-256.

Hint 3

To optimize deep folder permission checks without recursive database queries, consider caching evaluated access-control lists or using materialized paths.

Follow-up questions

  • How would you implement conflict resolution when two users upload different versions of the same file concurrently?
  • How does your design adapt to support end-to-end client-side encryption where the server has zero knowledge of file contents?
  • How would you implement a lifecycle tiering policy to move cold or archived document versions to cheaper object storage classes without breaking metadata search?