Katkat
Back

Design a Distributed Blob Storage System for Build Artifacts

TeslaSoftware Engineer · Onsite
HardSoftware EngineerOnsiteSystem Design

Last updated

Design a distributed storage system for build artifacts, similar to a distributed NoSQL database but storing opaque blobs with looser ACID requirements. It tests separating immutable data from consistent metadata, upload atomicity, replication and repair, handling download fan-out, and safe retention and garbage collection.

This system design round was set in the interviewing team's own domain: design a distributed storage system for build artifacts, the outputs of the build pipeline (compiled binaries, packages, archives and similar files) that later pipeline stages download in order to test and deploy them. The interviewer described it as similar to designing a distributed NoSQL database, with two differences: the stored values are opaque blobs rather than documents or key-value records, and the ACID requirements are less strict than a database's.

Design the system end to end: the API that build jobs and downstream consumers use, how artifacts and their metadata are stored and located, how data is replicated and kept durable, how the system scales for both uploads and downloads, and exactly which consistency guarantees you keep and which you relax.

Clarifying Questions

  • What is the size distribution of artifacts (many small files, a few multi-gigabyte images, or both), and how many are produced per day?
  • Who reads artifacts, how soon after upload, and how many readers fetch the same artifact?
  • Are artifacts immutable once published, or can a name be overwritten, for example a "latest build of this target" pointer?
  • How long must artifacts be kept, and do some, such as release builds, need longer retention than routine CI outputs?
  • How are artifacts looked up: by content hash, by build ID, or by commit and target name?
  • Does the system run in one data center or several regions, and what durability and availability targets apply?
  • May the design build on an existing object store, or is the storage layer itself part of the question?

Hints

Hint 1

Separate what never changes from what must be current: Look at which data is written once and never modified, and which small records must always be up to date, and give each kind only the consistency it actually needs.

Hint 2

Follow a popular artifact: Trace what happens when many test jobs request the same freshly built artifact within a minute, and find where that load lands in your design.

Follow-up questions

  • Two builds of the same commit produce byte-identical artifacts. How does deduplication work, and how does it interact with garbage collection?
  • Release artifacts must be kept far longer than routine CI outputs. How do you implement retention policies and make deletion safe?
  • A storage node holding copies of a hot artifact fails during a burst of downloads. What do clients see, and how does the system recover?
  • A build publishes a set of artifacts that must become visible together or not at all. How do you support that without general multi-object transactions?