Katkat
Back

Design a Cloud DevBox Platform

OpenAISoftware Engineer · Technical Screen
HardSoftware EngineerTechnical ScreenSystem Design

Last updated

This question evaluates a candidate's ability to design scalable, secure cloud development platforms, testing competencies in distributed systems architecture, orchestration and scheduling, storage and persistence, multi-tenancy and RBAC, runtime isolation, and operational observability.

Design a cloud DevBox platform: a service that gives developers disposable or persistent remote development machines, accessible through a browser, SSH, or IDE plugins (e.g. a VS Code / JetBrains remote tunnel).

The platform should support the full lifecycle of a dev box and the operational needs of an organization:

  • Lifecycle: users can create, start, stop, pause, resume, and delete dev boxes from predefined templates.
  • Box contents: each dev box bundles an operating system image, language/toolchain dependencies, a repository checkout, a user home directory, injected secrets, and allocated CPU, memory, storage, and optional GPU.
  • Multi-tenancy: organizations and teams, quota enforcement, role-based access control (RBAC), audit logs, and cost controls.
  • Quality bar: optimize for fast provisioning, high availability, scalability to many concurrent dev boxes, strong isolation, reliable persistence, and operational observability.

Walk through requirements, the high-level architecture, the lifecycle/state model, scheduling and provisioning, isolation, storage/persistence, failure handling, and observability/cost. Call out the major tradeoffs and where the system is actually bottlenecked.

Constraints & Assumptions

State these explicitly; they size the design. Reasonable defaults if the interviewer doesn't specify:

  • Workloads are untrusted — boxes run arbitrary, possibly hostile user code, so isolation must be a real kernel/hardware boundary.
  • Pause/resume (RAM snapshot) is in scope, not just stop/start.
  • Multi-region deployment for latency and resilience.
  • Scale: on the order of $10^5$ total boxes (e.g. ~200k), of which a smaller fraction (~10–20%) are running concurrently at peak; thousands of organizations.
  • Provisioning SLO targets: warm resume in single-digit seconds; cold create of a common template in tens of seconds, not minutes.
  • Availability target: control-plane HA; a control-plane outage must not terminate running boxes or live sessions.
  • Identity: an external OIDC/SSO identity provider already exists and is consumed (not designed here).
  • Out of scope: building the in-box IDE itself, and the billing/invoicing system (you emit metering events that feed it, not invoices).

Clarifying Questions to Ask

  • Trust model: are workloads untrusted (arbitrary, possibly hostile code) or trusted internal developers? This is the single biggest design fork.
  • Is pause/resume (RAM snapshot) required, or only stop/start?
  • What is the expected steady-state concurrency and the create rate (including peak bursts like a Monday-morning login spike)?
  • Single-region or multi-region for v1?
  • Are boxes mostly persistent (survive node failure) or ephemeral/scratch, and what is the acceptable data-loss bound for each?
  • Is there a GPU requirement, and what fraction of boxes need it?

Hints

Hint 1

Frame the split first: The single most important structural decision is a **control plane / data plane split**. The control plane (API, RBAC, scheduler, metadata DB) makes decisions and holds durable state; the data plane (compute nodes, runtimes, the connection gateway) runs the boxes and carries traffic. Aim for a design where a control-plane outage blocks only *new* mutations and never kills *running* boxes or live sessions.

Hint 2

Model lifecycle as desired vs actual state: Don't hand-code each transition imperatively. Give every box a `desired_state` (set by the API) and an `actual_state`, and have an idempotent, leased **reconciler** drive `actual → desired`. This is what makes the system self-healing and safe under retries, duplicate events, and crashed workers. Be precise about **stop** (release compute, keep the workspace disk, cold boot to resume) vs **pause** (snapshot RAM + disk, warm resume with processes intact).

Hint 3

Provisioning latency is the hard NFR: "Fast provisioning" is where the real engineering hides, and the win comes from *not doing work on the critical path* — pre-doing it, sharing it, or restoring it instead of rebuilding it. Decompose a cold create into its serial costs (boot, image fetch, dependency build, workspace attach) and attack whichever dominates. Identify the actual bottleneck before optimizing — it is *not* the small metadata DB.

Hint 4

Let the trust model force the isolation choice: Pin down the trust model early. Untrusted, arbitrary user code points you at a hardware boundary (**microVM** / VM), not a shared-kernel container. This one fork shapes the runtime, the node fleet, and the entire security posture, so state it explicitly.

Follow-up questions

  • How do you implement **pause/resume** so a warm resume hits single-digit seconds at scale, and what does that cost in storage?
  • A node dies with multiple running **persistent** boxes on it. Walk through exactly how the system detects this and recovers, and state the data-loss guarantee.
  • How do you keep **secrets** out of base images and RAM snapshots while still injecting them at runtime?
  • How does the **scheduler** decide where to place a new or resuming box, and how do you avoid both fragmentation and noisy-neighbor contention?