Katkat
Back

Design a Hosted Notebook Platform

OpenAISoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSystem Design

Last updated

This question tests the ability to design a scalable, low-latency infrastructure for hosted interactive notebooks, focusing on container orchestration, state persistence, and session management.

Problem Statement

Design a hosted notebook platform (similar to Jupyter Notebooks or Google Colab) where users can write, execute, and persist code in an interactive environment. The system must support thousands of concurrent active users, each running their own isolated compute session.

Functional Requirements

  • Users can create, edit, and save notebook files.
  • Users can execute code blocks, with output streamed back to the browser in real-time.
  • Environments must persist state (variables, installed packages) across short disconnections.

Non-Functional Requirements

  • Low latency for code execution results.
  • Fast "resume" time when a user reconnects to an existing notebook session.
  • System must be fault-tolerant; a crash in the execution environment should not cause data loss of the notebook file itself.

Scale Assumptions

  • 50,000 daily active users.
  • 5,000 concurrent sessions at peak.
  • Notebook execution environments are containerized (e.g., Docker).

Task

Design the architecture for this platform. Please include:

  1. A high-level architectural diagram (describe components like API Gateway, Control Plane, Compute/Worker nodes, and storage).
  2. How you would handle the separation of the persistent notebook file (the '.ipynb' structure) and the ephemeral compute state (runtime variables).
  3. A strategy for rapid container provisioning/warm-up to minimize user wait time.
  4. How to manage inter-service communication between the UI and the execution backend.

Hints

Hint 1

Consider using a WebSocket or gRPC streaming connection to handle bidirectional communication between the frontend and the compute container.

Hint 2

Decouple the control plane (metadata/file storage) from the data plane (the execution container) to ensure the system remains responsive even if a container crashes.

Hint 3

Evaluate techniques like pre-warmed 'pool' containers or snapshots of memory states to significantly reduce the cold-start latency of a notebook environment.

Follow-up questions

  • How would you implement a 'Collaborative Mode' where multiple users can edit the same notebook file simultaneously?
  • If a user needs access to a GPU, how would your provisioning logic change to account for limited hardware availability?
  • How would you enforce resource quotas (CPU/RAM limits) to ensure one heavy user doesn't degrade the experience for others on the same host?