Katkat
Back

Build a GPU VM Fleet CLI

HeygenSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSoftware Engineering Fundamentals

Last updated

This problem tests your ability to design a robust CLI interface and state management logic for an abstraction layer that interacts with multiple heterogeneous cloud GPU providers.

Problem Statement

You are tasked with building a CLI tool named gpulift to manage a fleet of GPU virtual machines across different cloud providers (e.g., AWS, GCP, Lambda Labs). Currently, each provider has a different API, and users are tired of managing configuration files manually.

Your goal is to define the architecture and the core logic for the following command: gpulift scale --gpu-type A100 --count 5 --region us-east-1

Requirements:

  1. Provider Abstraction: Define how you would represent the cloud provider interface so that adding a new provider requires minimal code changes.
  2. Idempotency/Reconciliation: The scale command must be idempotent. If the user requests 5 GPUs and 3 are already running, the system should only provision 2 more. If the system crashes mid-provisioning, the CLI should be able to recover its state on the next run.
  3. State Management: Propose a mechanism to track the 'desired state' vs. the 'actual state'.

Constraints:

  • Assume a local state.json file is available for persistence.
  • Focus on the structure of your provider interface and the reconciliation loop.

Example CLI Workflow:

# Initial request
$ gpulift scale --gpu-type A100 --count 2
[INFO] Provisioning 2 A100s...
[SUCCESS] 2 instances running (ID: vm-1, vm-2)

# Adding more
$ gpulift scale --gpu-type A100 --count 3
[INFO] 2 instances found. Provisioning 1 additional A100...
[SUCCESS] 3 instances running (ID: vm-1, vm-2, vm-3)

Edge Cases:

  • What happens if a provider API returns a 'Rate Limit Exceeded' error during provisioning?
  • How do you handle partial success (e.g., 1 of 2 requested VMs fails to boot)?

Hints

Hint 1

Use an Abstract Base Class (ABC) or Interface pattern to define common methods like `provision()`, `terminate()`, and `get_status()` for all providers.

Hint 2

Implement a 'Read-Plan-Apply' cycle: read the current cloud state, compare it with the desired state from the CLI arguments, and execute only the delta.

Hint 3

Store the unique cloud provider IDs and their associated metadata in your local state file to map your internal tracking to the actual cloud resources.

Follow-up questions

  • How would you extend this tool to support 'spot' instances with automated preemption handling?
  • If the CLI tool is used by multiple team members simultaneously, how would you prevent race conditions or conflicting state?
  • How would you implement a health check mechanism to detect if an instance is running but unresponsive?