Katkat
Back

Choose Between REST, WebSocket, and Server-Sent Events for an Agent Runtime

ByteDanceSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSoftware Engineering Fundamentals

Last updated

This question tests the ability to evaluate communication protocols based on specific architectural requirements, including bidirectional signaling, state recovery, and infrastructure overhead.

The Scenario

You are building an Agent Runtime service that executes long-running AI tasks. The service has the following requirements:

  1. Job Submission: The client triggers a task (e.g., 'Analyze Market Data') and receives a task ID.
  2. Progress Streaming: The runtime must push partial results and logs to the client as they become available.
  3. Cancellation: The client must be able to send an urgent interrupt signal to stop a running job.
  4. Resilience: If the client's network flickers, the client should be able to reconnect and resume receiving progress for the same job ID without restarting the task.

The Task

You are tasked with designing the communication layer for this service. Compare and contrast three protocols—REST (Polling), Server-Sent Events (SSE), and WebSockets (WS)—in the context of this specific use case.

Provide a recommendation on which protocol(s) to use, keeping in mind:

  • Infrastructure constraints (e.g., Load Balancers, Firewalls, CDNs).
  • Protocol overhead and connection management.
  • The complexity of implementing state recovery (resuming streams after a drop).

Hints

Hint 1

Consider the directionality requirements: Does the client need to send data back to the server frequently, or only for status updates and cancellations?

Hint 2

Think about the HTTP overhead of polling vs. the persistent connection state management required for WebSockets.

Hint 3

Evaluate how each protocol handles proxies, load balancers, and network timeouts in enterprise environments.

Follow-up questions

  • If you choose SSE for progress updates, how would you handle the 'cancellation' and 'job submission' parts of the API?
  • How does your choice of protocol impact the server's ability to scale horizontally and handle tens of thousands of concurrent agents?
  • What specific mechanisms would you implement to allow a client to 'resume' an SSE stream from a specific offset after a connection drop?