Katkat
Back

Design a real-time 3D playback system for recorded vehicle logs

Applied IntuitionSoftware Engineer · Onsite
MediumSoftware EngineerOnsiteSystem Design

Last updated

Design a system that plays recorded vehicle log data back as a smooth, seekable 3D scene without downloading the whole log first. It tests chunked, time-indexed streaming, prefetching and caching around the playhead, aligning data streams of different rates, and handling heavy sensor data at scale.

Design a system that plays back recorded vehicle log data as an interactive 3D scene. Each recorded drive is a set of logs of time-stamped data stored in object storage, such as an S3 bucket. An engineer opens a drive in a viewer and watches the vehicle and its recorded data move through a 3D scene in real time, with play, pause and a timeline to seek on.

Playback must start quickly and stay smooth, even though one recorded drive can be far too large to download before the first frame is shown.

Constraints and Clarifications

  • The logs already exist in object storage when playback is requested; the system reads them and does not control how they were recorded.
  • Assume every recorded message carries a timestamp, so all data in a drive can be placed on one timeline.

Clarifying Questions

  • What does a log contain: only the vehicle's pose over time, or also heavy sensor data such as point clouds and camera images, and at what rates?
  • How large is one recorded drive, and in what file format is it stored?
  • Does "real time" mean smooth playback of recorded logs at recorded speed, or must the viewer also follow a drive that is still being uploaded?
  • How many engineers watch playback at once, and do many of them open the same drives?
  • Does playback need speeds other than 1x, frame-by-frame stepping, or playing backward?
  • Must the viewer run in a web browser?

Hints

Hint 1

Think like a video player: A video site does not make you download a whole movie before the first frame plays. Consider what the equivalent of a video segment and a playlist would be for time-stamped log data.

Hint 2

Seeking into the middle: A user drags the timeline to the middle of a long drive. Decide exactly what the viewer must fetch to draw that moment correctly, and what it should fetch next.

Hint 3

Many streams, one clock: Different kinds of recorded data are sampled at different rates. Decide how the viewer chooses what to draw for a given playback time.

Follow-up questions

  • How would you support playback at 4x speed, or stepping backward one frame at a time?
  • How would the design change if engineers must watch a drive while the vehicle is still uploading it?
  • Two engineers want to review the same moment together and always see the same frame. How would you synchronize their viewers?
  • How would you let users search across many drives for moments of interest and jump straight to them?