Katkat
Back

Design a simulator to replay left-turn videos

Applied IntuitionSoftware Engineer · Onsite
MediumSoftware EngineerOnsiteSystem Design

Last updated

This question assesses the ability to architect a high-throughput simulation system capable of retrieving, syncing, and replaying synchronized sensor data streams for autonomous vehicle testing.

Problem Statement

Design a simulation service that allows engineers to replay recorded left-turn scenarios captured by autonomous vehicles. The system must support replaying multi-sensor data streams (camera frames, LiDAR point clouds, and vehicle telemetry) at variable playback speeds.

Requirements

  • Functional: Support seeking to a specific timestamp, variable playback speed (e.g., 0.5x, 1x, 2x), and filtering scenarios by metadata (e.g., "weather=rain", "turn_duration > 5s").
  • Non-functional: Low latency for frame seeking, high throughput for streaming data, and support for concurrent developers analyzing different scenarios.
  • Scale: Assume a dataset of 100,000 hours of video. Each hour includes ~100GB of metadata, frames, and point cloud logs.

Deliverables

  1. Data Model: How should sensor data and metadata be stored and indexed?
  2. API Design: Define the primary endpoints for metadata discovery and frame/data retrieval.
  3. System Architecture: Propose a high-level design, including storage choices, caching strategies, and streaming protocols.
  4. Scalability: Describe how to handle thousands of concurrent playback requests and optimize for high-bandwidth sensor data.

Considerations

  • How do you ensure synchronization between different sensors (e.g., a frame must align with the correct LiDAR packet)?
  • What are the trade-offs between pre-processing data (transcoding) vs. on-the-fly streaming?

Hints

Hint 1

Consider decoupling the metadata (searchable events) from the high-bandwidth telemetry and video blobs using a tiered storage approach.

Hint 2

Think about using a time-series database for telemetry and object storage (S3) for large media files, utilizing indexing to map timestamps to byte offsets.

Hint 3

Discuss whether a client-side buffer or server-side streaming (e.g., WebSockets or gRPC streams) is more appropriate for maintaining sync during variable playback speeds.

Follow-up questions

  • How would you modify your design if you needed to support real-time sensor fusion overlay injection during replay?
  • What strategies would you implement to minimize the cost of cold storage vs. hot cache for frequently accessed scenarios?
  • How would you handle schema evolution if the sensor suite on the vehicle changes, resulting in different data formats for newer recordings?