Katkat
Back

Design a Thread-Safe Buffered Writer with a Background Flush Thread

DatabricksSoftware Engineer · Onsite
MediumSoftware EngineerOnsiteSystem Design

Last updated

Tests the ability to design a high-throughput, thread-safe buffering system that decouples client write operations from high-latency storage I/O using concurrent synchronization, backpressure, and background worker threads.

Scenario

In high-performance analytics engines, logging frameworks, and distributed storage clients, producer threads frequently generate fine-grained log events, metrics, or row data. Directly writing each event to persistent storage (such as local SSDs or cloud object stores) introduces excessive I/O latency and system-call overhead.

You are tasked with designing an in-process, thread-safe BufferedWriter component that aggregates records in memory and delegates disk writes to one or more background flush threads.

Requirements

Functional Requirements

  • Client API: Provide an interface with append(record: bytes) -> bool, flush() -> None, and close() -> None.
  • Batching & Flushing Triggers: The background worker must flush records to disk whenever either condition is met:
    1. The accumulated buffer size reaches a byte threshold B (e.g., 4 MB).
    2. A maximum elapsed time T has passed since the oldest unwritten record arrived (e.g., 200 ms).
  • Graceful Shutdown: Calling close() must flush all remaining buffered data to disk and cleanly terminate the background thread.

Non-Functional Requirements & Scale

  • Throughput: Support up to 100,000 writes/sec from dozens of concurrent producer threads (~50 MB/sec aggregate throughput, bursting up to 200 MB/sec).
  • Low Contention: Minimize lock contention so producers spend minimal time waiting on concurrent writes or disk I/O.
  • Bounded Memory & Backpressure: In-memory buffer capacity must not exceed a configured limit M (e.g., 64 MB). If disk I/O stalls, the system must apply backpressure rather than causing an out-of-memory error.
  • Durability Semantics: Acknowledge trade-offs between asynchronous writes and data loss on unexpected process crashes.

Key Areas to Cover

  1. Component Architecture & Concurrency Model: Buffer data structures (e.g., double-buffering, lock-free ring buffers, segmented queues) and thread synchronization mechanisms (mutexes, condition variables, atomic counters).
  2. Buffer Swapping & Disk Flush Lifecycle: How the background thread extracts ready batches while producers continue appending with minimal disruption.
  3. Backpressure Strategy: Behavior when downstream I/O cannot keep pace (blocking producers, dropping lower-priority records, or adaptive timeouts).
  4. Shutdown & Error Handling: Failure recovery during disk writes and ensuring a clean drain of pending bytes on termination.

Hints

Hint 1

Consider using a double-buffering or buffer-swapping pattern to separate the buffer currently accepting writes from the buffer being flushed to disk under an I/O lock.

Hint 2

Use a condition variable with a timed wait to allow the background thread to sleep until either a size threshold is crossed or the maximum latency window expires.

Hint 3

Define a clear backpressure contract for producers when the buffer reaches capacity, such as blocking the caller on a condition variable until the flusher frees space.

Follow-up questions

  • How would you adapt the design to support zero-copy I/O using gather writes (such as `writev`) or memory-mapped files?
  • How can you implement a synchronous `append_sync(record)` variant that blocks only until that specific record has been persisted to disk?
  • If write throughput scales to multiple gigabytes per second, how would you partition the buffer across CPU cores to avoid cache-line bouncing and lock contention?