Design a multithreaded event logger
DatabricksSoftware Engineer · OnsiteLast updated
This problem tests your ability to design a thread-safe, high-throughput logging system that balances memory efficiency with I/O performance under concurrent load.
Problem Statement
Design a thread-safe EventLogger class that allows multiple threads to log messages simultaneously. The logger must ensure that logs are written to an underlying Sink (an interface with a write(message: String) method) in a way that minimizes contention while maintaining chronological ordering based on the time the logging call was made.
Requirements
- Thread Safety: Multiple threads must be able to call
log(message)concurrently without data corruption. - Ordering: Messages should ideally appear in the
Sinkin the order they were submitted. - Performance: Minimize the time a thread spends blocked waiting for I/O. Consider using an asynchronous or buffered approach.
- Durability: Ensure that even if the application shuts down, pending logs in the buffer are flushed to the
Sink.
Interface
class EventLogger:
def __init__(self, sink: Sink, buffer_size: int):
pass
def log(self, message: str):
"""Adds a message to the logger."""
pass
def shutdown(self):
"""Flushes remaining logs and stops the logger."""
pass
Constraints
sink.writeis a blocking I/O operation.- The system must handle high throughput without overwhelming memory if
sink.writebecomes slow.
Examples
- Input: Thread A calls
log("A"), Thread B callslog("B")shortly after. - Output: Sink receives "A" then "B".
Edge Cases
- The logger buffer is full: should it block the caller or drop logs?
- Sudden application termination before
shutdownis called.
Hints
Hint 1
Consider using a producer-consumer pattern with a thread-safe blocking queue to decouple logging threads from the I/O-bound sink.
Hint 2
Use a dedicated background thread to drain the queue and write messages to the sink sequentially.
Hint 3
Implement the shutdown method to signal the background thread to flush remaining items before exiting.
Follow-up questions
- How would you modify the design if you needed to support log levels (INFO, WARN, ERROR) and filter them based on a dynamic configuration?
- What would change if the system requires persistence, meaning logs must be written to disk before the log method returns?
- How would you handle a scenario where the sink becomes unresponsive for a long period?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.