Katkat

Design log-query stream processor

DatadogSoftware Engineer · Technical Screen

Last updated

Design log-query stream processor evaluates requirements, scale assumptions, API/data design, architecture, trade-offs, failure modes, and rollout in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Question

Design and implement a function that processes a mixed input stream of queries (prefix "Q:") and logs (prefix "L:"), assigns incremental IDs to new queries, outputs an acknowledgment for each query, and tags each log line with the IDs of matching queries as shown in the example. How would you modify your design to efficiently handle a very large volume of data? How would you support deletion of queries in your current implementation, and what inefficiencies need to be addressed?