Design a Scalable Error Message Counting Service
LinkedInSoftware Engineer · Onsite
Last updated
Design a service that receives application error messages and reports how often each distinct error occurs over time. It tests grouping varied messages into error signatures, absorbing bursty write traffic during incidents, time-windowed aggregation, top-error queries and count accuracy under failures.
Design an error message counter: a service that receives the error messages applications emit and reports how many times each error has occurred. Treat it as a shared service that many applications report to, and expect the same underlying error to produce messages that differ in details such as request IDs, user IDs or timestamps.
Clarifying Questions
- Which queries must be supported: the count for one error over a time range, the most frequent errors in a recent window, a time series per error, or all of these?
- Are counts broken down by service, host, region or release version?
- How many applications report errors, and what are the normal and peak error rates?
- How fresh must the counts be, and must they be exact, or is a small bounded error acceptable?
- How long must counts be kept, and at what time granularity?
- Is the counter also expected to raise alerts, or only to answer queries?
Hints
Hint 1
Decide what one error is: Two messages from the same failure rarely match byte for byte. Settle what key you count under before you decide where the counts are stored.
Hint 2
Plan for the incident, not the average: Error traffic is bursty: one bad deploy can multiply it within seconds, and that is exactly when people read the counts. Think about where the write path absorbs a burst.
Follow-up questions
- How would you alert when a new error signature appears, or when a known one rises well above its usual rate?
- A single error suddenly accounts for most of the traffic. Which component becomes a hot spot, and how do you spread the load?
- If normalization misses a variable part of a message and the number of distinct signatures explodes, what breaks and how do you contain it?
- How would you let a team merge two signatures that turn out to be the same error, without losing historical counts?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.