Katkat
Back

Design streaming error-log counting with moving average

MicrosoftSoftware Engineer · Online Assessment
MediumSoftware EngineerOnline AssessmentSystem Design

Last updated

This question evaluates understanding of real-time stream processing concepts such as event filtering, stateful per-service counting, sliding-window moving averages, late-event handling, scalability, and fault-tolerant state management for monitoring and alerting.

Design a core component in a streaming system:

Input:

  • Multiple upstream services continuously emit log events.
  • Each event includes at least: service_id, timestamp, log_level, message.

Tasks:

  1. Filter and output only error logs.
  2. Maintain real-time per-service error count.
  3. Maintain a moving average of error count per service over a sliding time window.
  4. Trigger an alarm when a service’s error rate/moving average crosses a threshold.

Describe the architecture, state management, windowing approach, and how you handle late events, scale, and fault tolerance.