Process Sharded Login Logs

Problem Statement

You are tasked with building a processing module that consumes login events from multiple independent shards. Each shard emits logs sequentially, but network delays and system rebalances mean logs across different shards may arrive out of chronological order. Additionally, logs may be duplicated due to retry mechanisms in the upstream producers.

Requirements:

1. Deduplication: Each log entry contains a unique request_id. You must ensure each request_id is processed exactly once.
2. Ordering: You must output login events in the correct chronological order based on the timestamp field.
3. Late-event Handling: Events may arrive with a delay of up to T seconds. You must buffer incoming events until you are confident no earlier events are pending.

Function Signature:

Implement a class LogProcessor with the following methods:

class LogProcessor:
    def __init__(self, buffer_time_seconds: int):
        # buffer_time_seconds defines the threshold for late events
        pass

    def process_event(self, event: dict):
        # event format: {'request_id': str, 'timestamp': int, 'user_id': str}
        # Add event to the pipeline
        pass

    def get_ordered_logs(self) -> list:
        # Returns and clears logs that are safe to emit based on buffer_time
        pass

Constraints:

• Assume timestamp is a Unix epoch integer.
• buffer_time_seconds ensures that if you receive an event at time t, you will not receive an event with timestamp < (t - buffer_time_seconds) in the future.
• The system must handle high throughput; minimize memory usage where possible.

Example:

If buffer_time_seconds is 5 and you receive events at times 100, 103, 101, when a new event arrives at 106, you can safely output events 100, 101, 103 because 106 - 5 = 101, and you are guaranteed no more events before 101 will arrive.

Edge Cases to Consider:

• Events arriving with identical timestamps.
• Large gaps in log ingestion (idle periods).
• System crashes (how do you maintain state for deduplication?).