Design a Multi-Region Malicious-IP Detection System

Scenario

Global platforms handle millions of incoming requests per second across geographically distributed Points of Presence (PoPs) and data centers. Attack vectors such as credential stuffing, scrapers, and distributed denial-of-service (DDoS) often rotate IPs rapidly. You need to design a multi-region Malicious-IP Detection System that intercepts ingress traffic, evaluates threat status in real time with minimal latency impact, and aggregates threat intelligence globally.

Requirements

Functional Requirements

• Edge Evaluation: Evaluate every incoming HTTP/HTTPS request against an IP reputation engine at edge gateways, returning an enforcement decision: ALLOW, CHALLENGE (e.g., CAPTCHA), or BLOCK.
• Shared Threat Intelligence: Aggregate behavioral signals (e.g., repeated 401s, abnormal request patterns) detected in one region and propagate threat updates to all other regions within seconds.
• False-Positive Mitigation & Rollback: Support real-time manual overrides, whitelisting (CIDR and ASN level), and global rollback of faulty rule or list updates within 10 seconds.
• Auditability: Provide comprehensive, tamper-evident audit logs of all block decisions and administrative rule changes to facilitate post-incident reviews.

Non-Functional Requirements

• Latency: Ingress evaluation overhead must be under 2 ms (p95) and 5 ms (p99).
• Fault Tolerance: If cross-region connectivity fails, local edge enforcement must continue operating autonomously using local state without service degradation.
• Scale & Accuracy: Handle false positives gracefully so legitimate users sharing NAT gateways or carrier-grade NATs are not broad-blocked.

Scale Assumptions

• Global Traffic: 1,000,000 QPS distributed across 6 global regions.
• Threat Dataset: 20,000,000 active malicious IPv4 addresses and 2,000,000 IPv6 /64 subnets.
• Update Rate: ~5,000 IP reputation state changes (additions, updates, expiries) per second globally.

Key Deliverables

1. High-Level Architecture: Detail the path of an ingress packet, the edge inspection mechanism, and how regional signals feed into global aggregation.
2. Edge Data Structures & Storage: Specify how multi-million IP entries (including CIDR ranges) are stored and queried in memory with sub-millisecond lookups.
3. Cross-Region Synchronization: Explain the replication and consensus topology used to sync blacklist updates without causing cross-region synchronous latency.
4. Rollback & Safety Controls: Design the emergency override pipeline and safety checks (e.g., canary rollouts of IP block lists, blast-radius limits).