Design email ranking and summarization in Outlook

Scenario

You are tasked with designing a system for Outlook that performs two primary functions:

1. Personalized Ranking: Ordering the user's inbox to prioritize important emails.
2. Abstractive Summarization: Generating concise, context-aware summaries for high-priority threads.

Requirements

• Scalability: The system must handle millions of daily active users with sub-500ms latency for ranking and generation.
• Privacy: Personal data must be processed with strict adherence to data residency and user-privacy constraints.
• Relevance: The ranking model should learn from user signals (opens, replies, deletes) while avoiding feedback loops that hide important information.
• Safety: Summaries must be faithful to the source and avoid hallucinating sensitive info or generating biased content.

Discussion Points

1. System Architecture: Propose a high-level design, including data ingestion, inference pipelines, and model serving strategies. How do you integrate the ranking model with the generative summarization engine?
2. Feature Engineering: What features are critical for ranking priority emails, and how do you handle features derived from long email threads versus short, single messages?
3. Model Strategy: Describe your approach to the ranking model (e.g., Two-tower architecture vs. gradient boosted trees) and the summarization strategy (e.g., small, distilled LLMs vs. API-based models).
4. Feedback & Safety: How do you address the 'filter bubble' effect where the system only shows what the user typically clicks? How do you implement guardrails to prevent PII leakage in generated summaries?