Explain production model drop to a PM

Role Play: Communicate Model Degradation to a Non-Technical PM

You are a data scientist. A PM says:

“Your model performed great on the validation set, but after we shipped it, real-world predictions got worse. Why did this happen?”

In a 5–10 minute explanation suitable for a non-technical audience:

1. Give plausible causes for the gap between offline validation and online performance.
2. Describe a debugging plan (what you would check first, what data you need).
3. Propose mitigations and how you would prevent recurrence (monitoring, retraining, guardrails).

Assume the PM cares about user impact, timelines, and what trade-offs are involved.