How would you investigate a metric decline?

Scenario

You are a Data Scientist supporting a consumer marketplace product. A key business metric (e.g., orders/day, conversion rate, revenue, profit, retention) has declined noticeably over the last 1–2 weeks.

Task

Describe a structured approach to investigate the decline.

Requirements

In your answer, cover:

1. Clarify the metric

• Exact definition, numerator/denominator, filters, attribution logic.
• Timezone, event time vs processing time, and whether the metric is cohort-based.

1. Detect whether it’s real vs measurement

• Data pipeline issues, logging changes, bot/traffic anomalies, backfills.

1. Localize the change

• When did it start? step vs trend? aligned to releases/experiments?

1. Decompose and segment

• Break down by funnel stage and by key segments (new vs returning, geo, device, acquisition channel, product surfaces, supply vs demand side, etc.).

1. Consider external and seasonal factors

• Seasonality/holidays, competitor actions, macro events, weather, outages.

1. Form hypotheses and propose follow-ups

• Additional analyses, dashboards, and (if appropriate) experiments or mitigations.

Output

Explain the investigation plan and what you would do in the first day vs first week.