Topic
Analytics & Experimentation interview questions
560 Analytics & Experimentation questions from 84 companies, with hints, runnable code and solution walkthroughs.
74 easy288 medium198 hard
Analytics & Experimentation interview questions by the numbers
Last updated
Katkat has 560 Analytics & Experimentation interview questions from 84 companies; the most come from Meta (21%), Capital One (11%) and DoorDash (9%). By difficulty: 13% easy, 51% medium and 35% hard. The most common round is Technical Screen (55%). Frequent concepts include Causal Inference, A/B Testing, Experimentation and Metrics.
Companies (84)
Difficulty
- Easy13% · 74
- Medium51% · 288
- Hard35% · 198
Interview rounds
- Technical Screen55% · 306
- Onsite39% · 218
- Online Assessment3% · 16
- HR Screen3% · 14
- Other1% · 6
Roles
- Data Scientist92% · 513
- Software Engineer2% · 13
- Data Analyst2% · 10
- Product Analyst1% · 7
- Data Engineer1% · 5
Frequent concepts
- Causal Inference13% · 73
- A/B Testing12% · 65
- Experimentation8% · 42
- Metrics7% · 41
- Product Analytics5% · 30
- Experiment Design3% · 18
Most mentioned in interview experiences
- 1.Analyze ad watch-time with Excel pivots2 reports
- 2.Define Ultra success and detect suspicious transactions2 reports
- 3.Define Ultra success metrics and detect suspicious transactions2 reports
- 4.Design a flu-shot A/B/n campaign experiment2 reports
- 5.Design an experiment to evaluate a new ads algorithm2 reports
- 6.Diagnose a dip in approval/conversion rate2 reports
- 7.Estimate impact without experiments and pick variant2 reports
- 8.Evaluate Factors Before Renewing TV-Series Contracts2 reports
- 9.How would you investigate a metric decline?2 reports
- 10.Investigate marketplace metrics and experiment rollout2 reports
Counted from 120 candidate interview experiences on Katkat. How we collect and count
Showing 331–360 of 560 results
Evaluates optimal News Feed ad-load strategy using marginal revenue, user tolerance, and long-term LTV. Strong answers test ad-load levels, segment tolerance, personalize caps, and monitor revenue, retention, and user-experience guardrails.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question tests the ability to design an observational study that identifies associations between a new feature adoption and sales growth while strictly avoiding causal claims in the presence of self-selection bias.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question evaluates a data scientist's skills in revenue modeling, funnel construction, causal inference for bias correction, experimentation scaling, and sensitivity analysis using organic engagement and purchase metrics.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question evaluates competency in product analytics, experimentation design, causal inference, metric definition, and measurement of downstream business and user-quality impacts for an AI-assisted advertising feature.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question assesses the ability to design an A/B test for a new AI feature, focusing on metric definition, experiment design, and mitigating potential trade-offs between advertiser efficiency and ad quality.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question evaluates a data scientist's competency in product analytics, experimental design, causal inference, and launch validation by requiring definition of primary and guardrail metrics, test unit and spillover considerations, power/duration planning, behavioral signal expectations, quasi-experimental alternatives, and scale/readiness criteria. Commonly asked in Analytics & Experimentation interviews because organizations need assurance a pilot can be scaled safely and reliably, it tests practical application of experimental methods alongside conceptual understanding of statistical power, spillover risk, diagnostic checks, and operational concerns such as privacy and abuse monitoring.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Evaluate Impact of Increasing Stranger Content in Feeds states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenEvaluates success metrics and experimentation for Instagram's short-video recommender feed. Strong answers choose a durable user-value metric, understand heavy-tailed engagement distributions, handle metric trade-offs, and outline a safe A/B test with retention, safety, creator, and app-level guardrails.
MetaData ScientistAnalytics & ExperimentationOnsiteEvaluates business viability for a Messenger P2P payments feature. Strong answers define networked adoption and transfer metrics, assess fraud, compliance, support, trust, and unit economics, and propose a beta or experiment design that handles spillovers and contamination.
MetaData ScientistAnalytics & ExperimentationOnsiteThis interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Evaluate New Ad Model with A/B Testing Experiment states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question tests the ability to design an experiment to measure the causal impact of a new notification strategy on account ranking quality and user engagement.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question tests the ability to design an experiment that accounts for spillover effects and user fatigue in a social graph, while proposing causal inference methods for environments where randomization is restricted.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question tests the ability to design a robust A/B test for an ads-ranking system while accounting for interference effects, budget constraints, and conflicting metric signals.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question tests the ability to design an evaluation framework for identifying malicious actors in an ad-tech environment and measuring the performance of automated content generation tools.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question evaluates a data scientist's competency in causal inference, experiment design, funnel metric instrumentation, revenue modeling, sensitivity analysis, and data‑quality triage within the Analytics & Experimentation domain.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenEvaluates Facebook Group health metrics and threaded-comments experiment design. Strong answers define normalized health scores, compare small and large groups fairly, and test deep interaction with safety guardrails.
MetaData ScientistAnalytics & ExperimentationOnsiteEvaluates feed UX experimentation for reducing post-card height and diagnosing divergent geographic revenue outcomes. Strong answers measure scroll efficiency, viewability, engagement, ad revenue, device/network effects, and geo-specific decisions.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenEvaluates Facebook Memories feature performance beyond click-through rate and surface-level engagement. Strong answers measure incremental engagement, retention, cannibalization, quality guardrails, frequency caps, sensitive-content handling, and launch criteria.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question tests the ability to design a measurement framework for evaluating content-quality problems on a large user-generated-content platform. It assesses competency in metric selection, experimentation design, and causal reasoning within the Analytics & Experimentation domain — skills central to data science roles focused on platform health.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Explain why IG Story usage exceeds Facebook states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
MetaData ScientistAnalytics & ExperimentationOnsiteA metrics question asking a product manager to set goals and success metrics for a scam-prevention product across Meta's apps. It tests goal prioritization, the choice and measurement of a north-star metric, leading indicators by scam type and surface, guardrails against over-enforcement, and measurement under adversarial behavior.
MetaProduct ManagerAnalytics & ExperimentationOtherThis question evaluates a data scientist's competency in diagnosing ad-spend anomalies, including data validation, attribution and reporting integrity, segmentation and placement effects, and causal inference within advertising analytics.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenEvaluates how to validate whether friends' content drives stronger social engagement than unconnected content. Strong answers define socialness metrics, control observational bias, test launch impact for unconnected feed content, and monitor discovery, retention, cannibalization, and quality guardrails.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenA Meta Data Scientist analytics-and-experimentation interview question: a US C2C second-hand marketplace is considering opt-in “similar listings” notifications. You must (1) decide whether to build it — hypotheses, opportunity sizing, risks, MVP plan — and (2) design the post-launch measurement: a user-randomized A/B test with a primary/diagnostic/guardrail metric stack, handling of opt-in selection bias, and a clear launch/rollback rule.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis problem tests your ability to design an A/B test for a marketplace notification feature, focusing on balancing buyer engagement with potential cannibalization and ecosystem health metrics.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question evaluates a data scientist's competency in product analytics and experimentation by assessing understanding of retention metric definitions, cohorting strategies, short- versus long-term trade-offs, common measurement pitfalls, and A/B test design considerations.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question evaluates product growth analytics competencies including experimentation design, funnel decomposition, metric definition and guardrails, segmentation, and causal inference within the Analytics & Experimentation domain for social media and mobile product features.
MetaProduct AnalystAnalytics & ExperimentationOnsiteThis question tests the ability to design an experimentation framework for a new feature in a social product, specifically focusing on metric selection, identifying interference, and isolating the causal impact of a feature rollout.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question tests the ability to build a comprehensive evaluation framework for a new feature, specifically focusing on defining success metrics, identifying potential side effects, and designing an experimental setup for pixel-issue notifications.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question tests your ability to design an A/B testing framework for evaluating ad ranking changes, including metric selection, segment analysis, and identifying potential biases in historical data.
MetaData ScientistAnalytics & ExperimentationTechnical Screen