Topic
Analytics & Experimentation interview questions
560 Analytics & Experimentation questions from 84 companies, with hints, runnable code and solution walkthroughs.
74 easy288 medium198 hard
Analytics & Experimentation interview questions by the numbers
Last updated
Katkat has 560 Analytics & Experimentation interview questions from 84 companies; the most come from Meta (21%), Capital One (11%) and DoorDash (9%). By difficulty: 13% easy, 51% medium and 35% hard. The most common round is Technical Screen (55%). Frequent concepts include Causal Inference, A/B Testing, Experimentation and Metrics.
Companies (84)
Difficulty
- Easy13% · 74
- Medium51% · 288
- Hard35% · 198
Interview rounds
- Technical Screen55% · 306
- Onsite39% · 218
- Online Assessment3% · 16
- HR Screen3% · 14
- Other1% · 6
Roles
- Data Scientist92% · 513
- Software Engineer2% · 13
- Data Analyst2% · 10
- Product Analyst1% · 7
- Data Engineer1% · 5
Frequent concepts
- Causal Inference13% · 73
- A/B Testing12% · 65
- Experimentation8% · 42
- Metrics7% · 41
- Product Analytics5% · 30
- Experiment Design3% · 18
Most mentioned in interview experiences
- 1.Analyze ad watch-time with Excel pivots2 reports
- 2.Define Ultra success and detect suspicious transactions2 reports
- 3.Define Ultra success metrics and detect suspicious transactions2 reports
- 4.Design a flu-shot A/B/n campaign experiment2 reports
- 5.Design an experiment to evaluate a new ads algorithm2 reports
- 6.Diagnose a dip in approval/conversion rate2 reports
- 7.Estimate impact without experiments and pick variant2 reports
- 8.Evaluate Factors Before Renewing TV-Series Contracts2 reports
- 9.How would you investigate a metric decline?2 reports
- 10.Investigate marketplace metrics and experiment rollout2 reports
Counted from 120 candidate interview experiences on Katkat. How we collect and count
Showing 271–300 of 560 results
This question tests your ability to design an experiment to measure the incremental lift of a coupon program and your strategy for segmenting users based on commuting behaviors to maximize ROI.
LyftData ScientistAnalytics & ExperimentationTechnical ScreenInvestigate Causes of Driver WOW Score Drop evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
LyftData ScientistAnalytics & ExperimentationOnsiteThis question tests your ability to diagnose a sudden drop in a core business metric, distinguish between technical tracking issues and product-driven behavior, and design a strategy to evaluate coupon effectiveness.
LyftData ScientistAnalytics & ExperimentationTechnical ScreenInvestigate Sudden Metric Changes and Design A/B Test evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
LyftData ScientistAnalytics & ExperimentationTechnical ScreenTests your ability to conduct a structured root-cause analysis on simultaneous marketplace metric regressions, decomposing supply-demand dynamics and separating correlation from causation.
LyftData ScientistAnalytics & ExperimentationTechnical ScreenAnalyze Algorithm's Impact on Diverse Demographics and Validate Causes evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
MetaData ScientistAnalytics & ExperimentationOnsiteThis interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Analyze Change in App Metrics and Feature Impact states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
MetaData ScientistAnalytics & ExperimentationOnsiteThis interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Analyze Comment Distribution Using Statistical Metrics and Tests states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
MetaData ScientistAnalytics & ExperimentationOnsiteThis interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Analyze Key Metrics for Notification System Success states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
MetaData ScientistAnalytics & ExperimentationOnsiteEvaluates revenue cannibalization analysis across creation sources. Strong answers define Source A, other-source revenue, total revenue, cannibalization ratio, and net lift, then use panel, rollout, difference-in-differences, or synthetic-control evidence to separate source shifting from organic growth.
MetaData ScientistAnalytics & ExperimentationOnsiteEvaluates reporting-system analytics for valid reports and misuse detection in moderation workflows. Strong answers define distinct reported-user share, write SQL for valid reports, and detect abusive reporters through frequency, hit-rate, burst, and coordination metrics.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question evaluates a candidate's competency in causal inference, experimentation design, metric definition, and observational analytics for product ranking and engagement problems within the Analytics & Experimentation domain for Data Scientist roles.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenEvaluates whether a proposed News Feed proxy metric is meaningful for long-term user value. Strong answers test predictive power, actionability, gaming risk, true-outcome alignment, collection cost, privacy, latency, and decision criteria.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question evaluates a data scientist's competency in lifecycle email optimization, attribution of incremental on-site engagement, experiment design, metric selection, and trade-offs involving deliverability and guardrails within the Analytics & Experimentation domain.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question evaluates a data scientist's competency in product analytics and experimentation, focusing on dashboard specification, precise event- and user-level metric definitions, core dimensions, data freshness SLAs, and instrumentation within the Analytics & Experimentation domain.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question evaluates proficiency in causal inference and experimentation analytics for a Data Scientist role, focusing on selecting appropriate identification strategies, designing placebo and pre‑trend tests, and specifying robustness and sensitivity analyses for non‑randomized feature adoption.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question evaluates a candidate's competency in experimental design, A/B testing and statistical analysis for product decisions, specifically testing metric selection, reliability and latency guardrails, stratified randomization, interference and clustering effects, heterogeneity detection, and operational stopping rules.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Compare Instagram and Facebook Stories Using Key Performance Metrics states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question evaluates causal inference, experimentation design, metric engineering, and ecosystem-level impact assessment skills for a Data Scientist in the Analytics & Experimentation domain, with emphasis on defining a single-objective OEC that prevents pure cannibalization and on measuring cross-app effects via randomized rollouts.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question evaluates a data scientist's proficiency in product analytics, causal inference, metric engineering, experimentation design, and post‑launch diagnostics within the Analytics & Experimentation domain.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question tests the ability to design a robust A/B test in the presence of network effects, interference, and cannibalization within a social notification ecosystem.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question evaluates understanding of A/B experimentation, auction-based ad metrics, metric decomposition for revenue impact, causal diagnostics for hidden heterogeneity (e.g., Simpson’s paradox), and executive-level results presentation within the Analytics & Experimentation domain.
MetaData ScientistAnalytics & ExperimentationOnsiteDesign a comprehensive measurement and experimentation framework for a new community feature, evaluating true incremental value while detecting cannibalization across existing surfaces and mitigating network spillover effects.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenDefine and validate product metrics evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
MetaData EngineerAnalytics & ExperimentationOnsiteThis question evaluates a data scientist's competency in metric framework design, distributional analysis of heavy-tailed user behavior, monitoring and anomaly detection, and experimental setup for measuring healthy engagement, including considerations like unit of analysis, treatment of heavy users, and anti-spam guardrails.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question evaluates a data scientist's competency in experimental design, metric selection and guardrails, power and sample-size calculations, logging and attribution specifications, and revenue impact estimation for a new feature on a social media platform.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question evaluates a data scientist's ability to design product success metrics and a rigorous experiment plan, covering KPI formulation, guardrail metrics and thresholds, randomization and experiment design, logging instrumentation, and quasi-experimental causal inference.
MetaData ScientistAnalytics & ExperimentationOnsiteMeta data scientist experimentation prompt on measuring Instagram Shopping impact with A/B tests, geo or cluster holdouts, engagement and revenue metrics, guardrails, selection bias, network effects, and rollout decisions.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question evaluates a data scientist's skills in experimental design, causal inference, clustering and interference analysis, metric specification, and power/sample-size calculation within the Analytics & Experimentation domain for Data Scientist roles.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Design Metrics to Track and Analyze Spam Impact states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
MetaData ScientistAnalytics & ExperimentationTechnical Screen