Topic
Analytics & Experimentation interview questions
560 Analytics & Experimentation questions from 84 companies, with hints, runnable code and solution walkthroughs.
74 easy288 medium198 hard
Analytics & Experimentation interview questions by the numbers
Last updated
Katkat has 560 Analytics & Experimentation interview questions from 84 companies; the most come from Meta (21%), Capital One (11%) and DoorDash (9%). By difficulty: 13% easy, 51% medium and 35% hard. The most common round is Technical Screen (55%). Frequent concepts include Causal Inference, A/B Testing, Experimentation and Metrics.
Companies (84)
Difficulty
- Easy13% · 74
- Medium51% · 288
- Hard35% · 198
Interview rounds
- Technical Screen55% · 306
- Onsite39% · 218
- Online Assessment3% · 16
- HR Screen3% · 14
- Other1% · 6
Roles
- Data Scientist92% · 513
- Software Engineer2% · 13
- Data Analyst2% · 10
- Product Analyst1% · 7
- Data Engineer1% · 5
Frequent concepts
- Causal Inference13% · 73
- A/B Testing12% · 65
- Experimentation8% · 42
- Metrics7% · 41
- Product Analytics5% · 30
- Experiment Design3% · 18
Most mentioned in interview experiences
- 1.Analyze ad watch-time with Excel pivots2 reports
- 2.Define Ultra success and detect suspicious transactions2 reports
- 3.Define Ultra success metrics and detect suspicious transactions2 reports
- 4.Design a flu-shot A/B/n campaign experiment2 reports
- 5.Design an experiment to evaluate a new ads algorithm2 reports
- 6.Diagnose a dip in approval/conversion rate2 reports
- 7.Estimate impact without experiments and pick variant2 reports
- 8.Evaluate Factors Before Renewing TV-Series Contracts2 reports
- 9.How would you investigate a metric decline?2 reports
- 10.Investigate marketplace metrics and experiment rollout2 reports
Counted from 120 candidate interview experiences on Katkat. How we collect and count
Showing 361–390 of 560 results
This question assesses your ability to systematically approach product growth by identifying bottlenecks, framing hypotheses, and designing experiments to improve key user engagement metrics.
MetaProduct AnalystAnalytics & ExperimentationOnsiteThis question evaluates a data scientist's ability to design a measurement framework encompassing product goal definition, north-star metric selection, success metrics for adoption, engagement and quality, precise retention definitions (app-level vs feature-specific and creator vs participant), cohort retention windows, and experimental design including guardrails, bias sources and network effects. It is asked in the Analytics & Experimentation domain to assess practical application of product analytics and quantitative reasoning about short-term versus long-term tradeoffs, segmentation and experimental validity, operating at a primarily practical application level with necessary conceptual understanding.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThe question evaluates a data scientist's competency in triangulating product demand from non-table sources, defining and de-biasing signals, specifying metrics and guardrails, and designing evidence-driven experiments within the Analytics & Experimentation domain for a Data Scientist role.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenEvaluates confidence-interval interpretation for choosing among feed-ranking treatments with engagement losses. Strong answers compare statistical significance, magnitude, uncertainty, business thresholds, downside risk, and launch recommendations.
MetaData ScientistAnalytics & ExperimentationOnsiteEvaluates diagnostic analytics for explaining why Instagram users consume more Stories than Facebook users. Strong answers validate metric parity, decompose supply and demand drivers, use cohort and within-user comparisons, and propose experiments for UI, ranking, content, or notification hypotheses.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question tests your ability to systematically isolate the root cause of a drop in advertising spend by decomposing metrics and evaluating external versus internal factors.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question evaluates a data scientist's product analytics, experimental design, and strategic decision-making skills, covering market sizing, competitive analysis, privacy-aware data and infrastructure assessment, risk identification, and A/B testing planning.
MetaData ScientistAnalytics & ExperimentationOnsiteDefine gaming monetization metrics, separate business changes from data failures, and validate repairs with reconciliation and guardrails.
MetaData EngineerAnalytics & ExperimentationTechnical ScreenThis interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Measure Harmful Content Impact with Key Metrics states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question evaluates experiment design, metric selection, causal inference and business-metric translation for social features, including defining primary and guardrail metrics, choosing experimental units to mitigate network interference, and estimating revenue impact from observed lifts.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question tests the ability to design an experimentation framework for notification systems while accounting for user fatigue, long-term ecosystem health, and signal interference.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question evaluates experiment design and causal inference skills for platform analytics, covering interference management, metric and guardrail definition, power and duration calculations, eligibility and exclusions, novelty/adaptation considerations, and diagnostics for bot‑mitigation interventions.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question evaluates proficiency in Bayesian statistical estimation, experimental design, product metric definition, and causal/confounder analysis for measuring feature reliability and engagement.
MetaData ScientistAnalytics & ExperimentationOnsiteA product manager metrics question on judging whether Meta's generative AI features, such as its in-app assistant, creation tools and ad tools, are succeeding. It tests choosing a north-star metric that reflects real value, supporting and guardrail metrics, and how to show the AI actually caused any change.
MetaProduct ManagerAnalytics & ExperimentationOther- Prove friends outperform unconnected; design metrics, observational analysis, and rollout experimentHard
A Meta data-scientist technical-screen question on Analytics & Experimentation: using only info_stream_views and post_reactions, prove whether Friend-authored content is 'more social' than Unconnected content. It tests denominator-complete metric design with relationship attribution, observational causal validation (fixed effects + propensity/AIPW), and a network-aware rollout experiment with power, CUPED, interference mitigations, and guardrails — plus quantifying the long-term and discovery value of Unconnected content beyond near-term engagement.
MetaData ScientistAnalytics & ExperimentationTechnical Screen This question tests your ability to design a rigorous causal experiment to validate a product hypothesis while accounting for selection bias, confounding variables, and operational constraints.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question evaluates causal inference and experimental-design competencies in analytics, testing skills in identification strategy, treatment and control definition, specification of difference‑in‑differences or randomized geo experiments, metric construction, power calculations, and robustness checks for attribution and cannibalization analysis. It is commonly asked to determine whether observed growth is incremental or substitution-driven, falls under the Analytics & Experimentation domain in Data Science, and emphasizes practical application accompanied by conceptual understanding of identification assumptions, statistical power, and measurement.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question evaluates skills in dashboard design, data visualization, metric selection, and executive-level analytical communication for product and growth measurement.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question evaluates a data scientist's competency in experimental design, variance reduction techniques, and regression-based covariate adjustment (CUPED/ANCOVA) within randomized A/B tests, including understanding of statistical power and diagnostic checks.
MetaData ScientistAnalytics & ExperimentationOnsiteA Meta product manager interview question about Instagram, where ad revenue has risen while user satisfaction has fallen. It tests validating both metrics, decomposing revenue and satisfaction to find a shared cause, testing hypotheses with holdouts, and weighing short-term revenue against long-term user health.
MetaProduct ManagerAnalytics & ExperimentationOtherThis question evaluates a data scientist's competency in experimental design, causal inference, and statistical power analysis for A/B tests, covering randomization unit choice, contamination prevention, metric and guardrail selection, clustering adjustments, and bias-control techniques.
MetaData ScientistAnalytics & ExperimentationOnsiteThis question evaluates a candidate's ability to design a robust metrics framework for product experiments, covering selection and normalization of success metrics, feature-only sizing and ecosystem metrics, explicit cannibalization reads, guardrails, multiple-testing control, and back-of-the-envelope LTV/ARPDAU impact estimation.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenA Meta Data Scientist technical-screen case: using only one-to-one call logs, decide whether to build a group calling feature and how to measure a launch's success. It tests product analytics, proxy-metric design, network analysis, causal inference, and experiment design under social-network interference.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis question evaluates a data scientist's skills in financial opportunity sizing, sensitivity analysis, and experimental design by requiring quantitative modeling of impacted users, incremental orders, revenue, and gross profit for a proposed product line.
MetaData ScientistAnalytics & ExperimentationOnsiteThis problem tests your ability to design an analytical framework to measure user engagement, account for selection bias in non-randomized groups, and define robust metrics for comparing heterogeneous user populations.
MetaData ScientistAnalytics & ExperimentationOnsiteTrack Metrics to Measure Push Notification Quality evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
MetaData ScientistAnalytics & ExperimentationOnsiteEvaluates success and guardrail metrics for a push-notification feature in a networked travel app. Strong answers measure incremental itinerary engagement and retention, monitor opt-outs and fatigue, and design a cluster or graph-aware experiment to reduce spillovers.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenMeta product analytics prompt on group calling, covering user-needs surveys, competitor benchmarking without direct data, experiment design, network effects, contamination, metrics, and guardrails.
MetaData ScientistAnalytics & ExperimentationTechnical ScreenThis interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Visualize Netflix metric trends states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
MetaData EngineerAnalytics & ExperimentationOnsiteCompare independent versus blocked ad insertion in a 100-post feed. Includes expected ads, tail risk, consecutive ad probabilities, longest streaks, product tradeoffs, and A/B test design.
MetaData ScientistAnalytics & ExperimentationOnsite