Analytics & Experimentation Interview Questions

Showing 151–180 of 218 results

• MetaMedium·Data Scientist
  Measure scheduled posts feature success

  This question evaluates proficiency in Bayesian statistical estimation, experimental design, product metric definition, and causal/confounder analysis for measuring feature reliability and engagement.

  Analytics & ExperimentationOnsite
• MetaHard·Data Scientist
  Prove source growth is cannibalization, not incremental

  This question evaluates causal inference and experimental-design competencies in analytics, testing skills in identification strategy, treatment and control definition, specification of difference‑in‑differences or randomized geo experiments, metric construction, power calculations, and robustness checks for attribution and cannibalization analysis. It is commonly asked to determine whether observed growth is incremental or substitution-driven, falls under the Analytics & Experimentation domain in Data Science, and emphasizes practical application accompanied by conceptual understanding of identification assumptions, statistical power, and measurement.

  Analytics & ExperimentationOnsite
• MetaMedium·Data Scientist
  Redesign an executive dashboard for C-suite

  This question evaluates skills in dashboard design, data visualization, metric selection, and executive-level analytical communication for product and growth measurement.

  Analytics & ExperimentationOnsite
• MetaMedium·Data Scientist
  Reduce variance with covariate adjustment

  This question evaluates a data scientist's competency in experimental design, variance reduction techniques, and regression-based covariate adjustment (CUPED/ANCOVA) within randomized A/B tests, including understanding of statistical power and diagnostic checks.

  Analytics & ExperimentationOnsite
• MetaMedium·Data Scientist
  Run a clean A/B test for recommendations

  This question evaluates a data scientist's competency in experimental design, causal inference, and statistical power analysis for A/B tests, covering randomization unit choice, contamination prevention, metric and guardrail selection, clustering adjustments, and bias-control techniques.

  Analytics & ExperimentationOnsite
• MetaMedium·Data Scientist
  Size opportunity for new product line

  This question evaluates a data scientist's skills in financial opportunity sizing, sensitivity analysis, and experimental design by requiring quantitative modeling of impacted users, incremental orders, revenue, and gross profit for a proposed product line.

  Analytics & ExperimentationOnsite
• MetaHard·Data Scientist
  Test if social users are more engaged

  This problem tests your ability to design an analytical framework to measure user engagement, account for selection bias in non-randomized groups, and define robust metrics for comparing heterogeneous user populations.

  Analytics & ExperimentationOnsiteAnalyticsCausal InferenceSelection Bias
• MetaMedium·Data Scientist
  Track Metrics to Measure Push Notification Quality

  Track Metrics to Measure Push Notification Quality evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationOnsite
• MetaMedium·Data Engineer
  Visualize Netflix metric trends

  This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Visualize Netflix metric trends states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationOnsite
• MetaMedium·Data Scientist
  [Analytical Reasoning] Comparing Two Newsfeed Ad Insertion Methods

  Compare independent versus blocked ad insertion in a 100-post feed. Includes expected ads, tail risk, consecutive ad probabilities, longest streaks, product tradeoffs, and A/B test design.

  Analytics & ExperimentationOnsite
• MicrosoftMedium·Data Scientist
  Evaluate a Product Change That Adds More User Options

  This question tests the ability to design an A/B test for a feature introducing choice-overload, specifically focusing on metric selection, trade-off analysis, and decision-making frameworks when metrics diverge.

  Analytics & ExperimentationOnsiteExperimentationA/B TestingProduct Metrics
• NetflixMedium·Data Scientist
  Compute ITT, TOT, and LATE with noncompliance

  This question evaluates understanding of causal inference and experimental analysis, specifically estimation and interpretation of ITT, TOT, and LATE under noncompliance along with classification of compliance types (compliers, always-takers, never-takers, defiers).

  Analytics & ExperimentationOnsite
• NetflixMedium·Data Scientist
  Estimate ATE of personalization on streaming

  This question evaluates causal inference and experimental-analysis competencies by asking for estimation of the average treatment effect (ATE) of personalization on minutes streamed, reporting a 95% confidence interval, and reasoning about the use of pre-treatment covariates, and it sits in the Analytics & Experimentation domain for a Data Scientist role. It is commonly asked to assess application of randomized experiment analysis and statistical inference while probing conceptual understanding of causal assumptions and covariate adjustment, testing both conceptual understanding and practical application.

  Analytics & ExperimentationOnsite
• NetflixEasy·Data Scientist
  Estimate ATE, ITT, and TOT from experiment

  This question tests the ability to distinguish and calculate Average Treatment Effect (ATE), Intent-to-Treat (ITT), and Treatment-on-the-Treated (TOT) in the presence of non-compliance.

  Analytics & ExperimentationOnsiteCausal InferenceExperimentationInstrumental Variables
• NetflixMedium·Data Scientist
  How to Design Effective A/B Tests for Onboarding

  Evaluates A/B test design for a redesigned onboarding flow in a subscription app. Strong answers define activation, user-level randomization, primary and guardrail metrics, sample size, sequential monitoring, and decisions when activation improves but support burden rises.

  Analytics & ExperimentationOnsite
• OneMain FinancialMedium·Data Scientist
  Calculate Profit-Maximizing Price and Validate with Additional Data

  Evaluates profit-maximizing pricing from demand curves, marginal revenue, marginal cost, and validation data. Strong answers derive optimal price and quantity, provide examples, and request experiments or elasticity evidence.

  Analytics & ExperimentationOnsite
• OpenAIHard·Data Scientist
  Design and Defend a Free-Trial Conversion Analysis

  Walk through how to plan, execute, and defend a free-trial conversion analysis end to end: cohort definition, data-quality checks, standards for a credible insight, and a recommendation deck that survives live pushback.

  Analytics & ExperimentationOnsiteexperimentationmetricscase-study
• OOtherHard·Data Scientist
  Design an A/B test with guardrails

  This question evaluates a data scientist's skills in experimental design, hypothesis testing, statistical power and sample-size calculation, randomization and contamination control, pre-registration planning, and multiplicity adjustment for A/B testing.

  Analytics & ExperimentationOnsite
• OOtherMedium·Data Scientist
  Separate demand from supply for jeans

  This question evaluates a data scientist's ability to perform data-driven diagnostics, funnel-metric construction, quantitative decision-rule design, and experimentation planning to distinguish demand-side from supply-side constraints in a marketplace jeans category.

  Analytics & ExperimentationOnsite
• PayPalMedium·Data Scientist
  Analyze Transactions for Risk and Implement Mitigation Strategies

  Analyze Transactions for Risk and Implement Mitigation Strategies evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationOnsite
• PayPalHard·Data Scientist
  Assess card transactions and plan risk strategy

  Assess card transactions and plan risk strategy evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationOnsite
• PayPalMedium·Data Scientist
  Boost User Login Rate: Key Metrics to Monitor

  This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Boost User Login Rate: Key Metrics to Monitor states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationOnsite
• PayPalMedium·Data Scientist
  Design and Analyze A/B Test for Cashback Program

  This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Design and Analyze A/B Test for Cashback Program states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationOnsite
• PayPalEasy·Data Scientist
  Design metrics and experiment for donation feature

  A PayPal data science onsite case asking you to design metrics and a randomized experiment for an optional checkout donation feature on a food-ordering platform. Covers goal and hypothesis framing, a primary/diagnostic/guardrail metric hierarchy with precise definitions, user-level experiment design with power and variance-reduction, confounder/marketplace-bias handling, and a launch decision framework.

  Analytics & ExperimentationOnsite
• PayPalMedium·Data Scientist
  Master A/B Testing: Key Concepts and Methodologies Explained

  Evaluates practical A/B testing and causal inference fundamentals for online products. Strong answers cover p-values, errors, power, workflow, Simpson's paradox, metrics, and observational causal methods.

  Analytics & ExperimentationOnsite
• PayPalEasy·Data Scientist
  Reduce airport ride cancellations under causal constraints

  A PayPal Data Scientist onsite product-sense and causal-inference case: design a measurement framework, hypotheses, and experimentation strategy to reduce airport ride cancellations in a ride-hailing marketplace without harming marketplace health. It tests metric trees, interference-aware experiment design (switchback, partial-population, encouragement/IV), satisfaction proxies, confounder handling, and stakeholder communication.

  Analytics & ExperimentationOnsite
• PinterestHard·Data Scientist
  Analyze a geo rollout and interpret charts

  This question evaluates causal inference and product analytics competencies — specifically specification of causal estimands and difference-in-differences, segmented (interrupted) time-series regression, power and duration calculations, spillover and donor-pool diagnostics, and selection of guardrail metrics.

  Analytics & ExperimentationOnsite
• PinterestHard·Data Scientist
  Decide if ad load is optimized

  This question evaluates competency in analytics and experimentation, covering experiment design, metric and guardrail definition, causal inference, statistical testing, segmentation for heterogeneous effects, and balancing user experience versus monetization.

  Analytics & ExperimentationOnsite
• PinterestHard·Data Scientist
  Diagnose CTR drop after recommendation launch

  This question evaluates a data scientist's competency in experiment diagnosis, metric and guardrail definition, instrumentation validation, segmentation analysis, and causal reasoning for recommender-system impacts.

  Analytics & ExperimentationOnsite
• PinterestMedium·Data Scientist
  Estimate a Launch Impact with Difference in Differences

  Tests your ability to design and evaluate a regional quasi-experiment using Difference-in-Differences (DiD), validate key identifying assumptions, structure robust inference, and address modern complications such as staggered rollout and network spillovers.

  Analytics & ExperimentationOnsiteDifference-in-DifferencesCausal InferenceQuasi-Experiments