Analytics & Experimentation Interview Questions

Showing 1–30 of 560 results

• AdobeMedium·Data Scientist
  Design Metrics Framework for Adobe Express Performance Evaluation

  This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Design Metrics Framework for Adobe Express Performance Evaluation states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationOnsite
• AffirmEasy·Data Scientist
  Calculate Profit of 4-Month Loan at 30% APR

  This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Calculate Profit of 4-Month Loan at 30% APR states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationTechnical Screen
• AffirmMedium·Data Scientist
  Diagnose Sudden Drop in Credit-Card Approval Rate

  This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Diagnose Sudden Drop in Credit-Card Approval Rate states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationTechnical Screen
• AffirmMedium·Data Analyst
  Investigate Rising Loan Delinquency with Stable Approval Rates

  Diagnose rising 30-day loan delinquency, separate borrower-mix shifts from model drift, and evaluate merchant underwriting changes.

  Analytics & ExperimentationTechnical Screen
• AffirmMedium·Data Analyst
  Investigate a Quarterly Increase in Loan Delinquency

  Investigate rising delinquency through rate definitions, reporting checks, loan vintages, seasoning, portfolio mix, and within-segment deterioration.

  Analytics & ExperimentationTechnical Screen
• AirbnbMedium·Data Scientist
  Define Success Metrics and Experiment Plan for Product Development

  Evaluates product-planning metrics and experiment design for a two-sided marketplace booking funnel. Strong answers define a primary booking metric, diagnostic and guardrail metrics, randomization unit, sample size, marketplace spillovers, and launch criteria before development.

  Analytics & ExperimentationTechnical Screen
• AirbnbMedium·Data Scientist
  Design a network-aware Wi‑Fi badge experiment

  This question evaluates a data scientist's proficiency in experimental design, causal inference under interference (including SUTVA risks), metric specification, power and sample‑size computation, analysis planning, and translating treatment effects into financial decision rules.

  Analytics & ExperimentationTechnical Screen
• AirbnbHard·Data Scientist
  Design an A/B test with causal inference

  This question evaluates skills in experimental design, causal inference, and applied statistics — including estimand selection, sample-size calculation under clustering, integrity monitoring, handling noncompliance and contamination, sequential monitoring, and two-proportion inference — within the Analytics & Experimentation domain for a Data Scientist role. It is commonly asked because interviewers need to assess the ability to design and analyze robust A/B tests under real-world constraints; the prompt requires both conceptual understanding of causal assumptions and practical application of power calculations, diagnostics, and monitoring procedures.

  Analytics & ExperimentationTechnical Screen
• AirbnbMedium·Data Scientist
  Design and Analyze Airbnb Locker Experiment

  This question evaluates a candidate's skills in experimental design and causal inference for marketplace features, including selecting the appropriate randomization unit, defining eligibility and treatment/control arms, and choosing primary, secondary, and guardrail metrics.

  Analytics & ExperimentationTechnical Screen
• AirbnbHard·Data Scientist
  Design and assess an A/B test

  This problem tests your ability to design a statistically sound A/B test for a marketplace platform, including metric selection, power analysis considerations, and handling potential interference effects.

  Analytics & ExperimentationOnsiteA/B TestingExperimental DesignMetrics
• AirbnbHard·Data Scientist
  Design robust primary and guardrail metrics

  This question evaluates proficiency in experiment metric design, guardrail definition, data quality diagnostics, and statistical analysis for A/B testing, including power calculations and sequential monitoring, within the Analytics & Experimentation domain.

  Analytics & ExperimentationTechnical Screen
• AirbnbHard·Data Scientist
  Estimate Causal Impact Using Synthetic Control Methods

  Evaluates causal impact estimation after a full product rollout with no holdout. Strong answers use interrupted time series, synthetic control, Bayesian structural time series, exposure variation, and placebo tests to build and validate a credible counterfactual.

  Analytics & ExperimentationTechnical Screen
• AirbnbHard·Data Scientist
  Estimate impact of global launch without holdout

  This question evaluates a candidate's competence in causal inference, observational study design, and statistical uncertainty quantification for product analytics, specifically estimating lift after a global launch without a holdout while addressing confounding, interference, and operational data-quality issues.

  Analytics & ExperimentationTechnical Screen
• AirbnbMedium·Data Scientist
  Walk Through an Experiment From Design to Decision

  Walk through an end-to-end A/B test for an online marketplace feature, covering metric definition, randomization design, SRM diagnosis, and making a launch decision with conflicting trade-offs.

  Analytics & ExperimentationHR ScreenExperimentationA/B TestingProduct Analytics
• AirtableEasy·Data Scientist
  Diagnose metric anomalies and evaluate new algorithm

  This question tests your ability to systematically investigate a sudden drop in a key product metric and design a robust A/B test to evaluate the impact of a new recommendation algorithm.

  Analytics & ExperimentationTechnical ScreenA/B TestingProduct AnalyticsRoot Cause Analysis
• AirwallexMedium·Data Scientist
  Diagnose a Structural Break in KYC Submission Rate

  Tests your ability to systematically diagnose sudden metric shifts, secular downward trends, and seasonality in a fintech onboarding funnel using structured hypothesis testing and slice analysis.

  Analytics & ExperimentationTechnical Screenmetric-diagnosisfunnel-analysismix-shift
• AirwallexMedium·Data Scientist
  Diagnose a dip in approval/conversion rate

  This Analytics & Experimentation prompt evaluates time-series diagnostics, ratio decomposition (numerator/denominator and traffic mix), segmentation and data-quality triage, and causal inference skills at a staff-level applied analytics abstraction.

  Analytics & ExperimentationOnsite
• AirwallexEasy·Data Scientist
  Diagnose conversion-rate time series and CTA swap

  This question evaluates a data scientist's competency in time-series diagnosis, conversion-rate decomposition, causal inference, and A/B experiment design within the Analytics & Experimentation category of Data Science.

  Analytics & ExperimentationOnsite
• AirwallexMedium·Data Scientist
  How would you measure billboard impact?

  This problem tests your ability to design a quasi-experimental framework for measuring the offline impact of an OOH (Out-of-Home) advertising campaign on digital conversion metrics.

  Analytics & ExperimentationOnsiteCausal InferenceMetricsExperimental Design
• AirwallexMedium·Data Scientist
  How would you test swapping two CTA buttons?

  This question evaluates A/B test design, causal inference, funnel metric selection and diagnostic/guardrail thinking within Analytics & Experimentation for a Data Scientist role, and is commonly asked to gauge the ability to reason about user behavior, conversion trade-offs, and risk management in sensitive domains like fintech.

  Analytics & ExperimentationOnsite
• AirwallexMedium·Data Scientist
  Investigate Harassment Surge and Mitigation

  An Airwallex Data Scientist onsite analytics question: a monthly moderation report shows the Harassment violation type spiking, and you must decide whether the surge is real or a measurement artifact, then propose and test mitigations. It probes diagnostic analytics, prevalence-metric design, denominator effects, Simpson's paradox, selection bias, label drift, model calibration, causal inference (ITS/DiD), and experiment design with guardrails.

  Analytics & ExperimentationOnsite
• AmazonHard·Data Scientist
  Analyze an A/B test over last 7 days

  This question evaluates competency in A/B test analysis and experiment design, covering conversion-rate computation, statistical inference (two-proportion testing and confidence intervals), sample-ratio checks, heterogeneity assessment (fixed versus random effects), and power/MDE reasoning.

  Analytics & ExperimentationOnsite
• AmazonMedium·Software Engineer
  Brainstorm a business problem approach

  Brainstorm a business problem approach evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationTechnical Screen
• AmazonMedium·Data Scientist
  Choose Effective Graphs for Data Exploration

  Evaluates chart selection for exploratory data analysis across categorical and numerical variables. Strong answers explain pie charts, distributions, comparisons, relationships, missingness, and when visuals mislead.

  Analytics & ExperimentationOnsite
• AmazonHard·Data Scientist
  Compare Tableau live vs extract and filters

  This question evaluates competency in BI data architecture and large-scale dashboard performance, covering extract vs live connections, refresh and incremental strategies, query pushdown versus in-tool processing, row-level security, filter order-of-operations, performance tuning, and platform migration mapping within the Analytics & Experimentation domain for a Data Scientist role. It is commonly asked to assess trade-off reasoning across data freshness, storage, query performance and security for a 100M-row fact table, combining practical application-level decisions (implementation and tuning) with conceptual understanding (filter ordering, calculations, and migration pitfalls) when moving between BI platforms.

  Analytics & ExperimentationOnsite
• AmazonHard·Data Scientist
  Design A/B Test for New Amazon Recommendation Module

  Design A/B Test for New Amazon Recommendation Module evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationOnsite
• AmazonHard·Business Intelligence Engineer
  Design a One-Page Warehouse Inbound Dashboard

  Design an executive warehouse inbound dashboard with throughput, dock-to-stock time, aged backlog, capacity, and quality metrics.

  Analytics & ExperimentationTechnical Screen
• AmazonMedium·Data Scientist
  Design an A/B Test for Dashboard Engagement Impact

  Amazon data scientist experimentation prompt on designing an A/B test for an energy-usage dashboard redesign, including hypotheses, metrics, guardrails, power, test duration, seasonality, and analysis methods.

  Analytics & ExperimentationOnsite
• AmazonHard·Data Scientist
  Design an operations dashboard with justifications

  This question evaluates a data scientist's ability to define precise operational metrics, justify visualization choices, design interactive drill-downs and alerting, and architect aggregation strategies for high-volume same-day delivery events.

  Analytics & ExperimentationOnsite
• AmazonHard·Data Scientist
  Design and analyze pricing-page A/B test

  This question evaluates data-science competencies in online experiment design and analysis, including statistical power and sample-size estimation, variance-reduction methods, guardrail and SRM monitoring, randomization and contamination handling, sequential monitoring and stopping rules, heterogeneity/subgroup analysis, and result communication within the analytics & experimentation domain. It is commonly asked to assess practical application of statistical inference and experimental rigor in product and business contexts, primarily testing practical application while requiring conceptual understanding of underlying statistical principles and operational trade-offs.

  Analytics & ExperimentationOnsite