Analytics & Experimentation interview questions

Showing 1–30 of 560 results

• Design Metrics Framework for Adobe Express Performance EvaluationMedium

  This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Design Metrics Framework for Adobe Express Performance Evaluation states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  AdobeData ScientistAnalytics & ExperimentationOnsite
• Calculate Profit of 4-Month Loan at 30% APREasy

  This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Calculate Profit of 4-Month Loan at 30% APR states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  AffirmData ScientistAnalytics & ExperimentationTechnical Screen
• Diagnose Sudden Drop in Credit-Card Approval RateMedium

  This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Diagnose Sudden Drop in Credit-Card Approval Rate states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  AffirmData ScientistAnalytics & ExperimentationTechnical Screen
• Investigate Rising Loan Delinquency with Stable Approval RatesMedium

  Diagnose rising 30-day loan delinquency, separate borrower-mix shifts from model drift, and evaluate merchant underwriting changes.

  AffirmData AnalystAnalytics & ExperimentationTechnical Screen
• Investigate a Quarterly Increase in Loan DelinquencyMedium

  Investigate rising delinquency through rate definitions, reporting checks, loan vintages, seasoning, portfolio mix, and within-segment deterioration.

  AffirmData AnalystAnalytics & ExperimentationTechnical Screen
• Define Success Metrics and Experiment Plan for Product DevelopmentMedium

  Evaluates product-planning metrics and experiment design for a two-sided marketplace booking funnel. Strong answers define a primary booking metric, diagnostic and guardrail metrics, randomization unit, sample size, marketplace spillovers, and launch criteria before development.

  AirbnbData ScientistAnalytics & ExperimentationTechnical Screen
• Design a network-aware Wi‑Fi badge experimentMedium

  This question evaluates a data scientist's proficiency in experimental design, causal inference under interference (including SUTVA risks), metric specification, power and sample‑size computation, analysis planning, and translating treatment effects into financial decision rules.

  AirbnbData ScientistAnalytics & ExperimentationTechnical Screen
• Design an A/B test with causal inferenceHard

  This question evaluates skills in experimental design, causal inference, and applied statistics — including estimand selection, sample-size calculation under clustering, integrity monitoring, handling noncompliance and contamination, sequential monitoring, and two-proportion inference — within the Analytics & Experimentation domain for a Data Scientist role. It is commonly asked because interviewers need to assess the ability to design and analyze robust A/B tests under real-world constraints; the prompt requires both conceptual understanding of causal assumptions and practical application of power calculations, diagnostics, and monitoring procedures.

  AirbnbData ScientistAnalytics & ExperimentationTechnical Screen
• Design and Analyze Airbnb Locker ExperimentMedium

  This question evaluates a candidate's skills in experimental design and causal inference for marketplace features, including selecting the appropriate randomization unit, defining eligibility and treatment/control arms, and choosing primary, secondary, and guardrail metrics.

  AirbnbData ScientistAnalytics & ExperimentationTechnical Screen
• Design and assess an A/B testHard

  This problem tests your ability to design a statistically sound A/B test for a marketplace platform, including metric selection, power analysis considerations, and handling potential interference effects.

  AirbnbData ScientistAnalytics & ExperimentationOnsite
• Design robust primary and guardrail metricsHard

  This question evaluates proficiency in experiment metric design, guardrail definition, data quality diagnostics, and statistical analysis for A/B testing, including power calculations and sequential monitoring, within the Analytics & Experimentation domain.

  AirbnbData ScientistAnalytics & ExperimentationTechnical Screen
• Estimate Causal Impact Using Synthetic Control MethodsHard

  Evaluates causal impact estimation after a full product rollout with no holdout. Strong answers use interrupted time series, synthetic control, Bayesian structural time series, exposure variation, and placebo tests to build and validate a credible counterfactual.

  AirbnbData ScientistAnalytics & ExperimentationTechnical Screen
• Estimate impact of global launch without holdoutHard

  This question evaluates a candidate's competence in causal inference, observational study design, and statistical uncertainty quantification for product analytics, specifically estimating lift after a global launch without a holdout while addressing confounding, interference, and operational data-quality issues.

  AirbnbData ScientistAnalytics & ExperimentationTechnical Screen
• Walk Through an Experiment From Design to DecisionMedium

  Walk through an end-to-end A/B test for an online marketplace feature, covering metric definition, randomization design, SRM diagnosis, and making a launch decision with conflicting trade-offs.

  AirbnbData ScientistAnalytics & ExperimentationHR Screen
• Diagnose metric anomalies and evaluate new algorithmEasy

  This question tests your ability to systematically investigate a sudden drop in a key product metric and design a robust A/B test to evaluate the impact of a new recommendation algorithm.

  AirtableData ScientistAnalytics & ExperimentationTechnical Screen
• Diagnose a Structural Break in KYC Submission RateMedium

  Tests your ability to systematically diagnose sudden metric shifts, secular downward trends, and seasonality in a fintech onboarding funnel using structured hypothesis testing and slice analysis.

  AirwallexData ScientistAnalytics & ExperimentationTechnical Screen
• Diagnose a dip in approval/conversion rateMedium

  This Analytics & Experimentation prompt evaluates time-series diagnostics, ratio decomposition (numerator/denominator and traffic mix), segmentation and data-quality triage, and causal inference skills at a staff-level applied analytics abstraction.

  AirwallexData ScientistAnalytics & ExperimentationOnsite
• Diagnose conversion-rate time series and CTA swapEasy

  This question evaluates a data scientist's competency in time-series diagnosis, conversion-rate decomposition, causal inference, and A/B experiment design within the Analytics & Experimentation category of Data Science.

  AirwallexData ScientistAnalytics & ExperimentationOnsite
• How would you measure billboard impact?Medium

  This problem tests your ability to design a quasi-experimental framework for measuring the offline impact of an OOH (Out-of-Home) advertising campaign on digital conversion metrics.

  AirwallexData ScientistAnalytics & ExperimentationOnsite
• How would you test swapping two CTA buttons?Medium

  This question evaluates A/B test design, causal inference, funnel metric selection and diagnostic/guardrail thinking within Analytics & Experimentation for a Data Scientist role, and is commonly asked to gauge the ability to reason about user behavior, conversion trade-offs, and risk management in sensitive domains like fintech.

  AirwallexData ScientistAnalytics & ExperimentationOnsite
• Investigate Harassment Surge and MitigationMedium

  An Airwallex Data Scientist onsite analytics question: a monthly moderation report shows the Harassment violation type spiking, and you must decide whether the surge is real or a measurement artifact, then propose and test mitigations. It probes diagnostic analytics, prevalence-metric design, denominator effects, Simpson's paradox, selection bias, label drift, model calibration, causal inference (ITS/DiD), and experiment design with guardrails.

  AirwallexData ScientistAnalytics & ExperimentationOnsite
• Analyze an A/B test over last 7 daysHard

  This question evaluates competency in A/B test analysis and experiment design, covering conversion-rate computation, statistical inference (two-proportion testing and confidence intervals), sample-ratio checks, heterogeneity assessment (fixed versus random effects), and power/MDE reasoning.

  AmazonData ScientistAnalytics & ExperimentationOnsite
• Brainstorm a business problem approachMedium

  Brainstorm a business problem approach evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  AmazonSoftware EngineerAnalytics & ExperimentationTechnical Screen
• Choose Effective Graphs for Data ExplorationMedium

  Evaluates chart selection for exploratory data analysis across categorical and numerical variables. Strong answers explain pie charts, distributions, comparisons, relationships, missingness, and when visuals mislead.

  AmazonData ScientistAnalytics & ExperimentationOnsite
• Compare Tableau live vs extract and filtersHard

  This question evaluates competency in BI data architecture and large-scale dashboard performance, covering extract vs live connections, refresh and incremental strategies, query pushdown versus in-tool processing, row-level security, filter order-of-operations, performance tuning, and platform migration mapping within the Analytics & Experimentation domain for a Data Scientist role. It is commonly asked to assess trade-off reasoning across data freshness, storage, query performance and security for a 100M-row fact table, combining practical application-level decisions (implementation and tuning) with conceptual understanding (filter ordering, calculations, and migration pitfalls) when moving between BI platforms.

  AmazonData ScientistAnalytics & ExperimentationOnsite
• Design A/B Test for New Amazon Recommendation ModuleHard

  Design A/B Test for New Amazon Recommendation Module evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  AmazonData ScientistAnalytics & ExperimentationOnsite
• Design a One-Page Warehouse Inbound DashboardHard

  Design an executive warehouse inbound dashboard with throughput, dock-to-stock time, aged backlog, capacity, and quality metrics.

  AmazonBusiness Intelligence EngineerAnalytics & ExperimentationTechnical Screen
• Design an A/B Test for Dashboard Engagement ImpactMedium

  Amazon data scientist experimentation prompt on designing an A/B test for an energy-usage dashboard redesign, including hypotheses, metrics, guardrails, power, test duration, seasonality, and analysis methods.

  AmazonData ScientistAnalytics & ExperimentationOnsite
• Design an operations dashboard with justificationsHard

  This question evaluates a data scientist's ability to define precise operational metrics, justify visualization choices, design interactive drill-downs and alerting, and architect aggregation strategies for high-volume same-day delivery events.

  AmazonData ScientistAnalytics & ExperimentationOnsite
• Design and analyze pricing-page A/B testHard

  This question evaluates data-science competencies in online experiment design and analysis, including statistical power and sample-size estimation, variance-reduction methods, guardrail and SRM monitoring, randomization and contamination handling, sequential monitoring and stopping rules, heterogeneity/subgroup analysis, and result communication within the analytics & experimentation domain. It is commonly asked to assess practical application of statistical inference and experimental rigor in product and business contexts, primarily testing practical application while requiring conceptual understanding of underlying statistical principles and operational trade-offs.

  AmazonData ScientistAnalytics & ExperimentationOnsite