Analytics & Experimentation interview questions

Showing 421–450 of 560 results

• Define goals and success metrics for subscriber-only featuresHard

  This question evaluates a data scientist's competency in product analytics and experimentation, specifically goal selection, success metric definition, ROI calculation, cannibalization analysis, and end-to-end experiment and instrumentation design for subscriber-only features.

  OtherData ScientistAnalytics & ExperimentationTechnical Screen
• Design an A/B test with guardrailsHard

  This question evaluates a data scientist's skills in experimental design, hypothesis testing, statistical power and sample-size calculation, randomization and contamination control, pre-registration planning, and multiplicity adjustment for A/B testing.

  OtherData ScientistAnalytics & ExperimentationOnsite
• Design metrics and experimentHard

  This question evaluates a candidate's competency in experimental design, success-metric definition, event instrumentation, causal inference for heterogeneous treatment effects, and statistical power and sample-size calculations for subscription-product analytics.

  OtherData ScientistAnalytics & ExperimentationTechnical Screen
• Design metrics resilient to data qualityMedium

  This question evaluates the ability to design robust data-quality metrics and apply SQL window-function techniques for deduplication and time-windowed completeness calculations, focusing on quantifying how ingestion delays, duplicates, and negative amounts can bias payment metrics.

  OtherData ScientistAnalytics & ExperimentationTechnical Screen
• How would you measure and experiment on harmful content?Easy

  This question evaluates a data scientist's competency in measuring and experimenting on harmful user-generated content, focusing on severity quantification, metric selection, and randomized experiment design within the analytics and experimentation domain.

  OtherData ScientistAnalytics & ExperimentationTechnical Screen
• Separate demand from supply for jeansMedium

  This question evaluates a data scientist's ability to perform data-driven diagnostics, funnel-metric construction, quantitative decision-rule design, and experimentation planning to distinguish demand-side from supply-side constraints in a marketplace jeans category.

  OtherData ScientistAnalytics & ExperimentationOnsite
• Analyze Transactions for Risk and Implement Mitigation StrategiesMedium

  Analyze Transactions for Risk and Implement Mitigation Strategies evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  PayPalData ScientistAnalytics & ExperimentationOnsite
• Assess card transactions and plan risk strategyHard

  Assess card transactions and plan risk strategy evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  PayPalData ScientistAnalytics & ExperimentationOnsite
• Boost User Login Rate: Key Metrics to MonitorMedium

  This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Boost User Login Rate: Key Metrics to Monitor states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  PayPalData ScientistAnalytics & ExperimentationOnsite
• Design an A/B for ATO ruleHard

  This question evaluates a data scientist's experiment-design and statistical-analysis competencies, including cluster-aware randomization, power/sample-size calculation, sequential monitoring, and translating detectable effects into net business impact for a real-time account-takeover prevention rule on a payment platform.

  PayPalData ScientistAnalytics & ExperimentationTechnical Screen
• Design and Analyze A/B Test for Cashback ProgramMedium

  This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Design and Analyze A/B Test for Cashback Program states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  PayPalData ScientistAnalytics & ExperimentationOnsite
• Design and evaluate a fraud detection strategyEasy

  This problem tests your ability to design a holistic fraud detection strategy by balancing technical model metrics with business-critical operational trade-offs.

  PayPalData ScientistAnalytics & ExperimentationTechnical Screen
• Design metrics and experiment for donation featureEasy

  A PayPal data science onsite case asking you to design metrics and a randomized experiment for an optional checkout donation feature on a food-ordering platform. Covers goal and hypothesis framing, a primary/diagnostic/guardrail metric hierarchy with precise definitions, user-level experiment design with power and variance-reduction, confounder/marketplace-bias handling, and a launch decision framework.

  PayPalData ScientistAnalytics & ExperimentationOnsite
• How to evaluate a new homepage featureEasy

  A PayPal data science technical screen on designing an A/B test to evaluate a new homepage feature. It covers experiment design and randomization, primary/secondary/guardrail metric selection with tradeoffs, sample-size and power estimation, and pitfalls like novelty effects and interference. The solution emphasizes value-aligned metrics (payment completion over CTR) and fraud guardrails specific to a payments product.

  PayPalData ScientistAnalytics & ExperimentationTechnical Screen
• Master A/B Testing: Key Concepts and Methodologies ExplainedMedium

  Evaluates practical A/B testing and causal inference fundamentals for online products. Strong answers cover p-values, errors, power, workflow, Simpson's paradox, metrics, and observational causal methods.

  PayPalData ScientistAnalytics & ExperimentationOnsite
• Reduce airport ride cancellations under causal constraintsEasy

  A PayPal Data Scientist onsite product-sense and causal-inference case: design a measurement framework, hypotheses, and experimentation strategy to reduce airport ride cancellations in a ride-hailing marketplace without harming marketplace health. It tests metric trees, interference-aware experiment design (switchback, partial-population, encouragement/IV), satisfaction proxies, confounder handling, and stakeholder communication.

  PayPalData ScientistAnalytics & ExperimentationOnsite
• Analyze a geo rollout and interpret chartsHard

  This question evaluates causal inference and product analytics competencies — specifically specification of causal estimands and difference-in-differences, segmented (interrupted) time-series regression, power and duration calculations, spillover and donor-pool diagnostics, and selection of guardrail metrics.

  PinterestData ScientistAnalytics & ExperimentationOnsite
• Decide if ad load is optimizedHard

  This question evaluates competency in analytics and experimentation, covering experiment design, metric and guardrail definition, causal inference, statistical testing, segmentation for heterogeneous effects, and balancing user experience versus monetization.

  PinterestData ScientistAnalytics & ExperimentationOnsite
• Design and Evaluate a Home CarouselMedium

  This question evaluates a candidate's expertise in product analytics, experimentation design, metric definition, and instrumentation for measuring the impact of a new UI feature.

  PinterestData ScientistAnalytics & ExperimentationTechnical Screen
• Design and Interpret a Video Pin ExperimentHard

  This question tests the ability to design a balanced A/B experiment for a two-sided marketplace (creators and viewers) and synthesize conflicting metrics into a launch decision.

  PinterestData ScientistAnalytics & ExperimentationTechnical Screen
• Design and assess a video-pin increase experimentMedium

  A Pinterest Data Scientist technical-screen question on Analytics & Experimentation: design a rigorous A/B test for increasing video-pin share in the Home Feed — unit of randomization, primary metric and guardrails, power/MDE, sequential analysis, and a quasi-experiment fallback — then interpret two 14-day readouts and recommend ship, iterate, or stop. It rewards balancing engagement lifts (CTR) against guardrail regressions (retention, complaints, crash rate), multiple-testing robustness, and cross-unit (per-impression vs per-session) trade-offs.

  PinterestData ScientistAnalytics & ExperimentationTechnical Screen
• Design metrics and experiment for Shopping launchHard

  This question evaluates a data scientist’s competency in experimental design and product analytics, including metric selection and precise definitions, guardrail and funnel diagnostics, spillover and novelty handling, power/MDE estimation, and multi-objective decision framing.

  PinterestData ScientistAnalytics & ExperimentationTechnical Screen
• Diagnose CTR drop after recommendation launchHard

  This question evaluates a data scientist's competency in experiment diagnosis, metric and guardrail definition, instrumentation validation, segmentation analysis, and causal reasoning for recommender-system impacts.

  PinterestData ScientistAnalytics & ExperimentationOnsite
• Estimate a Launch Impact with Difference in DifferencesMedium

  Tests your ability to design and evaluate a regional quasi-experiment using Difference-in-Differences (DiD), validate key identifying assumptions, structure robust inference, and address modern complications such as staggered rollout and network spillovers.

  PinterestData ScientistAnalytics & ExperimentationOnsite
• Evaluate Carousel and Billboard LiftMedium

  This question tests your ability to design an A/B test for multi-format advertising, specifically handling cannibalization and cross-format lift measurement.

  PinterestData ScientistAnalytics & ExperimentationTechnical Screen
• Evaluate Fresh Content and Video ExperimentsMedium

  This question evaluates competencies in metric design, experimental design, statistical inference, and product analytics—specifically defining and critiquing freshness metrics, formulating null and alternative hypotheses for content changes, interpreting A/B test results, and calculating sample size and power.

  PinterestData ScientistAnalytics & ExperimentationTechnical Screen
• Evaluate New Feed-Ranking Algorithm with A/B TestingMedium

  Pinterest experimentation prompt on evaluating a new feed-ranking algorithm, covering A/B hypotheses, daily active minutes, guardrails, sample size, MDE, experiment health, mid-test dips, and geo-based causal inference.

  PinterestData ScientistAnalytics & ExperimentationOnsite
• How to evaluate a new Carousel featureEasy

  This question tests your ability to design an A/B test for a feed-based UI feature, focusing on metric selection, counter-metrics, and the identification of potential cannibalization effects.

  PinterestData ScientistAnalytics & ExperimentationTechnical Screen
• How would you evaluate a carousel launch?Medium

  This question tests your ability to design a comprehensive evaluation framework for a product feature launch, specifically focusing on metric selection, causal impact measurement, and addressing potential cannibalization effects.

  PinterestData ScientistAnalytics & ExperimentationTechnical Screen
• Interpret A/B results for video-pin increaseMedium

  This question evaluates statistical inference and experimentation skills, including A/B test hypothesis formulation with a predefined MDE, estimation of absolute and relative lift with confidence intervals, proportion testing for CTR and retention, sample ratio mismatch detection, and multiple-comparison adjustments.

  PinterestData ScientistAnalytics & ExperimentationTechnical Screen