Analytics & Experimentation Interview Questions

Showing 181–210 of 218 results

• PinterestMedium·Data Scientist
  Evaluate New Feed-Ranking Algorithm with A/B Testing

  Pinterest experimentation prompt on evaluating a new feed-ranking algorithm, covering A/B hypotheses, daily active minutes, guardrails, sample size, MDE, experiment health, mid-test dips, and geo-based causal inference.

  Analytics & ExperimentationOnsite
• PinterestHard·Data Scientist
  Investigate Homepage Experiment Without Control Group: Methods and Metrics

  Evaluates quasi-experimental analysis and product-metric judgment after a homepage launch lacks a randomized control. Strong answers compare matching and propensity weighting, audit experiment hygiene, choose metrics, and diagnose mixed CTR results.

  Analytics & ExperimentationOnsite
• PinterestMedium·Data Scientist
  Measure Billboard Campaign Impact: Design, Bias, Test Strategy

  This question assesses your ability to design a quasi-experimental framework for measuring the offline impact of an online brand campaign on user acquisition, accounting for external noise and selection bias.

  Analytics & ExperimentationOnsiteCausal InferenceGeo-ExperimentsMetric Design
• PinterestMedium·Data Scientist
  Size an Opportunity and Present It with Clear Visuals

  Tests your ability to size an ambiguous product opportunity using top-down and bottom-up estimation, stress-test core assumptions with sensitivity analysis, and design executive-ready visuals to guide strategic investment decisions.

  Analytics & ExperimentationOnsiteopportunity sizingproduct analyticsdata visualization
• RedditEasy·Data Scientist
  Design A/B test for AI chat box

  This question tests the ability to define metrics, design an A/B testing framework, and account for potential biases when deploying a new AI-powered chat feature.

  Analytics & ExperimentationOnsiteA/B TestingProduct AnalyticsMetrics
• RedditEasy·Data Scientist
  Design experiment for ads in chat with budgets

  This question tests the ability to design an A/B test for a new ad placement in a chat feature while accounting for budget constraints and potential network effects.

  Analytics & ExperimentationOnsiteA/B TestingCausal InferenceExperiment Design
• RedditHard·Data Scientist
  Estimate Incremental Search-Ad Revenue Without an A/B Test

  Design an evaluation framework to estimate the incremental search-ad revenue of a small-scale manual pilot, incorporating variance reduction, inference under heavy-tailed revenue, and ecosystem cannibalization.

  Analytics & ExperimentationOnsiteexperimentationcausal-inferencead-tech
• RedditEasy·Data Scientist
  Measure impact of ads-manager automation feature

  This question tests your ability to design an evaluation framework for a new feature, specifically focusing on causal inference, metric selection, and identifying potential biases in a two-sided marketplace.

  Analytics & ExperimentationOnsiteA/B TestingCausal InferenceMetrics
• RobinhoodEasy·Data Scientist
  Design an experiment to launch fractional shares

  This question evaluates a candidate's competency in experimentation design, product analytics, statistical power and sample-size calculation, causal inference, and risk identification within a brokerage product context.

  Analytics & ExperimentationOnsite
• RobloxHard·Software Engineer
  Define success metrics and monitoring

  Define success metrics and monitoring evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationOnsite
• ShopifyMedium·Data Scientist
  Analyze Causes of November and June Shopify Traffic Spikes

  This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Analyze Causes of November and June Shopify Traffic Spikes states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationOnsite
• ShopifyHard·Machine Learning Engineer
  Collect labels without existing data

  This question evaluates a machine learning engineer's competency in label definition and leakage prevention, instrumentation and logging design, programmatic and human-in-the-loop labeling pipelines, experimental approaches for eliciting outcomes, bias mitigation, and continuous label quality monitoring.

  Analytics & ExperimentationOnsite
• ShopifyMedium·Data Scientist
  Diagnose Weekly Session Conversion Anomalies

  This question evaluates proficiency in time-series anomaly detection, conversion-rate analysis, data quality handling (especially parsing and validating week labels), segmentation by shop type and session-duration buckets, and causal reasoning about traffic composition effects.

  Analytics & ExperimentationOnsite
• ShopifyMedium·Data Scientist
  Measure Shopify App Store Success

  This question evaluates a data scientist's competency in product-metrics design for a multi-sided app marketplace, including defining a north-star metric, per-side success and guardrail metrics, required dataset schemas and join keys, and criteria for evaluating ranking/search/recommendation experiments.

  Analytics & ExperimentationOnsite
• SnapchatMedium·Data Scientist
  Design A/B Tests for Banner Ad and Group-Story Feature

  Evaluates A/B test design for a banner ad and a group-story feature in a social app. Strong answers define monetization, engagement, safety, and retention metrics, handle accidental clicks for ads, account for network effects in group stories, and set launch criteria with guardrails.

  Analytics & ExperimentationOnsite
• StripeMedium·Data Scientist
  Choose target customers and define success metrics

  This question tests the ability to design an experimental rollout strategy for a new financial product, requiring a balance between revenue growth, risk mitigation, and statistical validity.

  Analytics & ExperimentationOnsiteExperimentationRisk ModelingProduct Analytics
• StripeMedium·Data Scientist
  Evaluate Stripe Capital Lending Strategy

  This question evaluates data science competencies in product analytics, credit risk modeling, monitoring and instrumentation for a merchant lending product, including dashboard metric design, early-warning signal engineering, offer-structuring trade-offs, and portfolio-level profit attribution.

  Analytics & ExperimentationOnsite
• StripeMedium·Data Scientist
  How Should Stripe Capital Be Evaluated?

  This problem tests your ability to design a performance measurement framework for a lending product, focusing on balancing merchant growth with credit risk.

  Analytics & ExperimentationOnsiteCredit RiskProduct AnalyticsCohort Analysis
• ThumbtackHard·Data Scientist
  Design a robust pro-ranking A/B test

  It evaluates experimental design and causal inference competencies for two-sided marketplaces, focusing on metric definition and guardrails, randomization to limit interference, power and sample-size estimation, bias controls, monitoring and sequential stopping, and attribution of uplift versus cannibalization.

  Analytics & ExperimentationOnsite
• TikTokMedium·Data Scientist
  Balance Customer Satisfaction with Fraud Prevention: Key Metrics to Track

  Evaluates risk-decisioning trade-offs between customer experience and fraud prevention in payment products. Strong answers cover cost-sensitive thresholds, false declines, fraud loss, experimentation, monitoring, and delayed labels.

  Analytics & ExperimentationOnsite
• TikTokHard·Data Scientist
  Design A/B Test for Cost-Per-Conversion Efficiency Analysis

  Design A/B Test for Cost-Per-Conversion Efficiency Analysis evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationOnsite
• TikTokMedium·Data Scientist
  Design a creator posting-frequency experiment

  This problem tests the ability to design a controlled experiment while accounting for network interference, non-compliance, and selection bias in a two-sided platform.

  Analytics & ExperimentationOnsiteExperimentationA/B TestingCausal Inference
• TikTokMedium·Data Scientist
  Diagnose Search Issues with Relevant Metrics and Solutions

  Evaluates diagnosis of brand-name search relevance and safety issues using analytics and experimentation. Strong answers define relevance, satisfaction, and harmful-content metrics, find root causes, and validate fixes offline and online.

  Analytics & ExperimentationOnsite
• TikTokHard·Data Scientist
  Diagnose metric drop in Ads Manager

  This question evaluates a Data Scientist's competency in operational analytics, metric integrity, root-cause analysis, and experiment interpretation, covering concepts such as logging latency, sample ratio assessment, and product instrumentation awareness.

  Analytics & ExperimentationOnsite
• TikTokMedium·Data Scientist
  Improve TikTok's Algorithm for Diverse Content Discovery

  Evaluates product feedback and experiment design for improving content discovery in TikTok-style feeds. Strong answers propose specific improvements, justify impact, define metrics, and plan experiments with guardrails.

  Analytics & ExperimentationOnsite
• TikTokHard·Data Scientist
  Measure Billboard Campaign Effectiveness and Engagement Quantification

  Evaluates offline marketing measurement for a billboard campaign, including how to define attributable engagement, estimate exposed population, and forecast incremental traffic, sign-ups, retention, revenue, and ROI. Strong answers use geo-lift, matched markets, or synthetic controls rather than direct attribution alone.

  Analytics & ExperimentationOnsite
• TikTokHard·Data Scientist
  Test 15s to 60s video length change

  This question evaluates a data scientist's competency in experimental design, causal inference, metric definition and instrumentation, power/MDE calculations, interference and contamination handling on multi‑sided platforms, and rollout planning.

  Analytics & ExperimentationOnsite
• TuroHard·Data Scientist
  Identify Booking Drivers

  This question evaluates skills in product analytics, causal inference, feature importance modeling, metric definition, and experimental validation within a marketplace context.

  Analytics & ExperimentationOnsite
• UberMedium·Software Engineer
  Define and integrate room ranking factors

  Define and integrate room ranking factors evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

  Analytics & ExperimentationOnsite
• UberMedium·Data Scientist
  Design Rideshare Marketplace Causal Analyses

  This question evaluates a data scientist's skills in experimental design and causal inference, marketplace metrics and diagnostics, fairness and safety impact assessment, incentive and policy evaluation, and ETA modeling and forecast quality measurement.

  Analytics & ExperimentationOnsite