Katkat
Back

Answer basic probability and statistics questions

Morgan StanleyData Scientist · Online Assessment
MediumData ScientistOnline AssessmentMachine Learning

Last updated

This problem tests your conceptual understanding of probability distributions, statistical bias in data collection, and the relationship between covariance and linear correlation.

Part 1: Probability Distributions

Suppose a customer support team receives an average of 3 requests per hour. You observe the count of requests over several 1-hour intervals. If the probability mass function (PMF) of the observed counts shifts significantly such that the probability of observing 0 requests becomes much lower than the theoretical Poisson prediction, what does this suggest about the underlying process, and which parameter might be changing?

Part 2: Statistical Bias

A fintech startup wants to predict user churn. They sample their current user base, which consists only of users who have been active for at least 6 months, to train their predictive model. Explain the type of bias introduced here and how it would likely impact the model's performance on the general population of new users.

Part 3: Covariance and Correlation

Consider two random variables, X and Y.

  1. If the covariance Cov(X, Y) = 0, does it strictly imply that X and Y are independent? Provide a counterexample or a brief explanation.
  2. How does the Pearson correlation coefficient rho(X, Y) change if you perform a linear transformation on X such that X_new = aX + b, where a is a negative constant and b is any real number?

Hints

Hint 1

For Part 1, consider if the data truly follows a Poisson process or if there is a 'floor' or 'ceiling' effect on the observations.

Hint 2

For Part 2, focus on the criteria used to include data in the training set and how that excludes a segment of the population that eventually churned.

Hint 3

For Part 3, remember that covariance measures linear dependence, and look at the mathematical definition of Pearson correlation to see how scaling factors cancel out.

Follow-up questions

  • If you notice the variance of your Poisson-distributed data is significantly higher than the mean, what model would you use instead of Poisson regression?
  • How would you mitigate the sampling bias described in Part 2 if you cannot access historical data from users who already left?
  • Can you derive the relationship between Pearson correlation and the slope of the simple linear regression line of Y on X?