Katkat
Back

Derive correlation bounds and omitted-variable bias

Two SigmaData Scientist · Technical Screen
HardData ScientistTechnical ScreenMachine Learning

Last updated

This problem tests your understanding of the constraints on correlation matrices and your ability to analytically derive the bias introduced in linear models when a relevant predictor is ignored.

Part 1: Correlation Bounds

You are given three variables X, Y, and Z. Let rho_XY, rho_YZ, and rho_XZ represent their pairwise correlations. Prove that for any valid correlation matrix, the following inequality must hold: 1 + 2 * rho_XY * rho_YZ * rho_XZ - rho_XY^2 - rho_YZ^2 - rho_XZ^2 >= 0.

Then, consider the case where all pairwise correlations are equal to a constant c. What is the minimum possible value for c? Provide the range of c for which a 3x3 matrix with c as all off-diagonal elements remains positive semi-definite.

Part 2: Omitted-Variable Bias

Consider the true data-generating process: Y = beta_0 + beta_1 * X_1 + beta_2 * X_2 + epsilon, where E[epsilon|X_1, X_2] = 0. Suppose you instead estimate the model: Y = alpha_0 + alpha_1 * X_1 + u by omitting X_2.

  1. Express the probability limit of the estimator alpha_1_hat in terms of beta_1, beta_2, and the auxiliary regression coefficient delta, where delta is the slope coefficient from regressing X_2 on X_1.
  2. Under what specific conditions is the estimate alpha_1_hat unbiased even when beta_2 is not zero?

Hints

Hint 1

For Part 1, recall that a correlation matrix must be positive semi-definite, which implies its determinant must be non-negative.

Hint 2

For Part 2, substitute the true model into the expression for the OLS estimator `alpha_1_hat = Cov(X_1, Y) / Var(X_1)`.

Hint 3

The bias in Part 2 is zero if either the omitted variable has no effect on Y or if the omitted variable is uncorrelated with the included predictor.

Follow-up questions

  • How does the bias change if X_1 and X_2 are negatively correlated versus positively correlated?
  • If we add an interaction term (X_1 * X_2) to the true model, how does the omitted-variable bias derivation become more complex?
  • In a high-dimensional setting with many features, how do regularization techniques like Lasso or Ridge handle potential omitted-variable bias?