Katkat
Back

Estimate and validate weights for engagement actions

MetaData Scientist · Technical Screen
HardData ScientistTechnical ScreenStatistics & Math

Last updated

This question tests the ability to derive an engagement scoring model, handle high-dimensional collinear features, and establish a robust validation framework for point-in-time predictions.

Scenario

You are a Data Scientist at a social media platform. You want to estimate a 'Global Engagement Score' for users based on four primary actions: Likes (L), Comments (C), Shares (S), and Saves (V). You have a dataset of 1 million users with these counts over the last 30 days.

Task

  1. You decide to model the total engagement time (T) as a linear combination of these actions: T = wL*L + wC*C + wS*S + wV*V + epsilon.

    • Explain how you would estimate the weights (w) if you require all weights to be non-negative and the sum of weights to equal 1.
    • How would you address the issue of high multicollinearity between 'Likes' and 'Shares'?

  2. Validation and Uncertainty:

    • Describe a procedure to estimate the 95% confidence interval for each weight.
    • If your model performs well on a training set but shows high variance on out-of-sample data, how would you incorporate Bayesian priors to stabilize the estimates?

  3. Implementation:

    • Provide a high-level pseudo-code or logic flow for performing k-fold cross-validation to tune the regularization parameter if you chose to use Elastic Net regression instead of constrained weights.

Constraints & Considerations

  • Assume your features (L, C, S, V) are on different scales and highly skewed.
  • Consider the impact of sparse data for newer users who may have zero counts for some actions.
  • Clearly state your assumptions regarding the distribution of the error term epsilon.

Hints

Hint 1

Consider using Non-Negative Least Squares (NNLS) with a Lagrangian multiplier to enforce the sum-to-one constraint.

Hint 2

For multicollinearity, explore dimensionality reduction techniques like PCA or apply L2 regularization to shrink coefficients of correlated predictors.

Hint 3

To quantify uncertainty without assuming normality, utilize bootstrapping methods to generate empirical distributions for the weight estimates.

Follow-up questions

  • How would you detect if the relationship between these actions and engagement time is non-linear (e.g., diminishing returns for Likes)?
  • If 'Shares' are a much rarer event than 'Likes', how should this affect your weighting strategy or preprocessing steps?
  • How does your approach change if you want to optimize for long-term retention rather than immediate engagement time?