Katkat
Back

Encode Categorical Variables Across Model Families

Point72Quantitative Researcher · Online Assessment
MediumQuantitative ResearcherOnline AssessmentMachine Learning

Last updated

Choose categorical encodings for linear, tree, and neural models while handling intercept rank, unseen values, and high cardinality.

Encode Categorical Variables Across Model Families

A nominal feature has K categories. Explain how many dummy variables you would use in a linear regression, a random forest, and a basic feed-forward neural network. Discuss intercepts, multicollinearity, unseen categories, high cardinality, and model-native alternatives.

Constraints & Assumptions

  • Categories have no natural numeric order.
  • The transformation must be identical in training and serving.
  • Some production rows may contain missing or previously unseen categories.
  • The question asks about ordinary dummy encoding, while alternatives may be discussed.

Clarifying Questions to Ask

  • Does the linear model include an intercept?
  • Does the tree library support native categorical splits?
  • Is the neural network allowed to learn embeddings?
  • How large and stable is K?

Hints

Hint 1

Ask whether linear dependence matters to the learner: The reference-category rule addresses rank in a linear design matrix; split-based and nonlinear models have different reasons for retaining or replacing all indicators.

Follow-up questions

  • What changes in linear regression if the intercept is removed?
  • Why is integer label encoding risky for nominal categories?
  • When would target encoding be useful, and how would you prevent leakage?