Katkat
Back

Generate all hyperparameter combinations

PinterestMachine Learning Engineer · Technical Screen
MediumMachine Learning EngineerTechnical ScreenCoding & Algorithms

Last updated

This problem tests your ability to generate the Cartesian product of multiple lists, a fundamental task in grid search for hyperparameter tuning.

Problem Statement

Given a dictionary where keys are hyperparameter names and values are lists of possible settings, write a function generate_hyperparameters(config_space) that returns a list of all possible hyperparameter configurations.

Each configuration should be a dictionary containing exactly one value for every key present in the input config_space.

Function Signature

def generate_hyperparameters(config_space: dict) -> list[dict]:
    # Implementation goes here

Constraints

  • The input config_space will contain at most 10 keys.
  • Each list in the dictionary will have at most 5 elements.
  • The order of the resulting list does not matter.

Examples

Example 1:

input = {
    "learning_rate": [0.01, 0.1],
    "batch_size": [32, 64]
}
# Output:
# [
#   {"learning_rate": 0.01, "batch_size": 32},
#   {"learning_rate": 0.01, "batch_size": 64},
#   {"learning_rate": 0.1, "batch_size": 32},
#   {"learning_rate": 0.1, "batch_size": 64}
# ]

Edge Cases

  • What if the input dictionary is empty? The function should return a list containing an empty dictionary [{}].
  • What if one of the hyperparameter lists is empty? The function should return an empty list [].

Hints

Hint 1

Consider using recursion or a backtracking approach to build configurations one key at a time.

Hint 2

The problem is equivalent to finding the Cartesian product of the lists associated with the dictionary keys.

Hint 3

Python's `itertools.product` can simplify the Cartesian product logic significantly if you extract the keys and values separately.

Follow-up questions

  • How would you modify your solution to support random sampling (e.g., return N random configurations instead of all of them)?
  • If the search space were extremely large (e.g., 10^12 combinations), how would you rewrite this to return an iterator or generator instead of a list?
  • How would you represent a hyperparameter that follows a continuous distribution (like a log-uniform distribution) rather than a discrete list of values?