Katkat
Back

Optimize Hyper-parameter Search to Prevent Combinatorial Explosion

PinterestData Scientist · Onsite
MediumData ScientistOnsiteMachine Learning

Last updated

Evaluates Python grid-search generation and hyperparameter optimization strategy. Strong answers yield lazy Cartesian-product combinations and discuss random search, Bayesian optimization, pruning, early stopping, and budget limits.

Scenario

Hyper-parameter optimisation service must enumerate every grid-search combination.

Question

Write a generator that yields all combinations of values from a dict such as {'learning_rate':[0.1,0.2], 'feature':['A','B'], 'batch':[10,20]}. What alternatives exist to reduce explosion when grids are huge?

Hints

Use recursive backtracking or itertools.product; discuss random search, Bayesian optimisation, Sobol sampling.