[ML] Various ways for Hyperparameter Tuning in Machine Learning

bona-park·2022년 10월 13일


목록 보기

Hyperparameter Tuning

The process of finding the right combination of hyperparameters to maximize the model performance

Hyperparameter tuning methods

  • Random Search
  • Grid Search
    • Each iteration tries a combination of hyperparameters in a specific order. It fits the model on each combination, records the model performance, and returns the best model with the best hyperparameters.
  • Bayesian Optimization
  • Tree-structured Parzen estimators(TPE)

Hyperparameter tuning algorithms

  • Hyperband
  • Population-based Training(PBT)
    • a hybrid of Random Search and manual tuning
    • Many neural networks run in parallel but they are not fully independent of each other.
    • It uses the information from the rest of the networks to refine the hyperparameters and determine which hyperparameter to use based on the rests

Useful libraries for hyperparameter optimization

ref) https://neptune.ai/blog/hyperparameter-tuning-in-python-complete-guide

읏차 웃자

0개의 댓글