[ML] Various ways for Hyperparameter Tuning in Machine Learning

박제연·2022년 10월 13일


목록 보기

Hyperparameter Tuning

The process of finding the right combination of hyperparameters to maximize the model performance

Hyperparameter tuning methods

  • Random Search
  • Grid Search
    • Each iteration tries a combination of hyperparameters in a specific order. It fits the model on each combination, records the model performance, and returns the best model with the best hyperparameters.
  • Bayesian Optimization
  • Tree-structured Parzen estimators(TPE)

Hyperparameter tuning algorithms

  • Hyperband
  • Population-based Training(PBT)
    • a hybrid of Random Search and manual tuning
    • Many neural networks run in parallel but they are not fully independent of each other.
    • It uses the information from the rest of the networks to refine the hyperparameters and determine which hyperparameter to use based on the rests

Useful libraries for hyperparameter optimization

ref) https://neptune.ai/blog/hyperparameter-tuning-in-python-complete-guide

읏차 웃자

0개의 댓글