Ensemble Methods

• 단일 모델의 overfitting 문제를 해결하는 동시에, 여러 예측의 결합을 통해 높은 정확도를 도출하고자 하는 기법

Random Forest

• Strong Learner - Random Forest
• Weak Learners - Decision Trees

Perturbations on Columns

1. Pick random columns
2. Build a decision tree
3. Repeat 1-2
4. Calculate Probability

Bagging

1. Take random data subsets
2. Train weak learners on each datasets
3. Combine & calculate Probability

Forests of Randomized Trees

1. Choose # of trees
2. Use bagging to generate dataset
3. Create splits
4. Combine & calculate Probability