Week 47: From Decision Trees to Ensemble Methods, Random Forests and Boosting Methods and Summary of Course

Contents

Optimization Methods and Hyperparameters

Stochastic gradient descent
1. Stochastic gradient descent + momentum
State-of-the-art approaches:

RMSProp
Adam
and more

Which regularization and hyperparameters? \( L_1 \) or \( L_2 \), soft classifiers, depths of trees and many other. Need to explore a large set of hyperparameters and regularization methods.