Week 46: Decision Trees, Ensemble methods and Random Forests
Contents
Plan for week 46
Decision trees, overarching aims
Basics of a tree
A typical Decision Tree with its pertinent Jargon, Classification Problem
General Features
How do we set it up?
Decision trees and Regression
Building a tree, regression
A top-down approach, recursive binary splitting
Making a tree
Pruning the tree
Cost complexity pruning
Schematic Regression Procedure
A Classification Tree
Growing a classification tree
Classification tree, how to split nodes
Visualizing the Tree, Classification
Visualizing the Tree, The Moons
Other ways of visualizing the trees
Printing out as text
Algorithms for Setting up Decision Trees
The CART algorithm for Classification
The CART algorithm for Regression
Why binary splits?
Computing a Tree using the Gini Index
The Table
Computing the various Gini Indices
Computing the various Gini Indices, Hours slept
Computing the various Gini Indices, Hours studied
A possible code using Scikit-Learn
Further example: Computing the Gini index
Simple Python Code to read in Data and perform Classification
Computing the Gini Factor
Regression trees
Final regressor code
Pros and cons of trees, pros
Disadvantages
Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods
An Overview of Ensemble Methods
Why Voting?
Tossing coins
Standard imports first
Simple Voting Example, head or tail
Using the Voting Classifier
Voting and Bagging
Bagging
More bagging
Making your own Bootstrap: Changing the Level of the Decision Tree
Random forests
Random Forest Algorithm
Random Forests Compared with other Methods on the Cancer Data
Compare Bagging on Trees with Random Forests
Boosting, a Bird's Eye View
What is boosting? Additive Modelling/Iterative Fitting
Iterative Fitting, Regression and Squared-error Cost Function
Squared-Error Example and Iterative Fitting
Iterative Fitting, Classification and AdaBoost
Adaptive Boosting, AdaBoost
Building up AdaBoost
Adaptive boosting: AdaBoost, Basic Algorithm
Basic Steps of AdaBoost
AdaBoost Examples
Plan for week 46
Work and Discussion of project 2
Discussion of project 3 as well
Thursday: Basics of decision trees, classification and regression algorithms and ensemble models
Readings and Videos:
These lecture notes
Video of lecture
Whiteboard notes
Video on Decision trees
Decision Trees: Geron's chapter 6 covers decision trees while ensemble models, voting and bagging are discussed in chapter 7. See also lecture from
STK-IN4300, lecture 7
. Chapter 9.2 of Hastie et al contains also a good discussion.
«
1
2
3
4
5
6
7
8
9
10
11
...
63
»