Week 47: From Decision Trees to Ensemble Methods, Random Forests and Boosting Methods and Summary of Course
Contents
Plan for week 47
Bagging
More bagging
Making your own Bootstrap: Changing the Level of the Decision Tree
Random forests
Random Forest Algorithm
Random Forests Compared with other Methods on the Cancer Data
Compare Bagging on Trees with Random Forests
Boosting, a Bird's Eye View
What is boosting? Additive Modelling/Iterative Fitting
Iterative Fitting, Regression and Squared-error Cost Function
Squared-Error Example and Iterative Fitting
Iterative Fitting, Classification and AdaBoost
Adaptive Boosting, AdaBoost
Building up AdaBoost
Adaptive boosting: AdaBoost, Basic Algorithm
Basic Steps of AdaBoost
AdaBoost Examples
Gradient boosting: Basics with Steepest Descent/Functional Gradient Descent
The Squared-Error again! Steepest Descent
Steepest Descent Example
Gradient Boosting, algorithm
Gradient Boosting, Examples of Regression
Gradient Boosting, Classification Example
XGBoost: Extreme Gradient Boosting
Regression Case
Xgboost on the Cancer Data
Summary of course
What? Me worry? No final exam in this course!
What is the link between Artificial Intelligence and Machine Learning and some general Remarks
Going back to the beginning of the semester
Not so sharp distinctions
Topics we have covered this year
Statistical analysis and optimization of data
Machine learning
Learning outcomes and overarching aims of this course
Perspective on Machine Learning
Machine Learning Research
Starting your Machine Learning Project
Choose a Model and Algorithm
Preparing Your Data
Which Activation and Weights to Choose in Neural Networks
Optimization Methods and Hyperparameters
Resampling
Other courses on Data science and Machine Learning at UiO
Additional courses of interest
What's the future like?
Types of Machine Learning, a repetition
Why Boltzmann machines?
Boltzmann Machines
Some similarities and differences from DNNs
Boltzmann machines (BM)
A standard BM setup
The structure of the RBM network
The network
Goals
Joint distribution
Network Elements, the energy function
Defining different types of RBMs
More about RBMs
Autoencoders: Overarching view
Bayesian Machine Learning
Reinforcement Learning
Transfer learning
Adversarial learning
Dual learning
Distributed machine learning
Meta learning
The Challenges Facing Machine Learning
Explainable machine learning
Scientific Machine Learning
Quantum machine learning
Quantum machine learning algorithms based on linear algebra
Quantum reinforcement learning
Quantum deep learning
Social machine learning
The last words?
AI/ML and some statements you may have heard (and what do they mean?)
Best wishes to you all and thanks so much for your heroic efforts this semester
Plan for week 47
Work and Discussion of project 3
Last weekly exercise, course feedback, to be completed by Sunday November 26
Thursday: Basics of decision trees, classification and regression algorithms and ensemble models
Readings and Videos:
These lecture notes
Video of Lecture
Whiteboard notes
Video on Decision trees
Video on boosting methods by Hastie
Video on AdaBoost
Video on Gradient boost, part 1, parts 2-4 follow thereafter
Decision Trees: Geron's chapter 6 covers decision trees while ensemble models, voting and bagging are discussed in chapter 7. See also lecture from
STK-IN4300, lecture 7
. Chapter 9.2 of Hastie et al contains also a good discussion.
«
1
2
3
4
5
6
7
8
9
10
11
...
80
»