Processing math: 100%
Week 36: Linear Regression and Statistical interpretations
Contents
Plans for week 36
Material for lecture Monday September 2
Important technicalities: More on Rescaling data
Test Function for what happens with OLS, Ridge and Lasso
Linking the regression analysis with a statistical interpretation
Assumptions made
Expectation value and variance
Expectation value and variance for
β
Deriving OLS from a probability distribution
Independent and Identically Distributed (iid)
Maximum Likelihood Estimation (MLE)
A new Cost Function
More basic Statistics and Bayes' theorem
Marginal Probability
Conditional Probability
Bayes' Theorem
Interpretations of Bayes' Theorem
Example of Usage of Bayes' theorem
Doing it correctly
Bayes' Theorem and Ridge and Lasso Regression
Ridge and Bayes
Lasso and Bayes
Material for the active learning sessions Tuesday and Wednesday
Linear Regression and the SVD
What does it mean?
And finally
X
X
T
Code for SVD and Inversion of Matrices
Inverse of Rectangular Matrix
Ridge and LASSO Regression
From OLS to Ridge and Lasso
Deriving the Ridge Regression Equations
Note on Scikit-Learn
Comparison with OLS
SVD analysis
Interpreting the Ridge results
More interpretations
Deriving the Lasso Regression Equations
Simple example to illustrate Ordinary Least Squares, Ridge and Lasso Regression
Ridge Regression
Lasso Regression
Yet another Example
The OLS case
The Ridge case
Writing the Cost Function
Lasso case
The first Case
Simple code for solving the above problem
With Lasso Regression
Another Example, now with a polynomial fit
Ridge Regression
For Ridge regression our cost function is
C
(
β
)
=
p
−
1
∑
i
=
0
(
y
i
−
β
i
)
2
+
λ
p
−
1
∑
i
=
0
β
2
i
,
and minimizing we have that
ˆ
β
R
i
d
g
e
i
=
y
i
1
+
λ
.
«
1
...
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
...
49
»