Lasso case

For Lasso we need now, keeping a constraint on $\vert\beta_0\vert+\vert\beta_1\vert=1$, to take the derivative of the absolute values of $\beta_0$ and $\beta_1$. This gives us the following derivatives of the cost function

$$C(\boldsymbol{\beta})=(4-2\beta_0)^2+(2-\beta_1)^2+\lambda(\vert\beta_0\vert+\vert\beta_1\vert),$$ $$\frac{\partial C(\boldsymbol{\beta})}{\partial \beta_0}=-4(4-2\beta_0)+\lambda\mathrm{sgn}(\beta_0)=0,$$

and

$$\frac{\partial C(\boldsymbol{\beta})}{\partial \beta_1}=-2(2-\beta_1)+\lambda\mathrm{sgn}(\beta_1)=0.$$

We have now four cases to solve besides the trivial cases $\beta_0$ and/or $\beta_1$ are zero, namely

1. $\beta_0 > 0$ and $\beta_1 > 0$,
2. $\beta_0 > 0$ and $\beta_1 < 0$,
3. $\beta_0 < 0$ and $\beta_1 > 0$,
4. $\beta_0 < 0$ and $\beta_1 < 0$.

• «
• 1
• ...
• 37
• 38
• 39
• 40
• 41
• 42
• 43
• 44
• 45
• 46
• 47
• 48
• 49
• »