First network example, simple percepetron with one input

As yet another example we define now a simple perceptron model with all quantities given by scalars. We consider only one input variable $x$ and one target value $y$. We define an activation function $\sigma_1$ which takes as input

$$z_1 = w_1x+b_1,$$

where $w_1$ is the weight and $b_1$ is the bias. These are the parameters we want to optimize. The output is $a_1=\sigma(z_1)$ (see graph from whiteboard notes). This output is then fed into the cost/loss function, which we here for the sake of simplicity just define as the squared error

$$C(x;w_1,b_1)=\frac{1}{2}(a_1-y)^2.$$

• «
• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9
• 10
• 11
• 12
• 13
• 14
• 15
• 16
• 17
• ...
• 100
• »