Automatic differentiation

We can make this example more formal. Automatic differentiation is a formalization of the previous example (see graph).

We define $\boldsymbol{x}\in x_1,\dots, x_l$ input variables to a given function $f(\boldsymbol{x})$ and $x_{l+1},\dots, x_L$ intermediate variables.

In the above example we have only one input variable, $l=1$ and four intermediate variables, that is

$$\begin{bmatrix} x_1=x & x_2 = x^2=a & x_3 =\exp{a}= b & x_4=c=a+b & x_5 = \sqrt{c}=d \end{bmatrix}.$$

Furthemore, for $i=l+1, \dots, L$ (here $i=2,3,4,5$ and $f=x_L=d$), we define the elementary functions $g_i(x_{Pa(x_i)})$ where $x_{Pa(x_i)}$ are the parent nodes of the variable $x_i$.

In our case, we have for example for $x_3=g_3(x_{Pa(x_i)})=\exp{a}$, that $g_3=\exp{()}$ and $x_{Pa(x_3)}=a$.

• «
• 1
• ...
• 47
• 48
• 49
• 50
• 51
• 52
• 53
• 54
• 55
• 56
• 57
• 58
• 59
• 60
• 61
• 62
• 63
• 64
• ...
• 113
• »