Loading [MathJax]/extensions/TeX/boldsymbol.js

 

 

 

Chain rule, forward and reverse modes

In the above example we have introduced the variables a and b , and our function is

f(x) = f(a(x)) = b= \exp{a},

with a=x^2 . We can decompose the derivative of f with respect to x as

\frac{df}{dx}=\frac{df}{db}\frac{db}{da}\frac{da}{dx}.

We note that since b=f(x) that

\frac{df}{db}=1,

leading to

\frac{df}{dx}=\frac{db}{da}\frac{da}{dx}=2x\exp{x^2},

as before.