We can define the inputs to the activation functions for the various layers in terms of various matrix-vector multiplications and vector additions. The inputs to the first hidden layer are
[z(1)1z(1)2]=([w(1)11w(1)12w(1)21w(1)22])T[a(0)1a(0)2]+[b(1)1b(1)2],with outputs
[a(1)1a(1)2]=[σ(1)(z(1)1)σ(1)(z(1)2)].