Gradients of Vector-Valued and Matrix Functions #
Covers gradients when outputs or parameters are vectors/matrices.
If f: R^n -> R^m, the derivative is the Jacobian.
[ J = \begin{bmatrix} \frac{\partial f_1}{\partial x_1} & \dots & \frac{\partial f_1}{\partial x_n} \ \vdots & \ddots & \vdots \ \frac{\partial f_m}{\partial x_1} & \dots & \frac{\partial f_m}{\partial x_n} \end{bmatrix} ]For scalar f(x):
[ H = \nabla^2 f ]Hessian captures curvature.