Backpropagation and Automatic Differentiation #
Backpropagation applies the chain rule:
- efficiently across a computational graph.
- repeatedly.
Chain rule:
[ \frac{dL}{dx} = \frac{dL}{dy} \cdot \frac{dy}{dx} ]
flowchart LR
x --> y
y --> L
Automatic differentiation computes exact derivatives efficiently using computational graphs.