How Neural Networks Learn

From the perceptron to reverse-mode differentiation, derived and implemented from scratch

Abstract. Backpropagation is not a mysterious brain-inspired learning rule. It is an efficient application of the chain rule to a composed computation. The conceptual path from a perceptron to a multilayer network is a path from linear decision geometry to differentiable representation learning.

Planned (weeks 19–25). Done when every equation maps to a named variable in the implementation.