04 Introduction to Neural Networks

In Lecture 4 we progress from linear classifiers to fully-connected neural networks. We introduce the backpropagation algorithm for computing gradients and briefly discuss connections between artificial neural networks and biological neural networks.

• Keywords: Neural networks, computational graphs, backpropagation, activation functions, biological neurons

• slides

Backpropagation

Backprop is a recursive appllication of chain rule. Let's see an example.

Patterns in backward flow

Gradients for vectorized code

Vectorized operations

To be added: what is Jacobian

Modularized implementation

Summary