Implementing a Convolutional Neural Network Using Only NumPy

Consider this forward phase for a Max Pooling layer:

An example forward phase that transforms a 4×4 input to a 2×2 output

The backward phase of that same layer would look like this:

An example backward phase that transforms a 2×2 gradient to a 4×4 gradient

Each gradient value is assigned to where the original max value was, and every other value is zero. The forward phase caching is simple:

We’re primarily interested in the loss gradient for the filters in our conv layer, since we need that to update our filter weights. Doing the math confirms this:

We can put it all together to find the loss gradient for specific filter weights:

We’re ready to implement backprop for our conv layer!

Source: victorzhou.com