Exploring Neural Networks with Activation Atlases

For example, Karpathy’s CNN codes visualization gives a global view of a dataset by taking each image and organizing them by their activation values from a neural network. Roughly speaking, we can think of feature visualization as creating an idealized image of what the network thinks would produce a particular activation vector. Whereas we normally use a network to transform an image into an activation vector, in feature visualization we go in the opposite direction.

Source: distill.pub