The Unreasonable Effectiveness of Deep Feature Extraction

To turn a pretrained deep neural network into a feature extractor, you pick a layer toward the end of the network, feed in the image you want to extract features from, and then use the activations of that layer as features. Buried in the paper, though, is this very important aside:

In other words, if you take this huge network pretrained on a billion images, repurpose it as a feature extractor, and use it to train a simple linear model on your dataset — it’s almost as good as finetuning the whole thing. Unless you have literally the largest dataset in the world, training a neural network on it from scratch will probably give worse results than using a huge pretrained net as a feature extractor and training a simple linear model on that.

Source: www.basilica.ai