Neural nets typically contain smaller “subnetworks” that can often learn faster

In a new paper, researchers from MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) have shown that neural networks contain subnetworks that are up to one-tenth the size yet capable of being trained to make equally accurate predictions — and sometimes can learn to do so even faster than the originals. However, MIT Assistant Professor Michael Carbin says that his team’s findings suggest that, if we can determine precisely which part of the original network is relevant to the final prediction, scientists might one day be able to skip this expensive process altogether. “With a traditional neural network you randomly initialize this large structure, and after training it on a huge amount of data it magically works,” Carbin says.

Source: news.mit.edu