A “Network Pruning Network” Approach to Deep Model Compression


Pruning is something that I've seen come up more and more often recently and I find it very interesting. Here's the first paragraph of the paper that gives a good overview:

We present a filter pruning approach for deep model compression, using a multitask network. Our approach is based on learning a a pruner network to prune a pre-trained target network. The pruner is essentially a multitask deep neural network with binary outputs that help identify the filters from each layer of the original network that do not have any significant contribution to the model and can therefore be pruned. The pruner network has the same architecture as the original network except that it has a multitask/multi-output last layer containing binary-valued outputs (one per filter), which indicate which filters have to be pruned. The pruner’s goal is to minimize the number of filters from the original network by assigning zero weights to the corresponding output feature-maps.

The reason this seems like an obviously good idea is its correlate in the human brain: we know that our brains are constantly pruning less useful connections. I'm very interested in the future of this research.


Want to receive more content like this in your inbox?