Filter Pruning

Filter Pruning

Filter pruning is a type of structural pruning on Convolutional Neural Networks (CNNs), but does not apply to Transformer architectures. It narrows the width of a CNN and thereby reduces the required computations.

Research papers on filter pruning:

1. Q. Huang, K. Zhou, S. You, and U. Neumann, 2018, Learning to prune filters in convolutional neural networks, in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2018, pp. 709–718, https://arxiv.org/abs/1801.07365
2. Jian-Hao Luo, Jianxin Wu, and Weiyao Lin. 2017, Thinet: A filter level pruning method for deep neural network compression, arXiv preprint arXiv:1707.06342, July 2017. https://arxiv.org/abs/1707.06342 (Filter pruning.)
3. Z. You, K. Yan, J. Ye, M. Ma and P. Wang, 2019, Gate decorator: Global filter pruning method for accelerating deep convolutional neural networks, arXiv:1909.08174, 2019. https://arxiv.org/abs/1909.08174
4. Wang, Y., Zhang, X., Xie, L., Zhou, J., Su, H., Zhang, B., and Hu, X. 2020, Pruning from scratch, In AAAI, 2020. https://arxiv.org/abs/1909.12579
5. He, Y., Liu, P., Wang, Z., Hu, Z., and Yang, Y., 2019, Filter pruning via geometric median for deep convolutional neural networks acceleration, In CVPR, 2019. https://arxiv.org/abs/1811.00250, Code: https://github.com/he-y/filter-pruning-geometric-median
6. Wang, W., Fu, C., Guo, J., Cai, D., and He, X., 2019, COP: customized deep model compression via regularized correlation-based filter-level pruning, In IJCAI, 2019. https://arxiv.org/abs/1906.10337
7. Luo, J., Zhang, H., Zhou, H., Xie, C., Wu, J., and Lin, W., 2019, Thinet: Pruning CNN filters for a thinner net, TPAMI, 2019. https://ieeexplore.ieee.org/document/8416559
8. Molchanov, P., Tyree, S., Karras, T., Aila, T., and Kautz, J., 2017, Pruning convolutional neural networks for resource efficient inference, In ICLR, 2017. https://arxiv.org/abs/1611.06440
9. Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, Hans Peter Graf, 2017, Pruning Filters for Efficient ConvNets, In ICLR, 2017. https://arxiv.org/abs/1608.08710
10. Mingbao Lin, Rongrong Ji, Yan Wang, Yichen Zhang, Baochang Zhang, Yonghong Tian, Ling Shao, 2020, HRank: Filter pruning using high-rank feature map, in CVPR, 2020. https://arxiv.org/abs/2002.10179, Code: https://github.com/lmbxmu/HRank
11. Pavlo Molchanov, Arun Mallya, Stephen Tyree, Iuri Frosio, Jan Kautz, June 2019, Importance Estimation for Neural Network Pruning, in CVPR, 2019. https://arxiv.org/abs/1906.10771
12. Shaohui Lin, Rongrong Ji, Chenqian Yan, Baochang Zhang, Liujuan Cao, Qixiang Ye, Feiyue Huang, David Doermann, Mar 2019, Towards Optimal Structured CNN Pruning via Generative Adversarial Learning, https://arxiv.org/abs/1903.09291
13. Shaohui Lin, Rongrong Ji, Yuchao Li, Yongjian Wu, Feiyue Huang, and Baochang Zhang. 2018, Accelerating convolutional networks via global & dynamic filter pruning, In IJCAI, pages 2425–2432, 2018. https://typeset.io/papers/accelerating-convolutional-networks-via-global-dynamic-38cal1vpjb
14. Lucas Liebenwein, Cenk Baykal, Harry Lang, Dan Feldman, Daniela Rus, Mar 2020, Provable Filter Pruning for Efficient Neural Networks, in ICLR, 2020 https://arxiv.org/abs/1911.07412
15. B Wang, C Ma, B Liu, N Liu, J Zhu, Oct 2023, Filter Pruning For CNN With Enhanced Linear Representation Redundancy, arXiv preprint arXiv:2310.06344, https://arxiv.org/pdf/2310.06344.pdf

For additional research papers on filter pruning, see https://www.aussieai.com/research/width-pruning#filter.

• Next: • Up: Table of Contents • Buy: Generative AI in C++: Coding Transformers and LLMs

The new AI programming book by Aussie AI co-founders: AI coding in C++ Transformer engine speedups LLM models Phone and desktop AI Code examples Research citations Get your copy from Amazon: Generative AI in C++