Iterative low-rank approximation based on the redundancy of each network layer

Fan Yang; Weirong Liu; Jie Liu; Chaorong Liu; Yanchun Mi; Haowen Song

doi:10.1117/12.2589425

27 January 2021 Iterative low-rank approximation based on the redundancy of each network layer

Fan Yang, Weirong Liu, Jie Liu, Chaorong Liu, Yanchun Mi, Haowen Song

Proceedings Volume 11720, Twelfth International Conference on Graphics and Image Processing (ICGIP 2020); 117202G (2021) https://doi.org/10.1117/12.2589425
Event: Twelfth International Conference on Graphics and Image Processing, 2020, Xi'an, China

Abstract

Low rank approximation is an effective method in deep neural network (DNN) compression. In view of the fact that the redundancy information content of different network layers is different, a novel iterative low-rank approximation method based on the redundancy of each network layer is proposed. By giving priority to the network layer with higher redundancy, the loss of intrinsic information in each network layer is expected to be reduced and the performance of the compressed model is improved. Experimental results show that the performance of compressed model obtained by this method is improved with a slight reduction in compression ratio. It can be concluded that the proposed method can better retain intrinsic information in the pre-training network.

Citation Download Citation

Fan Yang, Weirong Liu, Jie Liu, Chaorong Liu, Yanchun Mi, and Haowen Song "Iterative low-rank approximation based on the redundancy of each network layer", Proc. SPIE 11720, Twelfth International Conference on Graphics and Image Processing (ICGIP 2020), 117202G (27 January 2021); https://doi.org/10.1117/12.2589425

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available