HALOC: Hardware-Aware Automatic Low-Rank Compression for Compact Neural Networks
arxiv(2023)
摘要
Low-rank compression is an important model compression strategy for obtaining
compact neural network models. In general, because the rank values directly
determine the model complexity and model accuracy, proper selection of
layer-wise rank is very critical and desired. To date, though many low-rank
compression approaches, either selecting the ranks in a manual or automatic
way, have been proposed, they suffer from costly manual trials or unsatisfied
compression performance. In addition, all of the existing works are not
designed in a hardware-aware way, limiting the practical performance of the
compressed models on real-world hardware platforms.
To address these challenges, in this paper we propose HALOC, a hardware-aware
automatic low-rank compression framework. By interpreting automatic rank
selection from an architecture search perspective, we develop an end-to-end
solution to determine the suitable layer-wise ranks in a differentiable and
hardware-aware way. We further propose design principles and mitigation
strategy to efficiently explore the rank space and reduce the potential
interference problem.
Experimental results on different datasets and hardware platforms demonstrate
the effectiveness of our proposed approach. On CIFAR-10 dataset, HALOC enables
0.07
models with 72.20
HALOC achieves 0.9
with 66.16
than the state-of-the-art automatic low-rank compression solution with fewer
computational and memory costs. In addition, HALOC demonstrates the practical
speedups on different hardware platforms, verified by the measurement results
on desktop GPU, embedded GPU and ASIC accelerator.
更多查看译文
关键词
compression,compact neural
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要