Improving Efficiency and Accuracy for Training and Inference of Hardware-aware Machine Learning Systems

Deep Neural Networks (DNNs) have been adopted in many systems because of their higher classification accuracy. While progress in achieving large scale, highly accurate DNNs has been made, however, the huge time and energy requirement for both inference and training pose a challenge to any DNN implementation. The large DNN size causes significant energy and area due to massive memory accesses and computations. In this thesis, we propose a new DNN architecture, LightNN, which replaces the multiplications to one shift or a constrained number of shifts and adds. Thus, LightNN inference uses hardware-efficient shift-based operations, and needs fewer memory accesses due to the fewer bits to represent each weight. Our experiment using image datasets show that LightNNs can achieve much higher hardware efficiency with a slight accuracy loss. LightNNs constrain all the weights of DNNs to be a limited combination (denoted as k∈{1, 2}) of powers of 2, and by varying the k one can obtain a group of LightNNs with different levels of accuracy and energy cost. To provide even more design flexibility, the k for each convolutional filter can be optimally chosen instead of being fixed for every filter. In this thesis, we formulate the selection of k to be differentiable, and describe model training for determining k-based weights on a per-filter basis. Over 46 FPGA-design experiments involving eight configurations and four data sets reveal that lightweight neural networks with a flexible k value (dubbed FLightNNs) fully utilize the hardware resources on Field Programmable Gate Arrays (FPGAs), our experimental results show that FLightNNs …