EvoSh: Evolutionary Search with Shaving to Enable Power-Latency Tradeoff in Deep Learning Computing on Embedded Systems

Deploying deep-learning applications on resource-constrained embedded systems requires exploration of large design spaces for mapping (to heterogeneous processing units) and hardware configuration selection (voltage/frequency levels) to balance power consumption and latency. Herein, we propose a search algorithm called Evolution with shaving to find optimized configurations in these search spaces. We evaluate our approach using 3 state-of-the-art image classification DNNs on Nvidia Jetson-TX2 platform, demonstrating the benefit of exploring the proposed search spaces and the ability of the proposed algorithm to successfully perform the search. We show that our approach achieves optimized mapping and hardware configuration in <0.1% of search-space exploration.