Towards efficient deep neural network inference and training for ubiquitous AI

Machine learning has become increasingly popular in recent years. Due to the high accuracy and excellent scalability, deep neural networks have emerged as a fundamental element in machine learning-based applications, such as autonomous driving, augmented reality, natural language processing, etc. And to promote portability and flexibility, more applications are now targeted on resource-limited edge devices, such as smartphones or IoT devices.However, deploying DNN models on resource-limited devices is very challenging. A deep and large model usually requires a lot of computation resources and huge power consumption as well. As a result, the huge model size, intensive computation, and high power consumption will be the biggest challenges for the genuine democratization of AI. To solve this problem, my research mainly focuses on optimizing and improving AI systems from the perspective of model size, training and inference computation cost, execution latency, and energy efficiency. And my work includes algorithm optimization, architecture design, and the incorporation of emerging devices. In the first part of this dissertation, I will present an algorithm-architecture co-design for the hardware-based DNN accelerator using emerging technology for efficient DNN inference.Recent work demonstrated the promise of using resistive random access memory (ReRAM) as an emerging technology to perform inherently parallel analog domain in-situ matrix-vector multiplication. One key problem is the weights that are signed values. However, in a ReRAM crossbar, weights are stored as conductance of the crossbar cells, and the in-situ computation assumes all cells on each crossbar column are of the same sign. The current architectures either use two ReRAM crossbars for positive and negative weights, or add an offset to weights so that all values become positive. Neither solution is ideal: they either double the cost of crossbars, or incur extra offset circuity. To better address this problem, we propose FORMS, a fine-grained ReRAM-based DNN accelerator with algorithm/hardware co-design. Instead of trying to represent the positive/negative weights, our key design principle is to enforce exactly what is assumed in the in-situ computation---ensuring that all weights in the same column of a crossbar have the same sign. It naturally avoids the cost of an additional crossbar. Besides, we also design a zero-skipping logic, which is especially suitable for the fine-grained computation paradigm, and can effectively skip unnecessary computations, thereby accelerating the inference speed of DNNs. In the second part, we investigate the efficient training technique to enhance the adaptiveness of DNN applications.Recently, sparse training has emerged as a promising paradigm for efficient deep learning on edge devices. The current research mainly devotes efforts to reducing training costs by further increasing model sparsity. However, increasing sparsity is not always ideal since it will inevitably introduce severe accuracy degradation at an extremely high sparsity level. To mitigate this issue, we intend to explore other possible directions to effectively and efficiently reduce sparse training costs while preserving accuracy. To this end, we investigate two techniques, namely, layer freezing and data sieving. First, the layer freezing approach has shown its success in dense model training and fine-tuning, yet it has never been adopted in the sparse training domain. Nevertheless, the unique characteristics of sparse training may hinder the incorporation of layer-freezing techniques. Therefore, we analyze the feasibility and potentiality of using the layer freezing technique in sparse training and find it has the potential to save considerable training costs. Second, we propose a data-sieving method for dataset-efficient training, which further reduces training costs by ensuring only a partial dataset is used throughout the entire training process. We show that both techniques can be well incorporated into the sparse training algorithm to form a generic framework, which we dub SpFDE. Our extensive experiments demonstrate that SpFDE can significantly reduce training costs while preserving accuracy from three dimensions: weight sparsity, layer freezing, and dataset sieving. Model compression strategies, including weight quantization and pruning, are widely recognized as effective approaches to significantly reduce computation and memory intensities, and have been implemented in many DNNs on edge devices.However, most state-of-the-art works focus on ad-hoc optimizations, and there lacks a thorough study to comprehensively reveal the potentials and constraints of different edge devices when considering different compression strategies. In the third part of this dissertation, we qualitatively and quantitatively compare the energy efficiency of FPGA-based and mobile-based DNN executions using mobile GPU and provide a detailed analysis. Based on the observations obtained from the analysis, we propose a unified optimization framework using block-based pruning to reduce the weight storage and accelerate the inference speed on mobile devices and FPGAs, achieving high hardware performance and energy-efficiency gain while maintaining accuracy.--Author's abstract

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.