Device Placement Optimization with Deep Reinforcement Learning

With the proliferation of machine learning, deep neural networks (DNNs) have become ubiquitous in various real-world applications, and their sizes have become increasingly massive. In order to train DNNs with hundreds of millions of parameters, it is customary to employ a cluster of accelerators, such as GPUs and TPUs, to expedite the training process. As such, there is a pressing need to coordinate the accelerators for efficient DNN training, which gives rise to the device placement problem. Recently, deep reinforcement learning (DRL) has been proposed as an approach to fully automated and effective device placement optimization. In this dissertation, we examine critical research problems concerning the utilization of DRL for discovering optimal device placements for training complex and large DNNs. We begin by reviewing some key concepts, unique properties, and principles of distributed machine learning and DRL. Then we present EAGLE to generate device placement for large deep neural networks. Specifically, EAGLE is an end-to-end framework that groups and places the operations of a deep neural network via two jointly trained policy networks. An advanced DRL algorithm proximal policy optimization (PPO) is applied to achieve better sample efficiency. In addition to EAGLE, we propose Mars, to further improve the granularity and generalizability of the DRL agent. Mars pre-trains a graph neural network with contrastive learning, to generate comprehensive embeddings for each operation. Then, the placer can easily decide the placement for each operation based on its embedding. Lastly, we study the device placement problem in a pipeline parallelism setting, where model parallelism and data parallelism are combined to speed up the DNN training. Specifically, we carefully redesigned the action space of the DRL agent for pipelined device placement, especially for the placement of multi-branching DNNs. We conducted extensive experiments on different DNN training workloads, and our experimental results demonstrated that our approach effectively achieves the highest speedup for various DNN training workloads.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.