Optimizing Reinforcement Learning: Fog and Edge Resource Management Through Bootstrapping and Reward Shaping

The rapid and extensive use of technology is unprecedented. From small devices like sensors and mobile phones to large systems like servers and data centers, a wide range of computing setups exists to meet human needs. However, the increased demand has raised concerns about whether these setups can handle the load. Fog and edge computing are concepts that bring servers closer to users to improve response time and service quality. But the availability of these fog devices is limited, highlighting the need for systems to manage computing resources. These systems' main task is to efficiently distribute services across available resources and adapt to changing needs. Existing resource management solutions still possess challenges and limitations with regards to the quality of decisions due to their increasing complexity. \n \nIn this thesis, our main motivation is leveraging AI in addressing resource management, which stems from its ability to intelligently handle complex and dynamic scenarios, such as optimizing service placement, predicting demands, and adapting to changing environments. AI's capacity to learn from data and make informed decisions offers a promising approach to efficiently manage computing resources in a rapidly evolving technological landscape. This research is motivated by four main goals: (1) creating a strong computing architecture that can meet diverse user needs across various applications managed by a resource management system; (2) using AI to develop resource management solutions that handle decisions like placement and scaling, as well as predict user demands and resource availability; (3) ensuring the AI solution is reliable despite potential errors by improving its performance or having a backup plan; (4) making the AI solution adaptable to sudden environmental changes to keep decisions effective. \n \nThe thesis aims to address these gaps by: (1) designing an effective networking and computing architecture in the context of on-demand fog and edge formation, while supporting an Intelligent Computing Resource Management solution (ICRM) for multi-types of applications through offline learning and bootstrapping; (2) using DRL to build the ICRM, driven by a Markov Decision Process (MDP) environment design that produces actions related to host selection and service placement while accounting for the change in user demands; (3) enhancing the proposed MDP by adding the support for predicting the change in both user demands and available computing resources, where the agent becomes capable for issues horizontal and vertical resource scaling decisions in multi-applications setting; (4) introducing the first solution to speed the learning speed of DRL agent by devising a Graph Convolutional Network solution as a potential-based reward shaping solution; (5) developing another reward shaping solution based on Convolutional Neural Network (CNN) carefully designed and inspired by the value iteration network (VIN), to speed learning. Besides these contributions, we present a set of experimental studies and simulations using real-world test cases for each of the contributions compared to state-of-the-art solutions. \n \nIn conclusion, this thesis identifies research gaps that warrant further exploration in the future.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.