Reproducibility and reusability in deep reinforcement learning

Reinforcement learning (RL) has been shown to be an effective mechanism for learning complex tasks via interaction with the environment.Recent advances in combining deep neural networks with RL have resulted in powerful tools that outperform previous stateof-the-art methods for many domains including: robotics, video games, and board games.However, due to the interactive nature of these algorithms, as well as both intrinsic and extrinsic stochasticity, learning performance can be highly variant and difficult to reproduce.Furthermore, reusing information between tasks using these techniques can be problematic since they may overfit to a single task or environment.In this thesis, we investigate both reproducibility and reusability in deep RL.We begin by demonstrating the difficulty in reproducing a subset of deep RL algorithms: policy gradient methods for continuous control.We propose guidelines for rigorous experimental methodology and several statistical methods to help prevent misleading results.Next, we provide open-source reproducible environments for multitask RL.We evaluate simple sequential learning on sets of these tasks to show their effectiveness as benchmarks for multitask learning.Finally, we leverage these benchmark environments to investigate the notion of reusability.We focus on one-shot transfer learning in inverse RL.That is, given expert demonstrations from a mixture of environments with different dynamics is it possible to learn to properly complete a task in a previously unseen environment with different dynamics.To do this, we extend the options framework with the notion of reward options and develop a method for learning join reward-policy options in the context of generative adversarial inverse RL.This method is able to reuse information from a mixture of different environments to successfully learn a task in its current environment and significantly outperforms inverse RL without options.i This thesis could not have been completed without the support, advice, and generosity of many people.Both of my supervisors, David Meger and Joelle Pineau, always go above and beyond in their support, constantly finding time and ways to help when I thought it would not be possible.They are role models for both their students and what supervisors should be, and I am extremely grateful for this.David Meger has been an amazing supervisor in encouraging me to pursue interesting problems, always being around for helpful and enlightening discussions, and understanding of an unusual path to the completion of this thesis.Equally, Joelle Pineau is a fantastic supervisor who is always there for her students no matter what, is a constant source of wise advice and steady support, and has helped me become a better researcher than I thought possible.I would also like to thank all my co-authors on publications stemming from the ideas in this thesis.The discussions, encouragement, and help from all co-authors made these impactful ideas possible.A special thank you to my family for inspiration and support throughout this experience.In particular, my mother, Julia, is a source of inspiration for overcoming adversity against all odds and I am

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.