Reinforcement Learning vs Supervised Learning: A tug of war to generate refactored code accurately

Automated source code refactoring, particularly extract method refactoring, is a crucial and frequently employed technique during software development. Despite its importance and frequent use by practitioners, current automated techniques face significant limitations such as the lack of automation. While machine learning-based approaches have shown promise in intelligent code refactoring, existing such approaches overlook code-specific sequence-level characteristics, including but not limited to compilability, syntactic correctness, and functional integrity. To address these challenges, we propose a novel reinforcement learning-based approach for fine-tuning and aligning code language models to perform automated, intelligent extract method refactoring on Java source code. Our approach fine-tunes sequence-to-sequence generative models and aligns them using the Proximal Policy Optimization (PPO) algorithm using code compilation and presence of the refactoring in the generated code as reward signals. Our experiments demonstrate that our approach significantly enhances the performance of language models in code refactoring. The supervised fine-tuned model, further aligned with PPO, surpasses traditional supervised fine-tuning by 11.96% and 16.45% in terms of BLEU and CodeBLEU scores, respectively. When subjected to a suite of 122 unit tests, the number of successful tests increased from 41 to 66 for the reinforcement learning aligned fine tuned Code-T5 model, highlighting the effectiveness of our approach in producing functionally correct refactorings. Our work paves the way for intelligent, automated code refactoring tools that can significantly reduce developers’ manual effort.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.