Sentiment Analysis with LLMs: Evaluating QLoRA Fine-Tuning, Instruction Strategies, and Prompt Sensitivity

In this study, we explore the performance of large language models (LLMs) and provide insights into the strengths and weaknesses of these models in sentiment analysis. In addition, we explore the impact of various instruction methods and fine-tuning techniques on the models' performance and analyze how sensitive they are to different prompts. Although there are studies comparing LLMs on sentiment analysis tasks, there is a limited number of studies on fine-tuning strategies, instruction variety, and prompt sensitivity of LLMs for sentiment analysis. For this purpose, we used Llama2-7B, Lama3-8B, Mistral-7B, Gemma-7B, and Bloom-7B and compared them with BERT and Roberta. We evaluated fine-tuned and original chat versions with zero and few-shot experiments. We generated various instruction strategies and observed their effects on the performance of the models. We also compared Quantized Low-Rank Adapters (QLoRA) fine-tuning and full fine-tuning with all parameters. The fine-tuned versions performed better compared to the original chat versions on almost all tasks. The fine-tuned and chat versions of Llama3-8B and Mistral-7B outperformed others on these tasks, while the Llama2-7B-chat model often produced invalid responses. Our research indicates that QLoRA is a practical substitute for full fine-tuning, as it lessens computational demands without sacrificing performance. Fine-tuned models were less affected by variations in test prompts compared to chat models. This is likely because their specialized training allows them to better handle queries within their domain. In this exploratory study, we provide insights into the potential of LLMs in sentiment analysis tasks.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.