A Comparison of Sentiment Analysis Tools
Notice bibliographique
Résumé
Sentiment analysis (SA), an analytics technique that assesses the ââ¬Åtoneââ¬Â of text, has emerged as a viable alternative to help users decide what to read without analyzing the whole text. In this study, we compare two main SA techniques (lexicon-based and machine-learning) for analyzing sentiments in the context of consumer product reviews. Given that a noted gap in prior research has been the almost sole focus on short textual information that concerns specific contexts (e.g., tweets about the Winter Olympics), we examine the role of contextual factors. Specifically, we examine reviews that address a multitude of product/service contexts, and which vary significantly in length. \\ \\ To test the research model, we collected 625 consumer reviews that ranged in length from 10 to 551 words, and which concerned six goods belonging to the three product/service categories commonly cited in the literature: a) search goods: laptop computers and paper notebooks; b) experience goods: hotels and restaurants; and c) credence goods: car repair and multi-vitamins. To analyze the reviews, we used two tools: 1) VADER (Valence Aware Dictionary for sEntiment Reasoning); a parsimonious rule-based sentiment analysis tool that has been shown to be especially effective in analyzing social media posts, and 2) Google Cloud Natural Language API (henceforth called Google), which uses a machine-learning approach. To assess the effectiveness of the tools, we compared the sentiment scores generated by each tool to the star ratings that were provided by the original authors of the consumer reviews. Hence, the (absolute) difference between the SA scores and the star ratings served as our dependent measure. \\ \\ The results of an ANOVA indicated that the lexicon-based approach (VADER) to sentiment analysis outperforms machine-learning in almost all product contexts regardless of review length (average error M = 16.6% vs. M = 19.8%; F = 13.81, p < 0.01). The main effect of review length was also statistically significant (F = 14.04, p < 0.01), where the SA toolsââ¬â¢ accuracy was better for short and medium length reviews. Overall, both toolsââ¬â¢ average accuracy was highest for credence goods, followed by experience and then search goods (F = 5.82, p < 0.05). Overall, VADER demonstrated higher accuracy over Google for credence and search goods. Machine learning (Google) had a better accuracy only in the case of long reviews about experience goods (F = 5.64, p < 0.05). \\ \\ The results of this study provide evidence that there are differences in the accuracy of various SA tools, and that it is important to consider contextual factors when choosing a SA tool. Building on these results, we will examine whether SA can be a substitute or a complement to star ratings in consumer decision making. \\
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction distillée sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Apprise à partir de 10 348 étiquettes directes de Codex et de 10 348 étiquettes directes de Gemma. Le mode candidate est l'union des têtes enseignantes seuillées; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont ni des étiquettes humaines ni des étiquettes directes de modèles de pointe.
Scores Codex et Gemma par catégorie
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,002 | 0,000 |
| Méta-épidémiologie (sens strict) | 0,000 | 0,000 |
| Méta-épidémiologie (sens large) | 0,000 | 0,000 |
| Bibliométrie | 0,000 | 0,001 |
| Études des sciences et des technologies | 0,000 | 0,000 |
| Communication savante | 0,000 | 0,001 |
| Science ouverte | 0,000 | 0,000 |
| Intégrité de la recherche | 0,000 | 0,000 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,000 | 0,000 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule tête enseignante, pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».