Identifying and Analyzing Provocative Text: An XAI Approach to Classification and Feature Selection

In this work, several AI models and methods are compared to see whichis best at classifying provocative texts in order to understand which features, such as specific words or kinds of words, are most influential in determining whether a piece of text is classified as provocative. This was done with two different categories of models: the more classical supervised learning models (e.g. RandomForests and Support Vector Machines (SVM) and the more novel transformer based classifiers. After performing a grid search and threshold tuning, the best perfoming classical model was an SVM, which achieved a minority-class F1 of 0.69 on a balanced test set, while the Bayesian XLNet was the best performing transformer, peaking at only 0.28 minority-F1 and 0.61 macro-F1 on skewed data. To understand the underwhelming performance of the models, the uncertainty was using the Bayesian XLNet model and found heteroscedastic aleatoric uncertainty in the data. This means that the task itself has inherent uncertainties. This uncertainty coupled with the skewed dataset explains the struggle of the models well. The models are then used to calculate the SHAP (SHapley Additive exPlanations) values of words to understand what impact individual words have on the classification. While some unexpected terms were found, like get, most words that indicate provocativeness are more explicit, such as stupid. These results are likely due to the underperformance of the models, which were unable to capture more nuanced forms of provocation. Together, the uncertainty and SHAP analyses demonstrate that provocativeness is anuanced, context-dependent phenomenon — data scarcity and label ambiguity limit performance, and no model can fully overcome that innatetask uncertainty. Note: This research paper presents some texts that might be perceived as offensive. Please note that these are only included when necessary for academic reasons and visualisations, and are by no means intended to hurt or offend readers.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.