Linguistic Markers of AI-Generated Text: A Comparative Analysis of Machine-Identified and Human-Inferred Predictors
Pourquoi ce travail est dans la base
Une base qui oublie comment elle a trouvé un travail ne peut pas être vérifiée. Voici les voies qui ont admis celui-ci.
Notice bibliographique
Résumé
The widespread use of generative AI tools has significantly changed academic and professional writing, due to their ability to produce texts that mimic human writing styles. As a result, there are growing concerns about academic integrity, authorship, and the possible spread of misinformation. This study addresses the challenge of finding clear language features that can distinguish AI-generated texts from human-written ones, which is a gap that current detection tools have not resolved. Prior work shows that AI can produce coherent and context-relevant text by learning from large data sets and that features such as readability, lexical diversity, perplexity and burstiness, and sentiment are useful in detection, though results have been mixed. Our main goal is to determine which of these language features best predict AI authorship and to compare these machine-identified signals with the cues that human reviewers use. We analyze 100 mental health abstracts from 2022, published before the release of ChatGPT from OpenAI, and generate 100 additional abstracts using ChatGPT. We use a quantitative approach, using natural language processing methods such as readability, analytic writing index, lexical diversity, including measures like the measure of textual lexical diversity and type-token ratio, perplexity, burstiness, sentiment, common word groups, term frequency-inverse document frequency scores, voice usage, punctuation, and tone. These measures are then used to train a machine learning model to pick out the top predictors of AI-generated content. In addition, we will conduct a survey of 200 participants (expected) from Toronto Metropolitan University to collect ratings on abstract quality and ask participants to identify if each abstract was written by a human or generated by AI, along with background and AI usage information. We expect our analysis to show that AI-generated abstracts tend to have lower lexical diversity, simpler sentence structures, and lower perplexity, and that human reviewers will struggle to correctly identify AI-generated abstracts, especially when the differences are subtle. The findings add to our existing knowledge of the key language features that signal AI authorship and support the creation of better detection tools that combine machine analysis with human insight, ultimately helping to protect academic integrity and guide ethical authorship.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Prédiction distillée sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Apprise à partir de 10 348 étiquettes directes de Codex et de 10 348 étiquettes directes de Gemma. Le mode candidate est l'union des têtes enseignantes seuillées; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont ni des étiquettes humaines ni des étiquettes directes de modèles de pointe.
Scores Codex et Gemma par catégorie
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,001 | 0,000 |
| Méta-épidémiologie (sens strict) | 0,000 | 0,000 |
| Méta-épidémiologie (sens large) | 0,000 | 0,000 |
| Bibliométrie | 0,000 | 0,001 |
| Études des sciences et des technologies | 0,000 | 0,000 |
| Communication savante | 0,000 | 0,000 |
| Science ouverte | 0,000 | 0,000 |
| Intégrité de la recherche | 0,000 | 0,000 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,000 | 0,000 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découle