Predicting Threats to Academic Integrity: A Text-Mining and Scenario Modeling Framework
Notice bibliographique
Résumé
Cleverly crafted words become the weapon of choice in the realm of adversarial stylometry, where authors don masks of deception, bending language to confound the very algorithms designed to unveil their true identity" -ChatGPT, May 2023 when asked to define adversarial stylometry.This thesis will combine text-mining and scenario modeling to identify and predict threats.The problem of academic integrity is significant in society today, given all of the technological advancements.This research focused explicitly on contract cheating to narrow the scope.Contract cheating can take place on an online platform where a customized paper can be purchased or arranged informally with family or a friend.A financial transaction is not always present.Because the work acquired through these means is original, it circumvents text-matching detection.Therefore, this presented an interesting problem to study, understand, and test.This interdisciplinary research demonstrated that text-mining and scenario modeling can predict future threats to academic integrity.The text-mining technique of topic modeling was utilized to identify weak signals, and two diverse, separate models were built to account for the abundance of varied information and the complexity of the problem.The similarities between the two models were triangulated through comparison using word embeddings.This aided with validation and allowed causal relationships to be more easily seen by reinforcing and introducing weak signals.Weak signals provide an opportunity or a threat in the business world.We focused on threats solely, as the common thread within the contract cheating literature was the lack of detection.The dominant weak signal of adversarial stylometry was discovered in the topic model through the measurement of distance.Adversarial stylometry refers to the deceptive manipulation of text to avoid authorship detection.This weak signal was applied, a definition and threat model were created, and the weakness was quantified based on a three-tier threat severity.As stories are often more effective as a catalyst for change than models or numbers, ChatGPT generated this output as narrative scenarios in the students and course instructors' voices.As a result, a methodological framework was established that combined explanatory and predictive components.Scholars can replicate the demonstrated methods to predict threats on similar or new problems.Moreover, practitioners can use this methodology as a tool i) to automate threat predictions over time to remain competitive and ii) strategically plan for the future of academic integrity.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction distillée sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Apprise à partir de 10 348 étiquettes directes de Codex et de 10 348 étiquettes directes de Gemma. Le mode candidate est l'union des têtes enseignantes seuillées; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont ni des étiquettes humaines ni des étiquettes directes de modèles de pointe.
Scores Codex et Gemma par catégorie
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,002 | 0,003 |
| Méta-épidémiologie (sens strict) | 0,000 | 0,000 |
| Méta-épidémiologie (sens large) | 0,000 | 0,000 |
| Bibliométrie | 0,000 | 0,001 |
| Études des sciences et des technologies | 0,001 | 0,000 |
| Communication savante | 0,000 | 0,000 |
| Science ouverte | 0,000 | 0,000 |
| Intégrité de la recherche | 0,006 | 0,011 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,000 | 0,000 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; les deux têtes enseignantes s’accordent sur ce qui est montré ici.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».